JP4613465B2 - Image conversion device and imaging and / or display device incorporating the same - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image conversion apparatus that converts an image having a so-called barrel distortion, such as an image obtained by an imaging apparatus using a wide-angle lens, into an image that is easy to visually recognize, and an imaging and / or display apparatus incorporating the image conversion apparatus. . INDUSTRIAL APPLICABILITY The present invention is useful for image processing and situation grasp in a moving body such as an automobile, a ship, and an aircraft, and a wide range monitoring in a facility.
[0002]
[Prior art]
In order to analyze the distortion caused by the imaging device, for example, a method is known in which a calibration pattern as shown in FIG. 11 is imaged and the distortion is analyzed based on which coordinate on each grid point has been converted. As this method, for example, a method described in Japanese Patent Laid-Open No. 2000-350239 is known.
[0003]
[Problems to be solved by the invention]
A converted image that has been “almost completely” free of distortion is not necessarily an image that is easily recognized by a viewer. Actually, it is assumed that images such as (a) and (b) in FIG. 12 are obtained as a rectangular image of 640 × 480 pixels, for example, as an input image by an imaging device using a wide-angle lens. Here, FIG. 12A shows the right side, front, and left side of three vehicles parked in the parking space. FIG. 12B shows the calibration pattern shown in FIG. 11 by the same imaging apparatus. When the image is converted to an image with almost no distortion by the above technique, the images are as shown in FIGS. 13A and 13B, respectively. The reason why the four sides that are the outer frame are curved is that the information in FIGS. 12A and 12B is used as much as possible. Further, in FIGS. 13A and 13B, the pixel is 1280 × 960. As can be seen from FIG. 13 (b), the distortion itself is certainly “almost completely”. However, from FIG. 13 (a), it can be seen that the vehicle seen in the periphery can be visually recognized. This is particularly because the sense of distance is viewed with a shift, and such a shift in sense of distance causes a problem in driving or maneuvering the moving body.
[0004]
Therefore, it is conceivable to extract only the image near the center of FIGS. 13A and 13B. However, when FIGS. 14A and 14B, which are extracted from 640 × 480 pixels, are viewed, the distortion is “almost completely”. Although it is not, it is clear that the amount of information is greatly reduced.
[0005]
The present invention has been made to solve such problems. That is, an object of the present invention is to provide an image conversion apparatus that outputs an image that can be easily recognized by a viewer.
[0006]
[Means for Solving the Problems]
In order to solve the above-described problem, according to the means of claim 1, in an image conversion apparatus that outputs an input image obtained by an imaging device using a wide-angle lens by performing coordinate conversion for each screen, the input of one screen is performed. The input image holding means for storing the image, the output image holding means for storing the output image of one screen, and the coordinates of the output image and the coordinates of the input image are associated with each other, thereby A coordinate conversion unit that forms data of an output image of one screen from the data, and the correspondence between the coordinates of the output image and the coordinates of the input image in the coordinate conversion unit is such that the coordinates of the output image are polar coordinates, the distance R ′, the angle When θ ′, the coordinates of the corresponding input image are polar coordinates with distance R and angle θ, and a calibration pattern for distortion correction provided on a plane perpendicular to the optical axis of the wide-angle lens is obtained by the imaging device. By analyzing in the image, it is completely eliminated converting the distortion of the input image,
_{R = R '+ k 1 R} ' 3 + k 2 R '5 + ... + k n R' 2n + 1
And θ = θ ′
A transformation formula that does not completely eliminate distortion when determined by
R = p (R ′ + k ′ ₁ R ′ ³ + k ′ ₂ R ′ ⁵ +... + K ′ _n R ′ ^{2n + 1} )
And θ = θ ′
However, p is a non-zero constant, and k ′ _i = 0 for i (1 ≦ i ≦ n) where k _i = 0.
For other i (1 ≦ i ≦ n), 1> k ′ ₁ / k ₁ > k ′ ₂ / k ₂ >…> k ′ _n / k _n > 0
It is characterized by comprising.
[0007]
Further, according to the means of claim 2, in claim 1, k ′ _i / k _i = a ⁱ , 0 <a <for all i (1 ≦ i ≦ n) where k _i ≠ 0. 1 is set.
[0008]
Claims 3 to 5 all have a common part. That is, in any image conversion apparatus that converts an input image by an imaging device using a wide-angle lens, an input image holding unit that stores an input image of one screen, an intermediate image holding unit that stores an intermediate image of one screen, By associating the output image holding means for storing the output image of one screen with the coordinates of the intermediate image and the coordinates of the input image, the intermediate image data of one screen is converted from the input image data of one screen of the input image holding means. The first coordinate conversion unit to be formed is associated with the coordinates of the output image and the coordinates of the intermediate image, thereby forming the data of the output image of one screen from the data of the intermediate image of one screen of the intermediate image holding unit. The coordinate conversion unit provided in the plane perpendicular to the optical axis of the wide-angle lens, in which the coordinates of the intermediate image and the input image in the first coordinate conversion unit are associated with each other. Is a first transformation that completely eliminates the distortion of the obtained intermediate image by imaging and analyzing the calibration pattern for the imaging device, and the coordinates of the intermediate image are expressed as distance R ′ and angle θ ′ in polar coordinates. When the coordinates of the corresponding input image are polar coordinates with distance R and angle θ,
R = p (R '+ k 1 R' 3 + k 2 R '5 + ... + k n R' 2n + 1)
And θ = θ ′
The correspondence between the coordinates of the output image and the coordinates of the intermediate image in the second coordinate transformation means is the coordinate of the output image in the coordinate system with the digital coordinates corresponding to the lens center point being (0,0). When (x ″, y ″) and the coordinates of the corresponding intermediate image are (x ′, y ′), each of the following conversions is performed.
[0009]
In claim 3,
x '= px''(1 + k 1''x''2 + k 2''x''4 + ... + k m''x''2m)
And y '= qy''
In claim 4,
x '= px''and
y '= qy''(1 + k 1''y''2 + k 2''y''4 + ... + k m''y''2m)
In claim 5,
x '= px''(1 + k 1''x''2 + k 2''x''4 + ... + k m''x''2m)
as well as
y '= qy''(1 + k 1''y''2 + k 2''y''4 + ... + k m''y''2m)
It goes without saying that pq ≠ 0 in any claim.
[0010]
A sixth aspect of the present invention is an imaging device, a display device, or an imaging and display device incorporating the image conversion device according to any of the first to fifth aspects.
[0011]
[Operation and effect of the invention]
In the analysis from the calibration pattern of the imaging device having a wide-angle lens, the principle can be easily understood if the origin is on the optical axis (point on the image) in the input / output image for image conversion. The conclusion is the same regardless of whether xy coordinates or polar coordinates are used with the optical axis as the origin. If an imaging apparatus having a wide-angle lens has C∞ point symmetry about the optical axis, a straight line perpendicular to the optical axis becomes a straight line passing through the origin on the input image by the imaging apparatus. The position on the input image is determined only by the half angle of view from the optical axis. That is, as shown in FIG. 1, the position on the input image at a point L away from the optical axis at a distance Z from the lens depends on the half angle of view φ = Arctan (L / Z).
[0012]
Then, the coordinates (x, y) and polar coordinates (R, θ) (where x = Rcos θ, y = R sin θ) on the input image having distortion on the focal length f are converted into the coordinates (x ′, y ′) and polar coordinates (R ′, θ ′) (where x ′ = R′cos θ ′, y ′ = R′sin θ ′) conversion depends only on the distance R from the origin in polar coordinate display. From well-known simple considerations, such a transformation can be represented by an odd function of R. Conversely, even if conversion from the coordinates of the output image is considered, it can be placed with an odd function of R ′.
[0013]
this
_{R = R '+ k 1 R} ' 3 + k 2 R '5 + ... + k n R' 2n + 1
And θ = θ ′
, K _n which are not 0 among the coefficients k ₁ , k ₂ ,..., K _n of the respective terms, all have the same sign and a small absolute value, and the coefficient of the higher-order term of R ′ is the absolute value Consider a conversion with a smaller size. It can be easily understood that the transformation is a transformation that leaves a distortion in the peripheral portion as compared with a transformation having “almost” no distortion. That is, when the distance R ′ from the origin of the output image is large, the coefficient of the higher-order term of R ′ that should be more greatly affected is a smaller absolute value, so that the distortion to be corrected is completely corrected. Instead, the strain remains relaxed. By appropriately selecting the coefficient of this new conversion, the loss of sense of distance due to the complete elimination of the distortion can be balanced between the sense of distance and the sense of distance (claim 1).
[0014]
In claim 1, for all i where k _i ≠ 0 (1 ≦ i ≦ n), if k ′ _i / k _i = a ⁱ and 0 <a <1, the coefficient can be easily set. it can. In this case, since all the coefficients k _i can be manipulated with only one variable a, the design of the image conversion apparatus is facilitated (claim 2).
[0015]
According to the third to fifth aspects of the present invention, an input image having distortion is once converted into an intermediate image having almost no distortion, and then converted into an output image having gentle distortion in the x-axis direction and / or the y-axis direction. is there. While the input images are point symmetric distortions, these output images will have moderate distortions that are not point symmetric. Since this distortion can be set arbitrarily, various distortions can be given depending on the object to be imaged by the imaging apparatus. A straight line parallel to the x-axis and the y-axis is stored for the intermediate image that has been almost free of distortion (claims 3 to 5).
[0016]
As described above, the image conversion apparatus that gives a gentle distortion compared to an image that has completely lost the distortion can eliminate the loss of sense of distance by setting the coefficient, and can provide an output image with good visibility. . Such an image conversion device may be incorporated into an imaging device, a display device, or an imaging and display device in which the imaging device and the display device are integrated (claim 6).
[0017]
DETAILED DESCRIPTION OF THE INVENTION
[First embodiment]
FIG. 2 is a configuration diagram illustrating the image conversion apparatus 100 according to the first embodiment of the present application, the imaging apparatus 2 having a wide-angle lens, and the display apparatus 3. The image conversion apparatus 100 includes an input memory 10 that holds one-screen image data input from the imaging device 2 having a wide-angle lens, a coordinate converter 11, a coordinate conversion table 12, and one-screen image data output to the display device 3. And an output memory 13 for holding The coordinate converter 11 is composed of a CPU according to the flowchart shown in FIG. This embodiment is for a monochrome image.
[0018]
The operation of the coordinate converter 11 of the image conversion apparatus 100 is shown in the flowchart of FIG. Hereinafter, S11 and the like indicate step 11. The flowchart of FIG. 3 starts after monochrome image data of one screen is read into the input memory 10. First, in S11, the initial coordinates (x ′, y ′) of the output pixel are read. The initial coordinates are arbitrary, but are, for example, the smallest coordinates (x ′, y ′) of one screen. Next, in S12, input coordinates (x, y) (not necessarily corresponding to each pixel) corresponding to the output pixel (coordinates (x ′, y ′)) are read from the coordinate conversion table 12.
[0019]
Next, in S13, it is determined whether or not there is an input coordinate (x, y) corresponding to the output coordinate (x ', y'). If it does not exist (if there is a corresponding point outside the input image), the process proceeds to S14, the luminance I of the input coordinates (x, y) is set equal to the preset I ₀ , and the process proceeds to S16. When the input coordinates (x, y) corresponding to the output coordinates (x ′, y ′) exist, the luminance I of the input coordinates (x, y) is calculated from the nearest pixel. For example, four luminances at four grid points surrounding the input coordinates (x, y) are read from the input memory 10, the luminance I of the input coordinates (x, y) is obtained by weighted averaging, and the process proceeds to S16.
[0020]
Thus, when the luminance I of the input coordinates (x, y) corresponding to the output coordinates (x ′, y ′) is obtained, the luminance I is set as the luminance of the output coordinates (x ′, y ′) in S16. The process proceeds to S17. In S17, the presence of the coordinates of the next pixel of the output pixel (coordinates (x ′, y ′)) is determined, and if it exists, the process proceeds to S18. In S18, the coordinates (x ′, y ′) of the output pixel are updated to the coordinates of the next pixel, and the process returns to S12. If the next pixel of the output pixel (coordinates (x ′, y ′)) does not exist, that is, if one screen of the output image is completed, the process ends. Thereafter, the image data is output from the output memory 13 to the display device 3.
[0021]
The conversion using the actual image data and the conversion using the calibration pattern are shown in FIGS. First, from a 640 × 480 pixel input image (polar coordinates (R, θ)) in FIG. 12B, a completely distorted 1280 × 960 pixel image (polar coordinates (R ′, θ) in FIG. 13B). The conversion to ')) was confirmed as follows.
[Expression 1]
R = R '+ k ₁ R' ³ + k ₂ R ' ⁵ + k ₃ R' ⁷ + k ₄ R ' ⁹ + k ₅ R' ¹¹
θ = θ '
k ₁ = -1.125 × 10 ^-1
k ₂ = 1.436 × 10 ^-2
k ₃ = -1.224 × 10 ^-3
k ₄ = 5.729 × 10 ^-5
k ₅ = -1.106 × 10 ^-6
The units of R and R ′ are mm, R ² = x ² + y ² , R ′ ² = x ′ ² + y ′ ² .
[0022]
Therefore, the conversion from the input image (polar coordinates (R, θ)) to the image (polar coordinates (R ′, θ ′)) with a = 2/3, k ′ _i / k _i = a ⁱ ,
[Expression 2]
_{R = R '+ k' 1} R '3 + k' 2 R '5 + k' 3 R '7 + k' 4 R '9 + k' 5 R '11
θ = θ '
R ² = x ² + y ² , R ' ² = x' ² + y ' ²
It was. This relationship is prepared in the conversion table 12 for all output pixels. Images obtained by performing this conversion from the input images shown in FIGS. 12A and 12B are shown in FIGS. As shown in FIG. 4B, this conversion is a conversion that leaves a gentle distortion, but it does not lose the sense of distance as shown in FIG. Thus, the conversion is performed to output an image with good balance between the sense of distance and the distortion.
[0023]
[Second Embodiment]
FIG. 5 is a configuration diagram showing the image conversion apparatus 200 according to the second embodiment of the present application, the imaging apparatus 2 having a wide-angle lens, and the display apparatus 3. The image conversion apparatus 200 includes an input memory 20 that holds one-screen image data input from the imaging apparatus 2 having a wide-angle lens, a first coordinate converter 21, a first coordinate conversion table 22, an intermediate memory 23, and a second coordinate. It comprises a converter 24, a second coordinate conversion table 25, and an output memory 26 that holds one screen of image data to be output to the display device 3. The first coordinate converter 21 is composed of a CPU according to the flowchart shown in FIG. 6, and the second coordinate converter 24 is composed of a CPU according to the flowchart shown in FIG. This embodiment is also for a monochrome image.
[0024]
The first coordinate converter 21 that is the first half of the operation of the image conversion apparatus 200 is shown in the flowchart of FIG. S21 and the like indicate step 21. The flowchart of FIG. 6 starts after monochrome image data of one screen is read into the input memory 20. First, in S21, the initial coordinates (x ′, y ′) of the intermediate pixel are read. The initial coordinates are arbitrary, but are, for example, the smallest coordinates (x ′, y ′) of one screen. Next, in S 22, input coordinates (x, y) (not necessarily matching each pixel) corresponding to the intermediate pixel (coordinates (x ′, y ′)) are read from the first coordinate conversion table 22.
[0025]
Next, in S23, it is determined whether or not there is an input coordinate (x, y) corresponding to the intermediate coordinate (x ', y'). If it does not exist (when there is a corresponding point outside the input image), the process proceeds to S24, the luminance I of the input coordinates (x, y) is set equal to the preset I ₀ , and the process proceeds to S26. When the input coordinate (x, y) corresponding to the intermediate coordinate (x ′, y ′) exists, the luminance I of the input coordinate (x, y) is calculated from the nearest pixel. For example, four luminances at four grid points surrounding the input coordinates (x, y) are read from the input memory 20, the luminance I of the input coordinates (x, y) is obtained by weighted averaging, and the process proceeds to S26.
[0026]
Thus, when the luminance I of the input coordinates (x, y) corresponding to the intermediate coordinates (x ′, y ′) is obtained, the intermediate memory 23 determines that the luminance I is the luminance of the intermediate coordinates (x ′, y ′) in S26. The process proceeds to S27. In S27, the presence of the coordinates of the next pixel of the intermediate pixel (coordinates (x ′, y ′)) is determined, and if it exists, the process proceeds to S28. In S28, the coordinates (x ′, y ′) of the intermediate pixel are updated to the coordinates of the next pixel, and the process returns to S22. If the next pixel of the intermediate pixel (coordinates (x ′, y ′)) does not exist, that is, if one screen of the intermediate image is completed, the process ends.
[0027]
The flowchart of FIG. 7 starts after monochrome image data of one screen is written in the intermediate memory 23. First, in S31, the initial coordinates (x ″, y ″) of the output pixel are read. The initial coordinates are arbitrary, but are, for example, the smallest coordinates (x ″, y ″) of one screen. Next, in S <b> 32, intermediate coordinates (x ′, y ′) (not necessarily corresponding to each pixel) corresponding to the output pixel (coordinates (x ″, y ″)) are read from the second coordinate conversion table 25. .
[0028]
Next, in S33, it is determined whether or not there is an intermediate coordinate (x ′, y ′) corresponding to the output coordinate (x ″, y ″). If it does not exist (if there is a corresponding point outside the intermediate image), the process proceeds to S34, the brightness I ′ of the intermediate coordinates (x ′, y ′) is set equal to the preset I ₀ , and the process proceeds to S36. When there is an intermediate coordinate (x ′, y ′) corresponding to the output coordinate (x ″, y ″), the luminance I ′ of the intermediate coordinate (x ′, y ′) is calculated from the nearest pixel. For example, four luminance values of four grid points surrounding the intermediate coordinates (x ′, y ′) are read from the intermediate memory 23, the luminance I ′ of the intermediate coordinates (x ′, y ′) is obtained by weighted averaging, and the process proceeds to S36.
[0029]
Thus, when the brightness I ′ of the intermediate coordinates (x ′, y ′) corresponding to the output coordinates (x ″, y ″) is obtained, the brightness I ′ is converted to the output coordinates (x ″, y ′) in S36. The brightness of ') is output to the output memory 26, and the process proceeds to S37. In S37, the presence of the coordinates of the next pixel of the output pixel (coordinates (x ″, y ″)) is determined, and if it exists, the process proceeds to S38. In S38, the coordinates (x ″, y ″) of the output pixel are updated to the coordinates of the next pixel, and the process returns to S32. If the next pixel of the output pixel (coordinates (x ″, y ″)) does not exist, that is, if one screen of the output image is completed, the process ends. Thereafter, the image data is output from the output memory 26 to the display device 3.
[0030]
8A and 8B show conversion using actual image data and conversion using a calibration pattern. First, from an input image (polar coordinates (R, θ)) of 640 × 480 pixels in FIG. 12B, an intermediate image of 1280 × 960 pixels (polar coordinates (R ′, The conversion to θ ′)) was determined as follows, as in the first embodiment.
[Equation 3]
R = R '+ k ₁ R' ³ + k ₂ R ' ⁵ + k ₃ R' ⁷ + k ₄ R ' ⁹ + k ₅ R' ¹¹
θ = θ '
k ₁ = -1.125 × 10 ^-1
k ₂ = 1.436 × 10 ^-2
k ₃ = -1.224 × 10 ^-3
k ₄ = 5.729 × 10 ^-5
k ₅ = -1.106 × 10 ^-6
The units of R and R ′ are mm, R ² = x ² + y ² , R ′ ² = x ′ ² + y ′ ² . This relationship is prepared in the first coordinate conversion table 22 for all intermediate pixels.
[0031]
Therefore, conversion from the intermediate image (coordinates (x ', y'), x '= R'cosθ', y '= R'sinθ') to the output image (coordinates (x '', y '')) is performed. ,
[Expression 4]
x '= x''(1 + 2.0 × 10 ^-6 x'' ² + 2.0 × 10 ^-11 x'' ⁴ + 4.0 × 10 ^-16 x'' ⁶ )
y '= 1.5 y''
It was. This relationship is prepared in the second coordinate conversion table 25 for all output pixels. Images obtained by performing this conversion from the intermediate images shown in FIGS. 13A and 13B are shown in FIGS. As shown in FIG. 8 (b), this transformation is a transformation in which a gentle distortion is applied in the lateral direction. However, it does not lose the sense of distance as shown in FIG. As shown in (a), the conversion is performed to output an image with good balance between a sense of distance and distortion.
[0032]
[Third embodiment]
In the present embodiment, the contents of the second coordinate conversion table 25 are changed as follows in the configuration of the second embodiment shown in FIG. That is, conversion from the intermediate image (coordinates (x ′, y ′), x ′ = R′cosθ ′, y ′ = R′sinθ ′) to the output image (coordinates (x ″, y ″)) is performed. ,
[Equation 5]
x '= x''
y '= 1.5 y''(1 + 2.0 × 10 ^-6 y'' ² + 2.0 × 10 ^-11 y'' ⁴ + 4.0 × 10 ^-16 y'' ⁶ )
It was. Images obtained by performing this conversion from the intermediate images in FIGS. 13A and 13B are shown in FIGS. As shown in FIG. 9 (b), this transformation is a transformation in which a gentle distortion is applied in the vertical direction, but it does not lose the sense of distance as in FIG. As shown in (a), the conversion is performed to output an image with good balance between a sense of distance and distortion.
[0033]
[Fourth embodiment]
In the present embodiment, the contents of the second coordinate conversion table 25 are changed as follows in the configuration of the second embodiment shown in FIG. That is, conversion from the intermediate image (coordinates (x ′, y ′), x ′ = R′cosθ ′, y ′ = R′sinθ ′) to the output image (coordinates (x ″, y ″)) is performed. ,
[Formula 6]
x '= x''(1 + 2.0 × 10 ^-6 x'' ² + 2.0 × 10 ^-11 x'' ⁴ + 4.0 × 10 ^-16 x'' ⁶ )
y '= 1.5 y''(1 + 2.0 × 10 ^-6 y'' ² + 2.0 × 10 ^-11 y'' ⁴ + 4.0 × 10 ^-16 y'' ⁶ )
It was. Images obtained by performing this conversion from the intermediate images shown in FIGS. 13A and 13B are shown in FIGS. As shown in FIG. 10 (b), this conversion is a conversion in which gentle distortion is applied in the vertical and horizontal directions, but it does not lose the sense of distance as in FIG. 13 (a). As in (a) of 10, this is a conversion that outputs an image with good balance between the sense of distance and the distortion and with good visibility.
[0034]
In the above-described embodiment, it is realized by a computer system including a memory and a CPU, but it can also be realized by a logic circuit. In the second to fourth embodiments, the intermediate memory is provided and the two-step coordinate conversion is performed. However, the conversion can be performed by one-step coordinate conversion that can be performed by one CPU. It is also clear.
[0035]
The invention described in the claims and the invention described in the section of the embodiment of the present invention relate to an image conversion apparatus, but it is naturally recognized as a method invention, that is, an image conversion method. In other words, when the coordinates of the output image are polar coordinates with distance R ′ and angle θ ′, the corresponding coordinates of the input image are polar coordinates with distance R and angle θ, and the distortion is provided on a plane perpendicular to the optical axis of the wide-angle lens. Conversion that completely eliminates distortion of the input image by imaging and analyzing the calibration pattern for correction by the imaging device,
_{R = R '+ k 1 R} ' 3 + k 2 R '5 + ... + k n R' 2n + 1
And θ = θ ′
A transformation formula that does not completely eliminate distortion when determined by
R = p (R ′ + k ′ ₁ R ′ ³ + k ′ ₂ R ′ ⁵ +... + K ′ _n R ′ ^{2n + 1} )
And θ = θ ′
However, p is a non-zero constant, and k ′ _i = 0 for i (1 ≦ i ≦ n) where k _i = 0.
For other i (1 ≦ i ≦ n), 1> k ′ ₁ / k ₁ > k ′ ₂ / k ₂ >…> k ′ _n / k _n > 0
K ′ _i / k _i = a ⁱ , 0 <a <1 for all i (1 ≦ i ≦ n) for which k _i ≠ 0 Image conversion method.
[0036]
Similarly, a first calibration pattern that completely eliminates distortion of the intermediate image obtained by imaging and analyzing a calibration pattern for distortion correction provided on a plane perpendicular to the optical axis of the wide-angle lens by the imaging device. Conversion,
When the coordinates of the intermediate image are polar coordinates with distance R ′ and angle θ ′, the corresponding input image coordinates are polar coordinates with distance R and angle θ,
R = p (R '+ k 1 R' 3 + k 2 R '5 + ... + k n R' 2n + 1)
And θ = θ ′
Is a second transformation that gives distortion in the peripheral portion from the undistorted intermediate image, where the coordinates of the output image are (x '', y ''), and the coordinates of the corresponding intermediate image are ( x ′, y ′), the image conversion method according to any one of (1) to (3) below.
(1)
x '= px''(1 + k 1''x 2 + k 2''x 4 + ... + k m''x''2m)
And y '= qy''
(2)
x '= px''and
y '= qy''(1 + k 1''y 2 + k 2''y 4 + ... + k m''y''2m)
(3)
x '= px''(1 + k 1''x 2 + k 2''x 4 + ... + k m''x''2m)
as well as
y '= qy''(1 + k 1''y 2 + k 2''y 4 + ... + k m''y''2m)
[Brief description of the drawings]
FIG. 1 is a conceptual diagram showing image distortion.
FIG. 2 is a configuration diagram showing a combination of an image conversion apparatus, an imaging apparatus, and a display apparatus according to the first embodiment of the present invention.
FIG. 3 is a flowchart showing the operation of the image conversion apparatus according to the first embodiment.
FIG. 4 is an output image by the image conversion apparatus according to the first embodiment.
FIG. 5 is a configuration diagram showing a combination of an image conversion apparatus, an imaging apparatus, and a display apparatus according to a second embodiment of the present invention.
FIG. 6 is a flowchart showing the first half of the operation of the image conversion apparatus according to the second embodiment.
FIG. 7 is a flowchart showing the second half of the operation of the image conversion apparatus according to the second embodiment.
FIG. 8 is an output image by the image conversion apparatus according to the second embodiment of the present invention.
FIG. 9 is an output image by the image conversion apparatus according to the third embodiment of the present invention.
FIG. 10 is an output image by the image conversion apparatus according to the fourth embodiment of the present invention.
FIG. 11 is a diagram showing an example of a calibration pattern.
FIG. 12 is an input image by an imaging device having a wide-angle lens.
13 is an “almost no distortion” output image generated from the input image of FIG.
FIG. 14 is an image obtained by enlarging the central portion of the image of FIG. 13 twice.
100, 200 Image conversion device 2 Imaging device 3 having wide-angle lens Display device 10 Input memory 11 Coordinate converter 12 Coordinate conversion table 13 Output memory 20 Input memory 21 First coordinate converter 22 First coordinate conversion table 23 Intermediate memory 24 First Two-coordinate converter 25 Second coordinate conversion table 26 Output memory

Claims (6)

In an image conversion apparatus that outputs an input image by an imaging apparatus using a wide-angle lens by converting coordinates for each screen,
Input image holding means for storing an input image of one screen;
Output image holding means for storing an output image of one screen;
Coordinate conversion means for forming the output image data of one screen from the input image data of one screen of the input image holding means by associating the coordinates of the output image with the coordinates of the input image,
Correspondence between the coordinates of the output image and the coordinates of the input image in the coordinate conversion means,
When the coordinates of the output image are polar coordinates with distance R ′ and angle θ ′, the corresponding input image coordinates are polar coordinates with distance R and angle θ,
By converting and analyzing a calibration pattern for distortion correction provided on a plane perpendicular to the optical axis of the wide-angle lens by the imaging device, conversion that completely eliminates distortion of the input image,
_{R = R '+ k 1 R} ' 3 + k 2 R '5 + ... + k n R' 2n + 1
And θ = θ ′
A transformation formula that does not completely eliminate distortion when determined by
R = p (R ′ + k ′ ₁ R ′ ³ + k ′ ₂ R ′ ⁵ +... + K ′ _n R ′ ^{2n + 1} )
And θ = θ ′
However, p is a non-zero constant, and k ′ _i = 0 for i (1 ≦ i ≦ n) where k _i = 0.
For other i (1 ≦ i ≦ n), 1> k ′ ₁ / k ₁ > k ′ ₂ / k ₂ >…> k ′ _n / k _n > 0
An image conversion apparatus comprising: In claim 1, for all i (1 ≦ i ≦ n) where k _i ≠ 0,
k ′ _i / k _i = a ⁱ , 0 <a <1
An image conversion device. In an image conversion device that converts an input image by an imaging device using a wide-angle lens,
Input image holding means for storing an input image of one screen;
Intermediate image holding means for storing an intermediate image of one screen;
Output image holding means for storing an output image of one screen;
First coordinate conversion means for forming intermediate image data of one screen from input image data of one screen of the input image holding means by associating the coordinates of the intermediate image with the coordinates of the input image;
A second coordinate conversion unit that forms the output image data of one screen from the intermediate image data of the one screen of the intermediate image holding unit by associating the coordinates of the output image with the coordinates of the intermediate image;
Correspondence between the coordinates of the intermediate image and the coordinates of the input image in the first coordinate conversion means is
A first conversion that completely eliminates distortion of an intermediate image obtained by imaging and analyzing a calibration pattern for distortion correction provided on a plane perpendicular to the optical axis of the wide-angle lens by the imaging device. There,
When the coordinates of the intermediate image are polar coordinates with distance R ′ and angle θ ′, the corresponding input image coordinates are polar coordinates with distance R and angle θ,
R = p (R '+ k 1 R' 3 + k 2 R '5 + ... + k n R' 2n + 1)
And θ = θ ′
Determined by
The correspondence between the coordinates of the output image and the coordinates of the intermediate image in the second coordinate conversion means is as follows:
A second transformation that gives distortion in the peripheral part from an intermediate image without distortion,
The coordinate of the output image in the coordinate system in which the digital coordinate corresponding to the lens center point is (0, 0) is (x '', y ''), and the corresponding intermediate image coordinate is (x ', y'). When
x '= px''(1 + k 1''x''2 + k 2''x''4 + ... + k m''x''2m)
And y '= qy''
An image conversion apparatus comprising: In an image conversion device that converts an input image by an imaging device using a wide-angle lens,
Input image holding means for storing an input image of one screen;
Intermediate image holding means for storing an intermediate image of one screen;
Output image holding means for storing an output image of one screen;
First coordinate conversion means for forming intermediate image data of one screen from input image data of one screen of the input image holding means by associating the coordinates of the intermediate image with the coordinates of the input image;
A second coordinate conversion unit that forms the output image data of one screen from the intermediate image data of the one screen of the intermediate image holding unit by associating the coordinates of the output image with the coordinates of the intermediate image;
Correspondence between the coordinates of the intermediate image and the coordinates of the input image in the first coordinate conversion means is
A first conversion that completely eliminates distortion of an intermediate image obtained by imaging and analyzing a calibration pattern for distortion correction provided on a plane perpendicular to the optical axis of the wide-angle lens by the imaging device. There,
When the coordinates of the intermediate image are polar coordinates with distance R ′ and angle θ ′, the corresponding input image coordinates are polar coordinates with distance R and angle θ,
R = p (R '+ k 1 R' 3 + k 2 R '5 + ... + k n R' 2n + 1)
And θ = θ ′
Determined by
The correspondence between the coordinates of the output image and the coordinates of the intermediate image in the second coordinate conversion means is as follows:
A second transformation that gives distortion in the peripheral part from an intermediate image without distortion,
The coordinate of the output image in the coordinate system in which the digital coordinate corresponding to the lens center point is (0, 0) is (x '', y ''), and the corresponding intermediate image coordinate is (x ', y'). When
x '= px''and
y '= qy''(1 + k 1''y''2 + k 2''y''4 + ... + k m''y''2m)
An image conversion apparatus comprising: In an image conversion device that converts an input image by an imaging device using a wide-angle lens,
Input image holding means for storing an input image of one screen;
Intermediate image holding means for storing an intermediate image of one screen;
Output image holding means for storing an output image of one screen;
First coordinate conversion means for forming intermediate image data of one screen from input image data of one screen of the input image holding means by associating the coordinates of the intermediate image with the coordinates of the input image;
A second coordinate conversion unit that forms the output image data of one screen from the intermediate image data of the one screen of the intermediate image holding unit by associating the coordinates of the output image with the coordinates of the intermediate image;
Correspondence between the coordinates of the intermediate image and the coordinates of the input image in the first coordinate conversion means is
A first conversion that completely eliminates distortion of an intermediate image obtained by imaging and analyzing a calibration pattern for distortion correction provided on a plane perpendicular to the optical axis of the wide-angle lens by the imaging device. There,
When the coordinates of the intermediate image are polar coordinates with distance R ′ and angle θ ′, the corresponding input image coordinates are polar coordinates with distance R and angle θ,
R = p (R '+ k 1 R' 3 + k 2 R '5 + ... + k n R' 2n + 1)
And θ = θ ′
Determined by
The correspondence between the coordinates of the output image and the coordinates of the intermediate image in the second coordinate conversion means is as follows:
A second transformation that gives distortion in the peripheral part from an intermediate image without distortion,
The coordinate of the output image in the coordinate system in which the digital coordinate corresponding to the lens center point is (0, 0) is (x '', y ''), and the corresponding intermediate image coordinate is (x ', y'). When
x '= px''(1 + k 1''x''2 + k 2''x''4 + ... + k m''x''2m)
as well as
y '= qy''(1 + k 1''y''2 + k 2''y''4 + ... + k m''y''2m)
An image conversion apparatus comprising: An imaging and / or display device incorporating the image conversion device according to any one of claims 1 to 5.

Images