JP2019075758A - Optical distortion calibration device, optical distortion calibration method, and optical distortion calibration program - Google Patents

Abstract

To provide a device, a method, and a program capable of calibrating optical distortion occurring in an image captured by imaging means with higher accuracy.SOLUTION: An optical distortion calibration device 1 for calibrating optical distortion occurring in an image includes a pattern generation unit 4 that generates three or more patterns whose luminances in a certain direction change sinusoidally and in which the phases of the sine waves are different from each other, a distortion estimation unit 6 that calculates a first position coordinate in image display means of pixels constituting the image according to the luminance of the image of the three or more patterns imaged by imaging means and estimates the distortion by comparing second position coordinates on the pattern for the pixels, and a correction unit 7 that corrects the image captured by the imaging means according to the distortion estimated by the distortion estimation unit 6.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a technique for calibrating optical distortion that occurs in an image captured by an imaging unit.

  In general, optical distortion occurs in an image captured using a lens. One of the typical examples of this optical distortion is distortion. The distortion appears prominently in the wide-angle lens, and the image is distorted as it gets closer to the field of view. This phenomenon adversely affects image processing assuming perspective projection of an object.

  From such a point of view, a technology for correcting optical distortion occurring in an image has been devised conventionally, and Patent Documents 1 and 2 analyze a deformed image of a pattern obtained by imaging a checkered pattern. Accordingly, a technique for calibrating the optical distortion is disclosed.

JP 2008-140401 A JP, 2013-109416, A

  However, in the techniques disclosed in Patent Documents 1 and 2, if it is necessary to increase the number of measurement points in order to improve the accuracy of calibration, it becomes necessary to finely copy the pattern into an image, but sampling or quantization of digital images There is a problem that the error of the coordinates of the measurement point becomes large because the cross point of the pattern becomes vague due to.

  On the other hand, when the pattern is copied to a large image, although the above-mentioned coordinates can be obtained more accurately, the coordinates of the obtained measurement point are reduced, so that the accuracy of the calibration is lowered. For this reason, such a so-called trade-off problem exists in the above-mentioned technology.

  The present invention has been made to solve such a problem, and can overcome the above-mentioned trade-off problem and correct the optical distortion occurring in the image captured by the imaging means with higher accuracy. An object is to provide an apparatus, a method, and a program.

  In order to solve the above problems, the present invention is an optical distortion calibration device that calibrates an optical distortion that occurs in an image, wherein the luminance in a certain direction changes sinusoidally, and the phases of the sinusoidal waves are three or more different from each other. A pattern generation unit configured to generate a pattern of the image and display the image on the image display unit, and the image display unit of each pixel constituting the image according to the brightness of the image of the three or more patterns captured by the imaging unit. The image pickup according to the distortion estimation means for estimating the distortion and the distortion estimated by the distortion estimation means by calculating one position coordinate and comparing each pixel with a second position coordinate on the pattern An optical distortion correction device comprising correction means for correcting an image captured by the means.

  Further, in order to solve the above problems, the present invention is an optical distortion calibration method for calibrating an optical distortion occurring in an image, in which the luminance in a certain direction changes sinusoidally, and the phases of the sinusoidal waves mutually change. A first step of generating three or more different patterns; a second step of causing the image display means to sequentially display the patterns generated in the first step; and imaging the respective patterns by the imaging means; A third step of calculating a first position coordinate in the image display means of each pixel constituting the image according to the brightness of the image captured in the step; a first position coordinate and a pattern for each pixel; A fourth step of estimating the distortion by comparing with the second position coordinate on the upper side, and a fifth step of correcting the image captured by the imaging means in accordance with the distortion estimated in the fourth step Light with steps To provide a distortion calibration method.

  Further, in order to solve the above problems, the present invention is an optical distortion calibration program for causing a computer to calibrate an optical distortion generated in an image, wherein the program has a luminance in a certain direction sinusoidal to the computer. Changing the phase of the sine wave to generate three or more patterns different from one another, and displaying the patterns generated in the first procedure on the image display means in sequence, and imaging each pattern A third procedure for calculating the first position coordinates of each pixel constituting the image in the image display means according to the second procedure for imaging the image and the luminance of the image captured in the second procedure; According to the fourth procedure for estimating the distortion by comparing the first position coordinate and the second position coordinate on the pattern for each pixel, and the distortion estimated in the fourth procedure, Above imaging Providing optical distortion calibration program having a fifth procedure for correcting the image captured by the stage.

  According to the present invention, since the positions of many measurement points can be accurately determined, it is possible to more accurately calibrate the optical distortion that occurs in the image captured by the imaging unit.

It is a block diagram showing composition of optical distortion calibration device 1 concerning an embodiment of the invention. It is a flowchart which shows the optical distortion correction method which concerns on embodiment of this invention. It is a figure which shows the example of the pattern shown by step S1 of FIG. It is a block diagram which shows the structure of the optical distortion calibration apparatus 20 which concerns on the other embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the drawings, the same reference numerals indicate the same or corresponding parts.

  FIG. 1 is a block diagram showing a configuration of an optical distortion calibration device 1 according to an embodiment of the present invention. As shown in FIG. 1, the optical distortion correction device 1 includes an input / output node 2, a bus 3 connected to the input / output node 2, and a pattern generation unit 4 and a distortion estimation unit 6 respectively connected to the bus 3. The correction unit 7, the storage unit 8, and the operation unit 9 are provided.

  FIG. 2 is a flowchart showing an optical distortion calibration method according to the embodiment of the present invention. In the following, although the case of realizing the present optical distortion calibration method by the operation of the optical distortion calibration device 1 shown in FIG. 1 will be described, the present method is not limited to the use of the optical distortion calibration device 1 and can be widely applied. Needless to say.

  In step S1, in response to the user's operation of the operation unit 9, the pattern generation unit 4 generates three or more patterns in which the luminance in a certain direction changes in a sine wave shape and the phases of the sine waves are mutually different. Here, as an example of this pattern, four transverse waves displayed in the images i0 to i3 shown in FIGS. All of these four transverse waves are those in which the luminance in the horizontal direction in the figure changes in a sine wave shape, and the positions of the black vertical lines indicating portions with low luminance are mutually offset in the horizontal direction among the images i0 to i3. Thus, it can be seen that the phases of the sine waves are different from each other. The pattern generated by the pattern generation unit 4 is stored in the storage unit 8 via the bus 3.

  Next, in step S2, in response to the operation of the operation unit 9 by the user, the pattern generation unit 4 sequentially displays the pattern generated in step S1 on an image display means such as a display connected to the input / output node 2. Each pattern is imaged by an imaging unit such as a camera. Then, the image captured in this step is stored in the storage unit 8.

  The pattern generation unit 4 causes the image display means to display the three or more patterns so that the direction in which the luminance changes sinusoidally is the first direction defining the plane forming the image display means. The image display means is displayed such that the direction in which the luminance changes is a second direction different from the first direction which defines the plane forming the image display means. Here, as an example, the first direction may be horizontal and the second direction may be vertical, but the first and second directions may be different directions defining the plane, It is not limited to the above example.

  As a result, since device coordinates to be described later can be obtained in the first direction and the second direction, it is possible to accurately specify the positions of all pixels on the image display means formed of a two-dimensional plane.

  Next, in step S3, according to the luminance of the image captured in step S2, the distortion estimation unit 6 operates the operation unit 9 by the user to control the number of pixels in the image display unit of the pixels. One position coordinate (device coordinate) is calculated and stored in the storage unit 8.

Here, the device coordinates are calculated by the phase shift method. For example, when the four-step method is adopted, image coordinates (u, v) of the images i0 to i3 shown in FIGS. Image coordinates (u, v) where the luminance in) is / I ₀ (u, v), / I ₁ (u, v), / I ₂ (u, v), / I ₃ (u, v) The phase of the sine wave in the above can be obtained by the following equation (1). The device coordinates can be obtained from the phase obtained by the equation (1). The method of calculating the device coordinates will be described in detail later.

  Next, in step S4, the distortion estimation unit 6 compares the distortion of the above-mentioned device coordinates stored in the storage unit 8 with the second position coordinates (image coordinates) on the pattern for each pixel. presume. Here, in the present estimation, the coordinates (functions) of the distortion model are determined using the coordinates of all pixels, or a correspondence table of pixels that appear distorted at the same point and pixels that appear on the image display means without distortion is displayed. It is created. The function and the correspondence table obtained in this step are stored in the storage unit 8.

  Next, in step S5, the correction unit 7 corrects the image captured by the imaging unit according to the distortion estimated in step S4 by the operation of the operation unit 9 by the user. More specifically, the image with distortion is converted into an image without distortion using the function or the correspondence table obtained in step S4.

  The optical distortion calibration method according to the embodiment of the present invention is a method of calibrating the optical distortion of an image generated due to an optical component such as a lens used in the imaging unit, the correction in step S5 is It is executed for all the images captured by the imaging unit used in step S2. In the following, the method of calculating the device coordinates in the above will be described in detail in order.

[How to create a pattern]
In step S1 shown in FIG. 2, for example, a pattern is created by the following method.

The waveform reference phase δ _t at time t is expressed by the following equation (2).

In the above equation (2), t represents time, and in the case of the four-step method, it takes a value of 0, 1, 2, or 3. Further, T represents the time of one cycle of the wave, and takes a value of 4 in the case of the four-step method. The unit of time is the interval between the virtual time reference phase [delta] _t of the wave pattern is switched.

Here, using the above reference phase [delta] _t, the brightness P _u of points in transverse and longitudinal pattern respectively, by obtaining in following equation P _v (3), to create the wave pattern.

In the above equation (3), u _p and v _p are the device coordinates in the horizontal and vertical directions of the pattern point, respectively, and L _u and L _v are the length of one period in the horizontal and vertical directions, respectively, A _p represents the brightness amplitude, and B _p represents the brightness bias component.

[About transformation from phase to device coordinates]
The brightness I _{u of the} transverse wave observed by the camera is expressed by the following equation (4).

In the above equation (4), u and v are image coordinates in the horizontal and vertical directions of an image (camera image) captured by the camera, φ (u, v) is the phase, and A is the brightness of the image. Amplitude, B indicates a bias component of brightness. Incidentally, the phase difference from the phase φ (u, v) [radian] reference phase [delta] _t, taking the value of π from - [pi].

Here, for the pixel whose coordinates are (u, v), the brightness P _u of the shear wave (see equation (3)) and the brightness I _{u of the} shear wave observed by the camera, as the created image is displayed Comparing (see equation (4)), the following equation (5) holds.

Therefore, when the phase φ (u, v) is obtained, the device coordinates u _p in the created transverse wave can be obtained from the following equation (6). The method of obtaining the phase φ (u, v) will be described later.

Similarly, in the case of the longitudinal wave, the device coordinates v _p in the created longitudinal wave can be obtained from the following equation (7).

[How to determine the phase]
The brightness I _t (u, v) of the wave at time t can be expressed as the following equation (8).

Here, the actual measurement values / I _t to the true value I _t of the brightness. Further, time t is from 0 to (N-1), and the reference phase is changed in N steps. Then, from the error / I _t , a probable phase φ is obtained by the least square method. That is, by determining a, b, and c as the square sum E of the difference between the measured value / I _t and the true value I _t represented by the following formula (9) is minimized, obtaining the phase phi.

  Then, the phase φ is obtained as in the following equation (10) by such a method.

  Further, the amplitude A and the bias component B are respectively obtained as in the following equation (11).

Since the above solution does not depend on the direction of the wave, the brightness I _t (u, v) may be either a transverse wave or a longitudinal wave.

[Extension to N-step method]
In the case of changing the reference phase N steps at equal intervals between 0 and 2π, the reference phase δ shown in the equation (2) can be expressed by the following equation (12).

  At this time, a solution with which the sum of squares E shown in the equation (9) is minimized is shown by the following equation (13).

  Therefore, the phase φ can be obtained by the following equation (14) by the equation (10).

  When N is 4 in the equation (12) and the reference phase δ is changed in four steps from 0 to π / 2 (90 degrees) from 0, the solution shown in the equation (13) is the following equation (15) .

  Therefore, the phase φ can be obtained as shown in equation (1) according to equation (10).

  In the optical distortion calibration method according to the embodiment of the present invention described above, the procedure shown in FIG. 2 is described as a computer-executable program, and the memory 22 of the optical distortion calibration device 20 as shown in FIG. It can be realized by storing the program and causing the central processing unit (CPU) 21 to execute the program.

  As described above, according to the optical distortion calibration device 1, 20, the optical distortion calibration method, or the optical distortion calibration program according to the embodiment of the present invention, the optical distortion generated in an image captured using an optical component such as a lens And can be calibrated more accurately than before. Further, the calibration according to the present invention is not limited to distortion but can be used for other optical distortions.

1, 20 optical distortion correction device 4 pattern generation unit 6 distortion estimation unit 7 correction unit 21 CPU

Claims (12)

An optical distortion calibration apparatus for calibrating optical distortion occurring in an image, comprising:
Pattern generation means for generating three or more patterns in which the luminance in a certain direction changes sinusoidally and the phases of the sine waves are mutually different; and
A first position coordinate in the image display means of each pixel constituting the image is calculated according to the brightness of the image of the three or more patterns imaged by the imaging means, and the pattern for each pixel is calculated. Distortion estimation means for estimating the distortion by comparison with a second position coordinate on the upper side;
An optical distortion correction device comprising: correction means for correcting an image captured by the imaging means according to the distortion estimated by the distortion estimation means.   The pattern generation means causes the image display means to display the three or more patterns such that the direction is a first direction defining a plane forming the image display means, and the three or more patterns are displayed. The optical distortion correction device according to claim 1, wherein the image display means is displayed such that the direction is a second direction different from the first direction which defines a plane forming the image display means.   The optical distortion calibration device according to claim 1, wherein the distortion estimation unit calculates a phase of each of the pixels, and calculates the first position coordinate according to the phase.   The optical distortion calibration device according to claim 1, wherein the distortion estimation unit estimates the distortion by obtaining a function or a correspondence table between the first position coordinate and the second position coordinate. An optical distortion calibration method for calibrating an optical distortion occurring in an image, comprising:
A first step of generating three or more patterns in which the luminance in a certain direction changes sinusoidally and the phases of the sine waves are mutually different;
A second step of causing the image display means to sequentially display the patterns generated in the first step, and imaging each pattern by the imaging means;
A third step of calculating first position coordinates of the pixels constituting the image in the image display means according to the luminance of the image captured in the second step;
A fourth step of estimating the distortion by comparing the first position coordinate and the second position coordinate on the pattern for each pixel;
5. An optical distortion calibration method comprising a fifth step of correcting an image captured by the imaging unit according to the distortion estimated in the fourth step. The second step is
Displaying the three or more patterns on the image display means such that the direction is a first direction defining a plane forming the image display means;
The method according to claim 1, further comprising the step of displaying the three or more patterns on the image display means so as to be a second direction different from the first direction in which the direction defines the plane forming the image display means. The optical distortion calibration method as described in 5. The third step is
Calculating the phase of each pixel;
The optical distortion calibration method according to claim 5, further comprising: calculating the first position coordinate according to the phase.   The optical distortion calibration method according to claim 5, wherein in the fourth step, the distortion is estimated by obtaining a function or a correspondence table between the first position coordinate and the second position coordinate. . An optical distortion calibration program for causing a computer to calibrate optical distortion occurring in an image, the program comprising:
A first procedure for generating three or more patterns in which the luminance in a certain direction changes sinusoidally and the phases of the sine waves are mutually different;
A second step of causing the image display means to sequentially display the patterns generated in the first step, and imaging each pattern on the image pickup means;
A third step of calculating first position coordinates of the pixels constituting the image in the image display means according to the luminance of the image captured in the second step;
A fourth procedure for estimating the distortion by comparing the first position coordinate and the second position coordinate on the pattern for each pixel;
An optical distortion calibration program comprising a fifth procedure for correcting an image captured by the imaging means according to the distortion estimated in the fourth procedure. The second procedure is
A procedure for causing the image display means to display the three or more patterns such that the direction is a first direction defining a plane forming the image display means;
The method according to claim 1, further comprising the step of displaying the three or more patterns on the image display means so as to be displayed in a second direction different from the first direction in which the direction defines the plane forming the image display means. The optical distortion calibration program described in 9. The third step is
A procedure for calculating the phase of each pixel;
The optical distortion calibration program according to claim 9, further comprising: a step of calculating the first position coordinate according to the phase. The optical distortion calibration program according to claim 9, wherein the fourth procedure estimates the distortion by obtaining a function or a correspondence table between the first position coordinate and the second position coordinate. .

Images