JP7321947B2 - Image correction device and image correction method - Google Patents

Description

The present invention relates to an image correction device and an image correction method for correcting distortion of images captured by one or more cameras.

Conventionally, as such a technical field, there is one described in Patent Document 1, for example. In the image correction device described in Patent Document 1, a planar chart on which a predetermined grid pattern is drawn is captured by a stereo camera with and without the windshield, and the grid on the captured image is captured. The coordinates of the pattern are obtained, and the image correction data is created based on the difference from the ideal coordinates. Furthermore, the geometric distortion of the image is removed based on the generated image correction data.

JP 2017-62198 A

However, the image correction apparatus described above has the following problems. That is, when the field of view of the camera is wide-angle, the size of the planar chart becomes long, and the area of the device including the chart becomes large. On the other hand, shortening the size of the chart by, for example, shortening the distance between the camera and the chart has been considered. Data for highly accurate image correction cannot be created. Therefore, the accuracy of the distance detected by the camera is lowered.

The present invention has been made to solve such technical problems, and provides an image correction device and an image correction method that can reduce the area of the device while maintaining the accuracy required for the distance detected by the camera. intended to

An image correction device according to the present invention includes a chart on which a plurality of markers are arranged, an image acquisition section that acquires an image captured by a camera that captures an image of the markers through a windshield, and an image acquired by the image acquisition section. and image correction data for creating data for correcting distortion of an image captured by the camera based on the marker positions detected by the marker position detection unit. and a creation unit, wherein the marker is arranged on a marker arrangement curve in which, when viewed from the imaging direction of the camera in a plan view, the allowable error of the detection distance for each camera viewing angle and the allowable error of the camera installation deviation match. Alternatively, it is characterized by being arranged behind the marker arrangement curve.

In the image correction device according to the present invention, when viewed from the imaging direction of the camera in a plan view, the marker has a marker arrangement curve in which the allowable error of the detection distance for each viewing angle of the camera and the allowable error of the misalignment of the camera match. placed above or behind the marker placement curve. As a result, the detection distance for each viewing angle can be detected with an allowable error or less, and the area of the chart can be reduced as compared with the case where the markers are provided on a plane. As a result, the area of the device can be reduced while maintaining the accuracy required for the distance detected by the camera.

According to the present invention, the area of the device can be reduced while maintaining the accuracy required for the distance detected by the camera.

1 is a plan view showing the configuration of an image correction device according to a first embodiment; FIG. It is a figure which shows the structure of a stereo camera. FIG. 4 is a diagram showing the shape of a marker; It is a figure which shows the image correction method. Fig. 10 shows ideal positions of markers on an undistorted image; It is a figure which shows the relationship between a viewing angle and detection distance. FIG. 4 is a diagram showing the relationship between the viewing angle and parallax tolerance; FIG. 4 is a diagram showing the relationship between the horizontal position with respect to the stereo camera and the distance from the stereo camera; It is a figure explaining the comparison of the image correction apparatus which concerns on 1st Embodiment, and the past. FIG. 11 is a plan view showing the configuration of a modification of the image correction device; It is a top view which shows the structure of the stereo camera of a modification, and a calculation part. It is a top view which shows the modification of an image correction apparatus. It is a perspective view which shows the modification of an image correction apparatus. It is a top view which shows the modification of an image correction apparatus. It is a perspective view which shows the modification of an image correction apparatus. It is a top view which shows the modification of an image correction apparatus. It is a figure which shows the marker pattern of the modification of an image correction apparatus. It is a figure which shows the marker pattern of the modification of an image correction apparatus. FIG. 9 is a plan view showing the configuration of an image correction device according to a second embodiment; It is a figure which shows the structure of a monocular camera. It is a figure which shows the image correction method. It is a figure which shows the situation which detects the vehicle and pedestrian on a correction|amendment image. FIG. 5 is a diagram showing the relationship between viewing angle and vertical positional displacement on an image; FIG. 4 is a diagram showing the relationship between the horizontal position with respect to the camera and the distance from the camera; It is a figure explaining the comparison of the image correction apparatus which concerns on 2nd Embodiment, and the past.

Embodiments of an image correction apparatus and an image correction method according to the present invention will be described below with reference to the drawings. An image correction device according to the present invention is a device for correcting an image of a stereo camera or a monocular camera mounted on a vehicle, for example, before shipment or during periodic inspection. In the following description, the vertical, horizontal, front and rear directions and positions are the directions and positions when viewed from the side of the camera that captures the image. Also, in the description of the drawings, the same elements are denoted by the same reference numerals, and redundant description thereof will be omitted.

[First Embodiment]
FIG. 1 is a plan view showing an image correction device according to the first embodiment, and FIG. 2 is a diagram showing the configuration of a stereo camera. The image correction apparatus 10 according to the present embodiment mainly includes a chart 30 on which a plurality of markers 301 are arranged, and an image acquisition for acquiring an image captured by a stereo camera 20 that captures an image of the markers 301 through the windshield 40. 215, a marker position detection unit 221 that detects the marker positions on the image acquired by the image acquisition unit 215, and an image captured by the stereo camera 20 based on the marker positions detected by the marker position detection unit 221. and an image correction data creation unit 222 that creates data for correcting the distortion of the image.

The markers 301 are a plurality of black circles arranged in a predetermined pattern and drawn on the chart 30, as shown in FIG. 3, for example. Note that the markers 301 may be arranged on the chart 30 by printing, pasting, or the like. As shown in FIG. 1 , the marker 301 is arranged on or behind the marker arrangement curve 302 when viewed from the imaging direction of the stereo camera 20 in plan view.

Preferably, the marker 301 is placed within an area 304 (see hatched portion in FIG. 1) surrounded by a marker placement curve 302 and a rectangle 303 surrounding the marker placement curve 302 . The marker placement curve 302 is a curve in which the permissible error of the detection distance for each viewing angle of the stereo camera 20 and the permissible error of the installation deviation of the stereo camera 20 are the same, the details of which will be described later.

The stereo camera 20 includes a left imaging system section 200a and a right imaging system section 200b, which are separated by a predetermined distance, and a computing section 210 that performs various computations related to the stereo camera 20. FIG. The left imaging system section 200a has an optical device section 201a and an imaging device section 202a. The optical element section 201a is composed of, for example, a lens, and refracts light to form an image on the imaging element section 202a. The image sensor unit 202a receives an image of light refracted by the optical element unit 201a and generates an image according to the intensity of the light.

The right imaging system section 200b has an optical device section 201b and an imaging device section 202b. The design values of the focal lengths of the left imaging system section 200a and the right imaging system section 200b are the same. The directions of the optical axes of the left imaging system section 200a and the right imaging system section 200b are generally the same. The optical element section 201b is composed of, for example, a lens, and refracts light to form an image on the imaging element section 202b. The imaging element section 202b receives an image of light refracted by the optical element section 201b and generates an image according to the intensity of the light.

The image captured by the left imaging system unit 200a is referred to as a reference image because it is an image that serves as a reference when creating a parallax image. The image captured by the right imaging system 200b is an image for which a region matching the region extracted from the reference image is searched for when generating the parallax image, and is called a reference image.

The calculation unit 210 includes, for example, a CPU (Central Processing Unit) that executes calculations, a ROM (Read Only Memory) as a secondary storage device that records programs for calculations, and storage of calculation progress and temporary control. It is composed of a microcomputer combined with a RAM (Random Access Memory) as a temporary storage device for storing variables. The calculation unit 210 includes a captured image storage unit 211, an image correction information storage unit 212, a marker information storage unit 213, a synchronization signal generation unit 214, a reference image acquisition unit 215a, a reference image acquisition unit 215b, and a creation unit 220. there is

The captured image storage unit 211 stores images output from the left imaging system unit 200a and the right imaging system unit 200b.

The image correction information storage unit 212 stores two-dimensional coordinates (that is, image correction data). The image correction information storage unit 212 stores the image correction data of the left imaging system unit 200a and the right imaging system unit 200b obtained in advance without the windshield 4 (that is, without the windshield). , the image correction data generated by the image correction data generating unit 222 in the state of passing through the windshield 4 are stored so as to correct the image through the windshield without distortion.

In this embodiment, an image without distortion is an image that has been perspectively projected using a pinhole model. The image correction data stored in the image correction information storage unit 212 is used for processing for correcting lens distortion and optical axis deviation of the captured image. A captured image refers to an image captured by a camera at the time of capturing, that is, an image before correction.

The marker information storage unit 213 stores a marker position (in other words, an ideal position of the marker) and a marker image on an image with no error in position and orientation between the marker 301 and the stereo camera 20 and without distortion.

Synchronization signal generator 214 generates and transmits a synchronization signal.

The image acquisition section 215 has a reference image acquisition section 215a and a reference image acquisition section 215b. The reference image acquisition unit 215a transmits the synchronization signal and exposure time information to the imaging device unit 202a in accordance with the synchronization signal of the synchronization signal generation unit 214, and then acquires the image generated by the imaging device unit 202a. The image is stored in the captured image storage unit 211 . The reference image acquisition unit 215b transmits the synchronization signal and exposure time information to the imaging device unit 202b in accordance with the synchronization signal of the synchronization signal generation unit 214, and then acquires the image generated by the imaging device unit 202b. The image is stored in the captured image storage unit 211 .

The creation unit 220 has a marker position detection unit 221 and an image correction data creation unit 222 . The marker position detection section 221 first corrects the images captured by the left imaging system section 200a and the right imaging system section 200b using the image correction data stored in the image correction information storage section 212 . Next, the marker position detection unit 221 performs pattern matching processing, and compares the marker image on the corrected image (in other words, the marker image on the corrected image) with the marker image stored in the marker information storage unit 213. By matching, the marker position (detection position) on the corrected image is detected.

The image correction data generation unit 222 detects the position of the marker 301 detected by the marker position detection unit 221, and the ideal position of the marker 301 and the image correction data stored in the marker information storage unit 213. Image correction data for correcting the standard image and the reference image to the standard image and the reference image without distortion through the windshield are created respectively. In this embodiment, the image correction data for correcting the distortion-free standard image and reference image through the windshield is data for correcting the distortion of the standard image and the reference image captured by the stereo camera 20 . The distortion of the standard image and the reference image includes the above-described lens distortion and optical axis deviation of the image.

Note that when image correction data is stored in the image correction information storage unit 212 in advance, the image correction data creation unit 222 stores the created image correction data in the image correction information storage unit 212 as new image correction data. That is, the image correction data creation section 222 can update the image correction data stored in the image correction information storage section 212 .

The calculation unit 210 further includes a distance detection unit 230 , a corrected image storage unit 216 and a parallax image storage unit 217 . The distance detection section 230 has an image correction section 231 , a parallax calculation section 232 and a recognition section 233 . The distance detection unit 230, the corrected image storage unit 216, and the parallax image storage unit 217 will be described later.

An image correction method using the image correction apparatus 10 will be described below with reference to FIG. The image correction method of this embodiment includes an image acquisition step S101, a marker position detection step S102, and an image correction data creation step S103.

In the image acquisition step S101, first, the synchronization signal generation unit 214 generates a synchronization signal and outputs it to the standard image acquisition unit 215a and the reference image acquisition unit 215b. After receiving the synchronization signal output from the synchronization signal generation unit 214, the reference image acquisition unit 215a outputs the synchronization signal and exposure time information to the imaging device unit 202a. After receiving the synchronization signal from the reference image acquisition unit 215a and the exposure time information, the image sensor unit 202a receives the image of the light refracted by the optical element unit 201a for the exposure time, and produces an image corresponding to the intensity of the light. is generated, and the generated image is output to the reference image acquisition unit 215a. The reference image acquisition unit 215a acquires an image output from the image sensor unit 202a and stores the acquired image in the captured image storage unit 211. FIG.

After receiving the synchronization signal output from the synchronization signal generation unit 214, the reference image acquisition unit 215b outputs the synchronization signal and exposure time information to the image sensor unit 202b. After receiving the synchronizing signal and exposure time information output from the reference image acquisition unit 215b, the imaging element unit 202b receives the image of the light refracted by the optical element unit 201b for the exposure time. A corresponding image is generated, and the generated image is output to the reference image acquisition unit 215b. The reference image acquisition unit 215b acquires an image output from the image sensor unit 202b and stores the acquired image in the captured image storage unit 211. FIG.

In the marker position detection step S102, the marker position detection unit 221 detects the captured reference image and the reference image stored in the captured image storage unit 211, and the reference image and the reference image stored in the image correction information storage unit 212 in advance. Read the image correction data of each reference image. Next, the marker position detection unit 221 calculates the coordinates (Fx(X2, Y2), Fy(X2) on the captured image corresponding to each pixel on the reference image after the image correction, using Equations 1 and 2. , Y2)) (ie, image correction data), the position (X1, Y1) of the captured reference image corresponding to the pixel (X2, Y2) of the reference image after image correction is calculated.

X1=Fx(X2, Y2) (1)
Y1=Fy(X2,Y2) (2)

In addition, the marker position detection unit 221 performs two-dimensional linear interpolation on the luminance values of the four pixels surrounding the position (X1, Y1) of the reference image to obtain the pixel (X2, Y2) of the reference image after the image correction. Calculate the luminance value. That is, the marker position detection unit 221 performs the above procedure for each pixel of the reference image after image correction, and calculates the luminance value of the reference image after image correction. Furthermore, the marker position detection unit 221 also performs the above procedure for the reference image to calculate the luminance value of the reference image after image correction.

Next, the marker position detection unit 221 reads the ideal position of the marker and the marker image stored in the marker information storage unit 213 . Subsequently, the marker position detection unit 221 calculates the sum of absolute values SAD (Sum of Absolute Difference) of the difference between the luminance value of the reference image after the image correction and the luminance value of the marker image in the vicinity of the ideal position of the marker, Find the position with the smallest SAD in the vicinity of the ideal position of the marker.

Subsequently, the marker position detection unit 221 performs equiangular straight line fitting based on the SADs of the pixels on the left and right of the position where the SAD is the smallest, and finds the sub-pixel at the position where the reference image after image correction and the marker image most match. calculate. Furthermore, the marker position detection unit 221 calculates the horizontal position of the marker by adding this sub-pixel to the horizontal position where the SAD is the smallest.

Next, the marker position detection unit 221 performs equiangular straight line fitting based on the SADs of the pixels above and below the position where the SAD is the smallest, and finds the sub-pixel at the position where the reference image after image correction and the marker image most match. calculate. Furthermore, the vertical position of the marker is calculated by adding this sub-pixel to the vertical position where the SAD is the smallest. Similarly, the marker position detection unit 221 performs similar processing for other markers, and calculates the positions of the markers on the reference image after image correction. The position of the marker calculated by the above process is called the detection position of the marker.

The marker position detection unit 221 also performs the above image correction and marker position calculation processing on the reference image.

In the image correction data creation step S103, the image correction data creation unit 222 first creates the image correction data of the standard image and the reference image stored in the image correction information storage unit 212 and the image correction data stored in the marker information storage unit 213. An undistorted reference image and the ideal position of the marker on the reference image are read, respectively. Next, the image correction data generation unit 222 generates a distortion-free image through the windshield based on the reference image corrected by the read image correction data, that is, the reference image converted into the distortion-free image without the windshield. Create image conversion data for converting to a reference image.

Subsequently, the image correction data creation unit 222 creates a reference image that is captured based on the created image conversion data and the image correction data that converts the captured reference image into an image without distortion without a windshield. image correction data for correcting to a distortion-free reference image through the windshield.

More specifically, as shown in FIG. 5, the image correction data generation unit 222 first selects the pixel 305 on the reference image without distortion through the windshield among the markers 301 on the reference image. , the nearest markers 301a, 301b, 301c and 301d at the top right, bottom left and bottom right.

Subsequently, the image correction data creation unit 222 performs two-dimensional linear interpolation using Equations 3 to 10 to convert the reference image corrected by the stored image correction data into a distortion-free reference image through the windshield. Convert to an image, that is, calculate the value (Xb, Yb) of the image conversion data of a certain pixel 305 (X, Y). By performing such calculations for each pixel, the above image transformation data are calculated.

Yamu=(Ya1-Ya2)(X-Xa2)/(Xa1-Xa2)+Ya2 (3)
Xbmu=(Xb1-Xb2)(X-Xa2)/(Xa1-Xa2)+Xb2 (4)
Ybmu=(Yb1-Yb2)(X-Xa2)/(Xa1-Xa2)+Yb2 (5)
Yamd=(Ya3-Ya4)(X-Xa4)/(Xa3-Xa4)+Ya4 (6)
Xbmd=(Xb3-Xb4)(X-Xa4)/(Xa3-Xa4)+Xb4 (7)
Ybmd=(Yb3-Yb4)(X-Xa4)/(Xa3-Xa4)+Yb4 (8)
Yb=(Xbmd-Xbmu)(Y-Yamu)/(Yamd-Yamu)+Xbmu (9)
Xb=(Ybmd-Ybmu)(Y-Yamu)/(Yamd-Yamu)+Ybmu (10)

In Equations 3 to 10, (Xa1, Ya1) is the ideal position of the marker 301a on the upper left with respect to a certain pixel, (Xa2, Ya2) is the ideal position of the marker 301b on the upper right with respect to the certain pixel, (Xa3, Ya3 ) indicates the ideal position of the lower left marker 301c with respect to a certain pixel, and (Xa4, Ya4) indicates the ideal position of the lower right marker 301d with respect to a certain pixel. Then, (Xb1, Yb1) is the detection position of the marker 301a on the upper left of a certain pixel, (Xb2, Yb2) is the detection position of the marker 301b on the upper right of the certain pixel, and (Xb3, Yb3) is the certain pixel. (Xb4, Yb4) indicates the detection position of the marker 301d on the lower right of a certain pixel, respectively.

Next, the image correction data generation unit 222 performs two-dimensional linear interpolation on a certain pixel on the undistorted reference image through the windshield using Equations 11 to 18 to obtain the imaged reference image. , the value of the image correction data to be converted into a distortion-free reference image through the windshield is calculated. By performing such calculation for each pixel, the above image correction data are calculated.

Ycmu=(Yc1-Yc2)(Xb-Xc2)/(Xc1-Xc2)+Yc2 (11)
Xdmu=(Xd1-Xd2)(Xb-Xc2)/(Xc1-Xc2)+Xd2 (12)
Ydmu=(Yd1-Yb2)(Xb-Xc2)/(Xa1-Xa2)+Yb2 (13)
Ycmd=(Yc3-Yc4)(Xb-Xc4)/(Xc3-Xc4)+Yc4 (14)
Xdmd=(Xd3-Xd4)(Xb-Xc4)/(Xc3-Xc4)+Xd4 (15)
Ydmd=(Yd3-Yd4)(Xb-Xc4)/(Xc3-Xc4)+Yd4 (16)
Yd=(Xdmd-Xdmu)(Yb-Ycmu)/(Ycmd-Ycmu)+Xdmu (17)
Xd=(Ydmd-Ydmu)(Yb-Ycmu)/(Ycmd-Ycmu)+Ydmu (18)

In equations 11 to 18, (Xc1, Yc1) is a pixel located on the upper left with respect to the value (Xb, Yb) of image conversion data of a certain pixel (X, Y), (Xc2, Yc2) is a certain pixel (X, Y ), and (Xc3, Yc3) is located at the lower left with respect to the image conversion data value (Xb, Yb) of a certain pixel (X, Y). A pixel (Xc4, Yc4) indicates a pixel on the lower right with respect to the image conversion data value (Xb, Yb) of a certain pixel (X, Y). (Xd1, Yd1) is the value of the image correction data of the upper left pixel (Xc1, Yc1), (Xd2, Yd2) is the value of the image correction data of the upper right pixel (Xc2, Yc2), and (Xd3, Yd3) is The value of the image correction data of the pixel (Xc3, Yc3) on the lower left, and (Xd4, Yd4) indicate the value of the image correction data of the pixel (Xc4, Yc4) on the lower right.

Note that the image correction data creation unit 222 also performs the above-described processing for the reference image in the same manner as for the reference image.

Then, the image correction data creation unit 222 stores the image correction data for correcting the captured standard image and the reference image to the standard image and the reference image without distortion through the windshield as new image correction data in the image correction information storage unit 212. memorize to That is, the image correction data creating section 222 updates the image correction data stored in the image correction information storage section 212 .

Next, the calculation unit 210 detects the distance of the three-dimensional object using the image correction data created by the image correction method shown in FIG. At this time, the corrected image storage unit 216, the parallax image storage unit 217, and the distance detection unit 230, which are not used in the image correction processing, are used. Specifically, the image correction unit 231 of the distance detection unit 230 acquires a reference image captured by the left imaging system 200a and a reference image captured by the right imaging system 200b. Next, the image correction unit 231 uses the image correction data created by the image correction method shown in FIG. 4 to correct the captured standard image and reference image to images without distortion through the windshield.

Subsequently, the parallax calculator 232 uses pattern matching processing to search for positions on the reference image that match each region in the reference image after image correction, and calculates parallax. The parallax calculation unit 232 stores the parallax images obtained by calculating the parallax of each region in the parallax image storage unit 217 .

Subsequently, the recognition unit 233 calculates the distance from the parallax image, determines that the object is a three-dimensional object when the area of the area within the range of the distance difference is within the threshold value, and determines the position of the three-dimensional object on the image. And the three-dimensional position of the three-dimensional object is calculated based on the distance.

The three-dimensional positions of three-dimensional objects calculated as described above are used in safety support systems such as collision prevention and automatic driving systems. For example, in collision prevention sensing at single roads and intersections, it is required to detect the detection distance at each viewing angle as shown in FIG. 6 with the required accuracy. In FIG. 6, the horizontal axis indicates the viewing angle of the camera, and the vertical axis indicates the detection distance. A hatched area 501 is a detection distance required for collision prevention for each viewing angle (hereinafter, "detection distance required for collision prevention" may be simply referred to as "required detection distance"). .

As shown in FIG. 6, viewing angles of about -30° to 30° represent the required detection distance for a single pass, and viewing angles of about -60° to -30° and about 30° to 60° represents the detection distance required at the intersection. Here, the optical axis direction of the camera is assumed to be 0°. At an intersection, the vehicle has a smaller turning radius and travels at a slower speed than on a single road, so the detection distance required for collision prevention is short.

FIG. 7 shows the permissible error of parallax at the viewing angle required for the stereo camera. This parallax tolerance can be calculated using Equation 19 based on the required detection distance and the required detection accuracy. In Equation 19, B is the baseline length, f is the focal length, c is the pixel pitch, Lr is the required detection distance, and Rr is the required detection accuracy.
Dr=(B f/c) {1/Lr-1/((1+Rr)Lr)} (19)

At a viewing angle with a small absolute value, the required detection distance is longer than at a viewing angle with a large absolute value, so the permissible error of parallax is small. In FIG. 7, if the parallax error is within a hatched area 601, the detection distance (that is, area 501) required for collision prevention shown in FIG. 6 can be detected with the required accuracy.

On the other hand, when the stereo camera 20 is installed in the vehicle, the installation deviation of the stereo camera may occur. This misalignment causes an error in image correction. The image correction error can be calculated using Equation 20.
Dr=(B f/c) {1/Lm-1/(Lm+ΔLm)} (20)

In Equation 20, Lm represents the distance from the stereo camera to the marker, and ΔLm represents the installation error (also referred to as installation deviation) of the marker. Here, since the installation position of the stereo camera and the installation position of the marker 301 are in a relative relationship, the installation error of the marker may be considered as the installation deviation of the stereo camera.

Then, the parallax tolerance required for collision prevention calculated by Equation 19 (that is, the viewing angle parallax tolerance required for the stereo camera), and the image due to the installation misalignment of the stereo camera calculated by Equation 20 The distance from the stereo camera to the marker is set so that the correction error (that is, the allowable error of the stereo camera installation misalignment) is the same, and the standard image and the reference image are created using the image correction data created by the image correction. is corrected, it becomes possible to detect the detection distance required for collision prevention with the required accuracy.

Using Equations 19 and 20, an equation for calculating the distance between the stereo camera and the marker at each viewing angle where the error in image correction due to misalignment of the stereo camera and the permissible error in parallax required for collision prevention are the same. Formula 21 is obtained by derivation.
Lm={- c Dr ΔLm±√(c^2 Dr^2 ΔLm^2+4 c Dr B f ΔLm)}/(2 c Dr) (21)

FIG. 8 shows the distance between the stereo camera and the marker obtained using Equation (21). If the marker is placed on the distance curve 701 between the stereo camera and the marker or at a distance farther than the curve 701, the parallax error will be within the area 601 in FIG. can be detected with the required accuracy. Preferably, by arranging the marker in an area 703 surrounded by a distance curve 701 between the stereo camera and the marker and a rectangle 702 surrounding the curve 701, the accuracy required for the distance detected by the stereo camera can be maintained. can. The distance curve 701 between the stereo camera and the markers is a curve that satisfies the detection distance and detection accuracy required for collision prevention, and is the marker placement curve 302 described above.

FIG. 9 is a diagram for explaining a comparison between the image correcting apparatus according to the first embodiment and the conventional one. In FIG. 9, the conventional image correcting apparatus is indicated by the one-dot chain line. As shown in FIG. 9, conventionally, markers are arranged on a planar chart. In order to detect the distance range with the required accuracy, it is necessary to arrange the planar chart 801 farther than the marker arrangement curve 302 . Therefore, in order to correct the image in the viewing angle of the detection range, the size of the chart 801 in the longitudinal direction becomes long, and the area of the image correction device becomes large.

On the other hand, in the image correction apparatus 10 according to the present embodiment, when viewed from the imaging direction of the stereo camera 20 in a plan view, the marker 301 has a permissible error in the detection distance for each viewing angle and a permissible deviation in the installation of the stereo camera 20. It is placed on or behind the marker placement curve 302 that coincides with the error. As a result, the detection distance for each viewing angle can be detected within the allowable error, and the area of the chart 30 can be reduced compared to the conventional case. As a result, the area of the device can be reduced while maintaining the accuracy required for the distance detected by the stereo camera 20 .

In particular, when the marker 301 is placed in an area 304 surrounded by a marker placement curve 302 and a rectangle 303 surrounding the marker placement curve 302, the detection distance and detection accuracy required for collision prevention are satisfied, and image correction is performed. The installation area of the device 10 can be reduced.

Moreover, according to the image correction method according to the present embodiment, the accuracy required for the distance detected by the stereo camera 20 can be maintained.

It should be noted that various modifications are conceivable for the image correction apparatus of this embodiment. Some of the modifications are described below. In addition, according to each modification, the above-mentioned effect is also obtained.

[Modification 1]
10 and 11, in the image correction apparatus shown in FIGS. 1 and 2, the marker information storage unit 213 and the creation unit 220 are removed from the calculation unit 210 and newly provided as a calculation unit 240. . That is, the calculation section 240 having the marker information storage section 213 and the creation section 220 is provided separately from the calculation section 210 . The structures and roles of the marker information storage unit 213, the marker position detection unit 221 of the creation unit 220, and the image correction data creation unit 222 are the same as those described above.

By removing the marker information storage unit 213 and the creation unit 220 from the calculation unit 210 in this way, the execution file for performing the image correction processing can be removed, and the amount of memory used in the calculation unit 210 can be reduced. Also, the newly provided calculation unit 240 may be installed exclusively for the process of mounting the stereo camera 20 on the vehicle. In this way, since the calculation processing capacity of the calculation unit 240 is faster than that of the calculation unit 210, the time required for the image correction processing can be shortened, and the number of devices capable of correcting images per hour can be increased.

[Modification 2]
1 and 2, instead of storing the ideal positions of the markers on the standard image and the reference image without distortion in the marker information storage unit 213, the marker 301 and the left imaging system The position between the sections 200a, the position between the marker 301 and the right imaging system section 200b, the focal lengths of the left imaging system section 200a and the right imaging system section 200b, the pixel pitch size, and the optical axis position on the image are stored.

Therefore, in the image correction data creation step S103, the image correction data creation unit 222 stores the marker information instead of reading the ideal positions of the markers on the undistorted standard image and the reference image stored in the marker information storage unit 213. The position between the marker 301 and the left imaging system 200a, the position between the marker 301 and the right imaging system 200b, the focal lengths of the left imaging system 200a and the right imaging system 200b, and the optical axis on the image stored in the unit 213 The positions are read, the ideal positions of the markers on the standard image and the reference image without distortion are calculated based on this information, and image correction data for the standard image and the reference image are created.

[Modification 3]
1 and 2, the image correction information storage unit 212 stores the image correction data of the left imaging system unit 200a and the right imaging system unit 200b obtained without the windshield. Instead of storing the design value of the position between the marker 301 and the left imaging system 200a, the design value of the position between the marker 301 and the right imaging system 200b, the distortion data of the left imaging system 200a and the right imaging system 200b , the design values of the image correction data calculated based on the design values of the focal length, pixel pitch size, and optical axis position on the image are stored.

Therefore, in the marker position detection step S102 and the image correction data creation step S103, instead of the image correction data of the left imaging system unit 200a and the right imaging system unit 200b obtained without the windshield, Using the design values of the image correction data of the left imaging system 200a and the right imaging system 200b, the captured reference image and the reference image are corrected, and the left imaging system corrects the distortion-free image through the windshield. 200a and image correction data for the right imaging system 200b.

In this case, since the image correction data of the left imaging system section 200a and the right imaging system section 200b obtained without the windshield is not used, images are separately captured in advance without the windshield, and the left imaging system section 200a is corrected. And it becomes unnecessary to obtain the image correction data of the right imaging system section 200b.

[Modification 4]
In Modified Example 4, the marker information storage unit 213 stores only the ideal positions of the markers, not the marker images, in contrast to the image correction apparatus shown in FIGS. Therefore, the following processing is performed in the marker position detection step S102.

In the marker position detection step S<b>102 , the marker position detection unit 221 first reads the captured standard image and reference image stored in the captured image storage unit 211 . Next, the marker position detection unit 221 performs binarization processing in which pixels having a luminance value equal to or less than a predetermined luminance value are set to 1 and other pixels are set to 0 in the captured standard image and reference image. Furthermore, the marker position detection unit 221 detects the positions of the markers by calculating the center-of-gravity position of each marker on the binarized image.

[Modification 5]
As explained in FIG. 6, the detection distance for a viewing angle of about -30° to 30° required for a single pass is less than that for a viewing angle of about -60° to -30° and about 30° to 60° required for an intersection. Longer than the detection distance of the viewing angle. Thus, viewing angles of about -30° to 30° have less parallax tolerance than viewing angles of about -60° to -30° and about 30° to 60° (see Figure 7). In addition, the markers within the viewing angles of about -60° to -30° and about 30° to 60° are more likely to have the left imaging system unit 200a and the right imaging system 200a than the markers within the viewing angles of about -30° to 30°. It can be installed near the part 200b (see FIG. 8).

Therefore, in Modified Example 5, in the chart 30 of the image correction apparatus shown in FIGS. A first region is defined as a second region, and a viewing angle region of about -60° to -30° and about 30° to 60° having a larger permissible value of parallax deviation relative to the permissible value of detection distance deviation than the first region is defined as a second region. . The markers placed in the second area are closer to the left imaging system section 200a and the right imaging system section 200b than the markers placed in the first area.

In this case, since the marker is placed in an area 304 (area 703 in FIG. 8) surrounded by the marker placement curve 302 and a rectangle 303 (rectangle 702 in FIG. 8) surrounding this marker placement curve, collision prevention is required. The required detection distance and detection accuracy can be satisfied, and the installation area of the image correction device can be made smaller than in the conventional technology in which markers are arranged on a planar chart.

[Modification 6]
In a modification 6 shown in FIGS. 12 and 13, the form of the chart 30 is changed from the image correction apparatus shown in FIGS. 1 and 2. FIG. Specifically, as shown in FIG. 12, the chart 30 is arranged on the left and right sides of the first region 30A, which has a viewing angle of about -30° to 30° used in a single pass, and the first region 30A, A second region 30B with viewing angles of about -60° to -30° and about 30° to 60° used at the intersection is separated. A plurality of markers 306 are arranged in the first area 30A substantially perpendicular to the optical axes of the left imaging system 200a and the right imaging system 200b. A plurality of markers 307 are arranged in the second area 30B.

Here, the marker 307 arranged in the second area 30B is positioned closer to the left imaging system section 200a and the right imaging system section 200b than the marker 306 arranged in the first area 30A. The chart 30 has the shape shown in FIG.

Therefore, in the image correction data creation step S103, the image correction data creation unit 222 uses the detection positions of the markers for the first area 30A and the second area 30B of the chart 30 on the reference image, as in the above-described process. Image correction data for converting the first region 30A and the second region 30B on the reference image captured through the windshield into the distortion-free first region 30A and the second region 30B of the reference image through the windshield are calculated, respectively. Subsequently, the image correction data creation unit 222 calculates image correction data for the boundary between the first region 30A and the second region 30B of the image correction data of the reference image in the horizontal direction using spline interpolation. Next, the image correction data generation unit 222 combines the image correction data of the first area 30A, the image correction data of the second area 30B, and the image correction data of the boundary between these areas, and performs image correction of the entire reference image. Create data. Note that the image correction data creation unit 222 performs similar processing on the reference image as well.

[Modification 7]
In a modification 7 shown in FIGS. 14 and 15, the form of the chart 30 is changed from the image correction apparatus shown in FIGS. 1 and 2. FIG. Specifically, as shown in FIG. 14, markers are arranged on a long side 303A of a rectangle 303 and a pair of opposing short sides 303B and 303C in a region 304, respectively. More specifically, a marker 308 is arranged on the long side 303A, a marker 309 is arranged on the short side 303B, and a marker 310 is arranged on the short side 303C. The chart 30 has the shape shown in FIG.

For this reason, in the image correction data creation step S103, the image correction data creation unit 222 calculates the detection positions of the markers for the regions of the long side 303A and the short sides 303B and 303C of the chart 30 on the image, as in the above-described process. image correction data for converting the area of the long side 303A and the short sides 303B and 303C of the imaged reference image to the area of the long side 303A and the short sides 303B and 303C of the reference image without distortion through the windshield. Using spline interpolation, horizontal boundaries between the long side 303A and the short side 303B and between the long side 303A and the short side 303C of the image correction data of the reference image are calculated. Furthermore, the image correction data creating unit 222 creates image correction data for the entire reference image by combining the image correction data for the areas of the long side 303A and the short sides 303B and 303C and the boundaries of these areas. Note that the image correction data creation unit 222 performs similar processing on the reference image as well.

[Modification 8]
In a modified example 8 shown in FIG. 16, the form of the chart 30 is changed with respect to the image correction apparatus shown in FIGS. Specifically, as shown in FIG. 16, the chart 30 is arranged on the left and right sides of the first region 30A, which has a viewing angle of about -30° to 30° used in a single pass, and the first region 30A, and a second region 30B with viewing angles of about -60° to -30° and about 30° to 60° used at crossings. In the first area 30A, the standard image and the reference image of the left imaging system section 200a and the right imaging system section 200b are superimposed, that is, stereo vision is achieved. On the other hand, in the second region 30B, the standard image and the reference image of the left imaging system 200a and the right imaging system 200b do not overlap, that is, monocular vision.

Therefore, the recognition unit 233 calculates the distance from the parallax image for the first area 30A as described above, and calculates the distance for the second area 30B by the method described in the second embodiment. Also, the marker placement curve in this case is divided into a marker placement curve 302A in the first region 30A and a marker placement curve 302B in the second region 30B.

The marker placement curve 302A is calculated based on the detection distance required for a single pass and its tolerance using Equations 19-21 above. The marker placement curve 302B is calculated using Equations 22 to 25, which will be described later, based on the detection distance required at the intersection and its allowable error. The marker 301 is placed within an area 304 surrounded by marker placement curves 302A and 302B and a rectangle 303 surrounding the marker placement curves 302A and 302B.

In this case, in the image correction data creation step S103, the image correction data creation unit 222 creates the image correction data of the stereo camera 20 based on the stereo vision of the first area 30A and the monocular vision of the second area 30B.

[Modification 9]
The shape of the markers arranged on the chart is not limited to the black circles described above, and may be, for example, a lattice pattern as shown in FIG. 17 or a checkerboard pattern as shown in FIG. In the case of the grid pattern shown in FIG. 17, the marker position detection unit 221 detects the position of the intersection of the horizontal line and the vertical line. In the case of the checkered pattern shown in FIG. 18, the marker position detector 221 detects the positions of the corners of the square.

[Second embodiment]
A second embodiment of the image correction apparatus and image correction method will be described below with reference to FIGS. 19 and 25. FIG. The image correction apparatus and image correction method of this embodiment are different from those of the above-described first embodiment in that an image captured by a monocular camera is corrected. The difference will be mainly described below.

As shown in FIGS. 19 and 20 , the image correction apparatus 100 according to the present embodiment has a chart 30 on which a plurality of markers 301 are arranged, and a monocular camera 120 that captures the markers 301 through the windshield 40 . a marker position detection unit 1221 that detects the marker position on the image acquired by the image acquisition unit 1214; and an image correction data creation unit 1222 that creates data for correcting distortion of an image captured by the camera 120 .

The markers 301 are, as described in the first embodiment, a plurality of black dots arranged in a predetermined pattern and arranged on the chart 30 . As shown in FIG. 19 , the marker 301 is arranged on or behind the marker placement curve 311 when viewed from the imaging direction of the monocular camera 120 in plan view.

Preferably, marker 301 is positioned within area 304 bounded by marker placement curve 311 and rectangle 303 surrounding marker placement curve 311 . The marker placement curve 311 is a curve in which the permissible error of the detection distance for each viewing angle of the monocular camera 120 and the permissible error of the installation deviation of the monocular camera 120 are the same, the details of which will be described later.

The monocular camera 120 includes an imaging system section 1200 and a calculation section 1210 that performs various calculations related to the monocular camera 120 . The imaging system section 1200 has an optical element section 1201 and an imaging element section 1202 . The optical element unit 1201 is composed of, for example, a lens, and refracts light to form an image on the imaging element unit 1202 . The imaging device unit 1202 receives the image of the light refracted by the optical device unit 1201 and generates an image according to the intensity of the light.

The computation unit 1210 includes, for example, a CPU (Central Processing Unit) that executes computation, a ROM (Read Only Memory) as a secondary storage device that records a program for computation, and storage of computation progress and temporary control. It is composed of a microcomputer combined with a RAM (Random Access Memory) as a temporary storage device for storing variables. The calculation unit 1210 includes a captured image storage unit 1211 , an image correction information storage unit 1212 , a marker information storage unit 1213 , an image acquisition unit 1214 and a creation unit 1220 .

The captured image storage unit 1211 stores images output from the imaging system unit 1200 .

The image correction information storage unit 1212 stores two-dimensional coordinates (that is, image correction data) on the distorted captured image corresponding to each pixel on the undistorted image for the image output from the imaging system unit 1200. . This image correction information storage unit 1212 stores the image correction data of the imaging system unit 1200 obtained in advance without the windshield 4 (that is, without the windshield), and also stores the image correction data obtained through the windshield 4. The image correction data created by the image correction data creating unit 1222 in the state is stored to correct the image through the windshield without distortion.

In this embodiment, an image without distortion is an image that has been perspectively projected using a pinhole model. The image correction data stored in the image correction information storage unit 1212 is used for processing for correcting lens distortion and optical axis deviation of the captured image. A captured image refers to an image captured by a camera at the time of capturing, that is, an image before correction.

The marker information storage unit 1213 stores a marker position (in other words, an ideal position of the marker) and a marker image on an image without errors in position and orientation between the marker 301 and the monocular camera 120 and without distortion.

The image acquisition unit 1214 acquires an image generated by the image sensor unit 1202 after sending the imaging signal and exposure time information to the image sensor unit 1202 and stores the acquired image in the captured image storage unit 1211 .

Creation unit 1220 has marker position detection unit 1221 and image correction data creation unit 1222 . The marker position detection unit 1221 first corrects the captured image of the imaging system unit 1200 using the image correction data stored in the image correction information storage unit 1212 . Next, the marker position detection unit 1221 performs pattern matching processing, and compares the marker image on the corrected image (in other words, the marker image on the corrected image) with the marker image stored in the marker information storage unit 1213. By matching, the marker position (detection position) on the corrected image is detected.

The image correction data creation unit 1222 detects the position of the marker 301 detected by the marker position detection unit 1221, and the ideal position of the marker 301 and the image correction data stored in the marker information storage unit 1213. Create image correction data for correcting the image through the windshield to an image without distortion. In this embodiment, the image correction data for correcting the distortion-free image through the windshield is data for correcting the distortion of the image captured by the monocular camera 120 . Image distortion includes lens distortion and optical axis deviation of the image described above.

Note that when image correction data is stored in advance in the image correction information storage unit 1212, the image correction data creation unit 1222 stores the created image correction data in the image correction information storage unit 1212 as new image correction data. That is, the image correction data creating section 1222 can update the image correction data stored in the image correction information storage section 1212 .

The calculation unit 1210 further includes a distance detection unit 1230 and a corrected image storage unit 1215 . The distance detection unit 1230 has an image correction unit 1231 , a recognition unit 1232 and a three-dimensional object distance detection unit 1233 . The image correction unit 1231, the recognition unit 1232, and the three-dimensional object distance detection unit 1233 will be described later.

An image correction method using the image correction apparatus 100 will be described below with reference to FIG. The image correction method of this embodiment includes an image acquisition step S1101, a marker position detection step S1102, and an image correction data creation step S1103.

In image acquisition step S<b>1101 , first, the image acquisition unit 1214 outputs an imaging signal and exposure time information to the image sensor unit 1202 . After receiving the imaging signal and exposure time information from the image acquisition unit 1214, the image sensor unit 1202 receives an image of light refracted by the optical element unit 1201 for the exposure time, and produces an image corresponding to the intensity of the light. , and outputs the generated image to the image acquisition unit 1214 . The image acquisition unit 1214 acquires an image output from the image sensor unit 1202 and stores the acquired image in the captured image storage unit 1211 .

In marker position detection step S1102, the marker position detection unit 1221 reads the captured image stored in the captured image storage unit 1211 and the image correction data stored in the image correction information storage unit 1212 in advance. Next, the marker position detection unit 1221 calculates the coordinates (Fx(X2, Y2), Fy( X2, Y2)) (ie, image correction data), the position (X1, Y1) of the captured image corresponding to the pixel (X2, Y2) of the image after image correction is calculated.

In addition, the marker position detection unit 1221 performs two-dimensional linear interpolation on the luminance values of four pixels around the position (X1, Y1) of the image to obtain the luminance value of the pixel (X2, Y2) of the image after image correction. Calculate That is, the marker position detection unit 1221 performs the above procedure for each pixel of the image after image correction, and calculates the luminance value of the image after image correction.

Next, the marker position detection unit 1221 reads the ideal position of the marker and the marker image stored in the marker information storage unit 1213 . Subsequently, the marker position detection unit 1221 calculates the sum of absolute values SAD (Sum of Absolute Difference) of the difference between the luminance value of the image after image correction and the luminance value of the marker image in the vicinity of the ideal position of the marker. A position with the smallest SAD is found in the vicinity of the ideal position of .

Subsequently, the marker position detection unit 1221 performs equiangular straight line fitting based on the SADs of the pixels on the left and right of the position where the SAD is the smallest, and calculates the sub-pixel at the position where the image after image correction and the marker image most match. do. Furthermore, the marker position detection unit 1221 calculates the horizontal position of the marker by adding this sub-pixel to the horizontal position where the SAD is the smallest.

Next, the marker position detection unit 1221 performs equiangular straight line fitting based on the SADs of the pixels above and below the position where the SAD is the smallest, and calculates the sub-pixel at the position where the image after image correction and the marker image most match. do. Furthermore, the vertical position of the marker is calculated by adding this sub-pixel to the vertical position where the SAD is the smallest. Similarly, the marker position detection unit 1221 performs similar processing for other markers, and calculates the position of the marker on the image after image correction. The position of the marker calculated by the above process is called the detection position of the marker.

In image correction data generation step S1103, the image correction data generation unit 1222 first generates image correction data for an image stored in the image correction information storage unit 1212 and an undistorted image stored in the marker information storage unit 1213. Read the ideal position of the upper marker, respectively. Next, the image correction data generation unit 1222 converts the image corrected by the read image correction data, that is, the image converted into the distortion-free image without the windshield, to the distortion-free image through the windshield. Create image conversion data for conversion.

Subsequently, the image correction data creation unit 1222 converts the captured image to the front windshield based on the created image conversion data and the image correction data for converting the captured image into an image without distortion without the windshield. To create image correction data for correcting an image without distortion through glass.

Specifically, as shown in FIG. 5 above, the image correction data creation unit 1222 first, among the markers 301 on the image, for the pixel 305 on the image without distortion through the windshield, the upper left, Search the nearest markers 301a, 301b, 301c and 301d in the upper right, lower left and lower right.

Subsequently, the image correction data creation unit 1222 performs two-dimensional linear interpolation using the above-described equations 3 to 10 to generate an image corrected by the stored image correction data through the windshield without distortion. Convert to an image, that is, calculate the value (Xb, Yb) of the image conversion data of a certain pixel 305 (X, Y). By performing such calculations for each pixel, the above image transformation data are calculated.

Next, the image correction data creation unit 1222 performs two-dimensional linear interpolation on a certain pixel on the undistorted image through the windshield using the above equations 11 to 18, so that the captured image is , the value of the image correction data for converting the image through the windshield into a distortion-free image is calculated. By performing such calculation for each pixel, the above image correction data are calculated.

Furthermore, the image correction data creation unit 1222 stores image correction data for correcting the image captured through the windshield into an image without distortion as new image correction data in the image correction information storage unit 1212 . That is, the image correction data creation unit 1222 updates the image correction data stored in the image correction information storage unit 1212. FIG.

Next, the calculation unit 1210 detects the distance of the three-dimensional object using the image correction data created by the image correction method shown in FIG. At this time, the corrected image storage unit 1215 and the distance detection unit 1230 that are not used in image correction processing are used. Specifically, the image correction unit 1231 of the distance detection unit 1230 acquires the captured image from the imaging system unit 1200, and uses the image correction data created by the image correction method shown in FIG. Corrects the image through the windshield to an image without distortion.

Subsequently, as shown in FIG. 22, the recognition unit 1232 calculates the feature amount of the vehicle in each area on the image 1401, and detects an area 1402 that matches the feature of the vehicle. The recognition unit 1232 also calculates the feature amount of the pedestrian in each area on the image 1401 and detects an area 1403 that matches the feature of the pedestrian. The three-dimensional object distance detection unit 1233 calculates the distance between the vehicle and the pedestrian based on the position where the vehicle area 1402 and the pedestrian area 1403 are in contact with the road surface.

The three-dimensional positions of three-dimensional objects calculated as described above are used in safety support systems such as collision prevention and automatic driving systems. For example, in collision prevention sensing at single roads and intersections, it is required to detect the detection distance at each viewing angle as shown in FIG. 6 with the required accuracy.

FIG. 23 shows the vertical positional tolerance for viewing angles required for a monocular camera. This vertical position tolerance can be calculated using Equation 22 based on the required sensing distance and required sensing accuracy. In Expression 22, H is the installation height of the monocular camera, f is the focal length, c is the pixel pitch, Lr is the required detection distance, and Rr is the required detection accuracy.
Gr=(Hf/c)(1/Lr-1/(RrLr)) (22)

At a viewing angle with a small absolute value, the required detection distance is longer than at a viewing angle with a large absolute value, so the vertical positional tolerance is small. In FIG. 23, if the vertical positional error is within a hatched area 1501, the detection distance (that is, area 501) required for collision prevention shown in FIG. 6 can be detected with the required accuracy.

On the other hand, when the monocular camera 120 is installed in the vehicle, the monocular camera 120 may be installed out of alignment. This misalignment causes an error in image correction. The image correction error can be calculated using Equation 23.
Gr-(fΔYm/(c(Lm+ΔLm))=(fHm/c)(1/Lm-1/(Lm+ΔLm)) (23)
Hm=H Lm/L (24)

In Equations 23 and 24, Lm is the distance from the monocular camera to the marker, ΔLm is the distance error in setting the marker, ΔYm is the positional error in setting the marker, and Hm is the height of the marker.

Then, the vertical position tolerance required for collision prevention calculated by Equation 22 (that is, the vertical position tolerance of the viewing angle required for the monocular camera) and the monocular camera calculated by Equation 23 Set the distance from the monocular camera to the marker so that the error in image correction due to installation misalignment (that is, the tolerance for monocular camera installation misalignment) matches, and use the image correction data created by image correction By correcting the image, it becomes possible to detect the detection distance required for collision prevention with the required accuracy.

Using Equations 22 and 23, calculate the distance between the monocular camera and the marker at each viewing angle where the image correction error due to misalignment of the monocular camera and the vertical position tolerance required for collision prevention match. Equation 25 is obtained by deriving an equation for
Lm=f(HΔLm+ΔYmL)/(cGrL) (25)

FIG. 24 shows the distance between the monocular camera and the marker obtained using Equation 25. In FIG. If the marker is placed on the distance curve 1601 between the monocular camera and the marker or at a distance farther than the curve 1601, the positional error in the vertical direction falls within the region 1501 in FIG. A detection distance region 501 can be detected with the required accuracy. Preferably, by arranging the marker within an area 1603 surrounded by a distance curve 1601 between the monocular camera and the marker and a rectangle 1602 surrounding the curve 1601, it is possible to maintain the accuracy required for the distance detected by the monocular camera. can. The distance curve 1601 between the monocular camera and the marker is a curve that satisfies the detection distance and detection accuracy required for collision prevention, and is the marker placement curve 311 described above.

FIG. 25 is a diagram for explaining a comparison between the image correcting apparatus according to the second embodiment and the conventional one. In FIG. 25, the conventional image correcting apparatus is indicated by a dashed line. Conventionally, markers are arranged on a planar chart. In order to detect the distance range with the required accuracy, it is necessary to arrange the planar chart 1701 farther than the marker arrangement curve 311 . Therefore, in order to correct the image in the viewing angle of the detection range, the size of the chart 1701 in the longitudinal direction becomes long, and the area of the image correction device becomes large.

On the other hand, in the image correction apparatus 100 according to the present embodiment, when viewed from the imaging direction of the monocular camera 120 in a plan view, the marker 301 has an allowable error in detection distance for each viewing angle and an allowable deviation in installation of the monocular camera 120. It is placed on or behind the marker placement curve 311 that coincides with the error. As a result, the detection distance for each viewing angle can be detected within the allowable error, and the area of the chart 30 can be reduced compared to the conventional case. As a result, the area of the device can be reduced while maintaining the accuracy required for the distance detected by the monocular camera 120 .

In particular, when the marker 301 is placed in the area 304 surrounded by the marker placement curve 311 and the rectangle 303 surrounding the marker placement curve 311, the detection distance and detection accuracy required for collision prevention are satisfied, and image correction is performed. The installation area of the device 100 can be reduced.

Further, according to the image correction method according to the present embodiment, the accuracy required for the distance detected by the monocular camera 120 can be maintained.

As with the first embodiment, various modifications can be considered for the image correction apparatus 100 of the present embodiment. For example, Modifications 1 to 7 and 9 described above are also applied to the image correction apparatus 100 of this embodiment. A detailed description thereof is omitted here.

Although the embodiments of the present invention have been described in detail above, the present invention is not limited to the above-described embodiments, and various designs can be made without departing from the spirit of the invention described in the claims. Changes can be made.

1,100 image correction device 20 stereo camera 30 chart 30A first area 30B second area 40 windshield 120 monocular camera 200a left imaging system 200b right imaging system 210, 1210 arithmetic units 211, 1211 captured image storage units 212, 1212 Image correction information storage unit 213, 1213 Marker information storage unit 214 Synchronization signal generation unit 215, 1214 Image acquisition unit 215a Reference image acquisition unit 215b Reference image acquisition unit 216, 1215 Corrected image storage unit 217 Parallax image storage unit 220, 1220 Creation unit 221, 1221 marker position detection units 222, 1222 image correction data creation units 230, 1230 distance detection units 231, 1231 image correction unit 232 parallax calculation units 233, 1232 recognition unit 240 calculation units 301, 306, 307, 308, 309, 310 Markers 302, 302A, 302B, 311 Marker arrangement curve 303 Rectangle 304 Area 1233 Three-dimensional object distance detector

Claims (10)

a chart on which a plurality of markers are arranged;
an image acquisition unit that acquires an image captured by a camera that captures an image of the marker through the windshield;
a marker position detection unit that detects a marker position on the image acquired by the image acquisition unit;
an image correction data creation unit that creates data for correcting distortion of an image captured by a camera based on the marker position detected by the marker position detection unit;
with
In a plan view, when viewed from the imaging direction of the camera, the marker is located on or behind the marker arrangement curve where the allowable error of the detection distance for each camera viewing angle and the allowable error of the camera installation deviation match. An image correction device, characterized in that it is arranged in a 2. The image correction device according to claim 1, wherein, in plan view, the marker is arranged within a region surrounded by the marker arrangement curve and a rectangle surrounding the marker arrangement curve. 3. The method according to claim 1, wherein, when the camera is a stereo camera, the marker arrangement curve is a curve in which a viewing angle parallax tolerance required for the stereo camera and a stereo camera installation deviation tolerance are the same. Image corrector. 3. When the camera is a monocular camera, the marker placement curve is a curve in which a vertical positional tolerance of a viewing angle required for the monocular camera and a tolerance of monocular camera installation deviation are the same. The image correction device according to . further comprising an image correction information storage unit for storing image correction data,
The marker position detection unit corrects the image using image correction data stored in advance in the image correction information storage unit, detects the marker position on the corrected image,
The image correction data creation unit creates image correction data based on the deviation between the marker position on the corrected image detected by the marker position detection unit and the ideal position of the marker on the distortion-corrected image. 5. The image correction apparatus according to any one of claims 1 to 4, wherein the corrected image correction data is stored as new image correction data in said image correction information storage unit. The chart has a first region with a relatively small tolerance of detection distance for each camera viewing angle and a second region with a relatively large tolerance of detection distance for each camera viewing angle,
The image correction device according to any one of claims 1 to 5, wherein the markers arranged in the second area are closer to the camera than the markers arranged in the first area. The image correction data creation unit is configured to provide image correction data for correcting distortion of the image of the first area based on the marker positions of the first area detected by the marker position detection unit, and the image correction data detected by the marker position detection unit. image correction data for correcting the distortion of the image of the second area based on the marker position of the second area, and calculating the image correction data of the first area and the image correction data of the second area; The image correction data of the boundary between the first region and the second region is calculated based on the above, and the image correction data of the first region, the image correction data of the second region, and the image correction data of the boundary are combined to obtain the image of the entire image. 7. The image correction device according to claim 6, which creates image correction data. The image acquisition unit acquires an image from a stereo camera having two cameras,
The first area is an area where the imaging areas of the two cameras overlap,
8. The image correcting device according to claim 6, wherein the second area is an area where imaging areas of two cameras do not overlap. Through the windshield, a camera was used to capture an image of a marker placed on or behind the marker arrangement curve where the allowable error of the detection distance for each camera viewing angle and the allowable error of the camera installation deviation match. an image acquisition step of acquiring an image;
a marker position detection step of detecting marker positions on the image acquired in the image acquisition step;
an image correction data creating step of creating data for correcting distortion of an image captured by a camera based on the marker positions detected in the marker position detecting step;
An image correction method comprising: in the marker position detection step, correcting the image using pre-stored image correction data to detect the marker position on the corrected image;
In the image correction data creation step, image correction data is created based on the deviation between the marker position on the corrected image detected in the marker position detection step and the ideal position of the marker on the distortion-corrected image. 10. The image correction method according to claim 9, wherein the image correction data is new image correction data.

Images