KR102024834B1 - Tolerance compensating apparatus and method for automatic vehicle-mounted camera - Google Patents

Abstract

The present invention relates to a calibration device of a camera and a method thereof. More specifically, the present invention relates to a calibration device of a camera installed in a vehicle which calibrates an error generated after a camera installed in a vehicle is installed in the vehicle during the driving of the vehicle; and to a method thereof. According to an embodiment of the present invention, the calibration of the camera installed in the vehicle can be performed from images obtained while driving without a pattern of being installing the camera in front and rear sides, or left and right sides.

Description

Device and method for calibrating camera installed in vehicle {TOLERANCE COMPENSATING APPARATUS AND METHOD FOR AUTOMATIC VEHICLE-MOUNTED CAMERA}

The present invention relates to an apparatus and method for calibrating a camera, and more particularly, to a camera and an apparatus for calibrating a camera installed in a vehicle for calibrating an error occurring after the camera installed in the vehicle is installed in a vehicle.

Recently, due to the development of the automobile industry, the spread of automobiles has been commercialized to the age of one family and one automobile, and various high-tech electronic technologies are applied to automobiles in order to improve the safety of vehicles and the convenience of drivers. Among these advanced electronic technologies, there is an ambient view monitoring system (AVM system), which allows a driver to conveniently check the surrounding environment of the vehicle by photographing and displaying the surrounding environment of the vehicle. The AVM system photographs the surrounding environment through cameras installed on the front, rear, left and right sides of the vehicle, and displays the surrounding environment of the vehicle on the screen by correcting the overlapped region to look natural based on the captured image. Accordingly, the driver can accurately recognize the surroundings of the vehicle through the displayed surrounding environment and can conveniently park without looking at the side mirror or the rear mirror.

1 is a view for explaining a correction system of a camera installed in a conventional vehicle.

Referring to FIG. 1, the vehicle V is disposed at a predetermined position at a place where the correction plates PL1, PL2, PL3, and PL4 are installed. For accurate tolerance correction, the vehicle V is preferably placed exactly at a predetermined position. For example, the front and rear wheel shafts (not shown) of the vehicle V may be disposed at the reference lines L1L1 'and L2L2'. To this end, it is equipped with a vision system (not shown) for photographing the vehicle or a sensor system (not shown) for detecting the position of the vehicle, and may be implemented to check whether or not the vehicle V is disposed at a precisely predetermined position. have. The correction plates PL1, PL2, PL3, and PL4 may be installed to be located on the bottom of each corner of the vehicle V (on the ground on which the vehicle is located) when the vehicle V is disposed at a predetermined position. That is, the correction plates PL1, PL2, PL3, and PL4 are installed at the bottom of the front left corner, the front right corner, the rear left corner, and the rear right corner of the vehicle V, respectively. Of course, the installation position of the correction plates (PL1, PL2, PL3, PL4) can be changed, for example, the correction plates (PL1, PL2, PL3, PL4) is to be installed at a certain distance on each wheel axis of the vehicle (V). Can be. Here, the correction plates PL1, PL2, PL3, and PL4 may be installed at a predetermined distance from each corner of the vehicle V such that two correction plates are included in each image photographed by the cameras 210, 220, 230, and 240. . That is, the image captured by the camera 210 installed in the front includes the correction plate (PL1, PL2), the image captured by the camera 220 installed in the rear includes the correction plate (PL3, PL4), the camera installed on the left The image photographed at 230 includes correction plates PL1 and PL3, and the image captured by the camera 240 installed on the right includes correction plates PL2 and PL4 at a predetermined distance from each corner of the vehicle V. Correction plates PL1, PL2, PL3, and PL4 are installed.

On the other hand, when the camera for the vehicle position automatic display system is mounted on the vehicle, an error may occur in the production process. There is a need for a method for automatically and accurately correcting errors occurring in such a process without operator intervention to increase vehicle production efficiency. In addition, when an error occurs due to an accident or other causes that occur during the vehicle driving process after the vehicle is also required to solve the problem.

As a prior art document on the present invention, there is a patent registration 10-0948886 (2010.03.15).

The present invention provides an apparatus and method for calibrating a camera installed in a vehicle that corrects a tolerance of a camera that provides a different field of view (image) due to a tolerance in mass production.

The present invention provides an apparatus and a method for calibrating a camera installed in a vehicle from an image acquired while driving without a pattern installed on the front, rear, left and right sides.

The present invention provides a device and a method for calibrating a camera installed in a vehicle that does not require a device for aligning a pattern and a vehicle, and is hardly influenced by a rate of change of brightness, such as light reflection during outdoor shooting.

According to one aspect of the invention, there is provided a device for calibrating a camera installed in a vehicle.

An apparatus for calibrating a camera installed in a vehicle according to an exemplary embodiment of the present disclosure may include an image input unit for inputting continuously captured images of surrounding images of a vehicle, and extracting a residual vehicle edge from an image of continuously inputting surrounding images of a vehicle. A rotation estimator is configured to calculate rotation angle information using cross-correlation and dispersion values of transformed values obtained by Hough transforming the edge image of the vehicle extracted from the input surrounding image of the vehicle and the vehicle edge image of the previously stored reference image. And a moving estimator for estimating a moving value using a normalized cross correlation between the edge image of the vehicle extracted from the input surrounding image captured by the calculated rotation angle information and the edge image of the vehicle extracted from the stored reference image. can do.

According to another aspect of the present invention, a method for calibrating a camera installed in a vehicle is provided.

According to an embodiment of the present disclosure, a method of calibrating a camera installed in a vehicle may include extracting an edge image of a vehicle from a continuously captured image of a surrounding vehicle, and extracting an edge image of the vehicle extracted from the input surrounding image of the vehicle. Computing the rotation angle information by using the cross-correlation and dispersion values of the transformed values of the transformed vehicle image of the stored reference image Hough transformed from the input image around the vehicle corrected using the calculated rotation angle information Estimating a moving value using the normalized cross correlation between the extracted frame image of the vehicle and the frame image extracted from the pre-stored reference image.

As described above, according to an exemplary embodiment of the present disclosure, a tolerance of a camera providing different views (images) may be corrected due to a tolerance during mass production.

According to an embodiment of the present invention, a camera installed in a vehicle may be corrected from an image acquired while driving without a pattern installed on the front, rear, left, and right sides.

According to an embodiment of the present invention, there is no need for a device for aligning the pattern with the vehicle, and the camera installed in the vehicle can be calibrated without being affected by the brightness change rate such as light reflection during outdoor shooting.

BRIEF DESCRIPTION OF THE DRAWINGS The figure for demonstrating the correction system of the camera installed in the conventional vehicle.
2 to 6 are views for explaining a correction apparatus of a camera installed in a vehicle according to an embodiment of the present invention.
7 to 13 are views for explaining a calibration method of a camera installed in a vehicle according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art may easily implement the present invention. As those skilled in the art would realize, the described embodiments may be modified in various different ways, all without departing from the spirit or scope of the present invention. In addition, when a part is said to "include" a certain component, it means that it may further include other components, except to exclude other components unless specifically stated otherwise.

2 to 6 are views for explaining a correction apparatus of a camera installed in a vehicle according to an embodiment of the present invention.

2, the camera calibration system installed in a vehicle according to the present invention includes a camera calibration apparatus 100 and a camera 200 installed in a vehicle.

The camera calibrating apparatus 100 receives an image captured by the camera 200 installed in the vehicle, and corrects the camera 200 installed in the vehicle by comparing the input image with a previously stored reference image. Hereinafter, the camera calibrating apparatus 100 will be described in more detail with reference to FIGS. 2 to 6.

The camera 200 installed in the vehicle is installed in the vehicle to photograph the surroundings of the vehicle. The camera 200 installed in the vehicle may be installed in at least one of front, rear, left and right sides to implement the AVM system. The camera 200 installed in the vehicle transmits the surrounding image of the vehicle to the camera calibrating apparatus 100.

Referring to FIG. 3, the camera calibrating apparatus 100 may include an image input unit 110, an error extraction unit 120, a rotation estimating unit 130, a movement estimating unit 140, and a reference image storing unit 150. .

The image input unit 110 inputs continuously captured images of the vehicle, which are continuously photographed from the camera 200 installed in the vehicle.

The host vehicle extracting unit 120 extracts an edge image of the host vehicle from the continuously input surrounding image of the vehicle. The porcelain extractor 120 will be described in more detail below with reference to FIG. 4.

The rotation estimator 130 converts the rotation angle information using cross-correlation and variance values of transform values obtained by Huff transforming the edge image of the vehicle extracted from the input surrounding image of the vehicle and the vehicle edge image of the pre-stored reference image. Calculate. The rotation estimator 130 will be described in more detail later with reference to FIG. 5.

The motion estimator 140 uses a normalized cross correlation between the edge image of the vehicle extracted from the input surrounding image of the vehicle corrected using the calculated rotation angle information and the edge image of the vehicle extracted from the stored reference image. The moving values DELTA X and DELTA Y are estimated. The movement estimator 140 will be described in more detail later with reference to FIG. 6.

The reference image storage unit 150 stores a reference image of the camera 200 installed in the vehicle. Here, the reference image may be an edge image of the vehicle that does not need correction when photographed by the camera 200 installed in the vehicle.

Referring to FIG. 4, the deviation extractor 120 includes an edge extractor 122, a change rate calculator 124, and an error tolerance frame calculator 126.

The edge extractor 122 extracts an arbitrary edge from the input surrounding image captured by the edge extracting method. The edge extracting unit 122 may extract an arbitrary edge from the surrounding image of the vehicle input by using, for example, a canny edge or a sobel edge extraction method.

The change rate calculator 124 calculates a moving average filter value of each pixel value of arbitrary edges extracted from continuously input images of surrounding images of the vehicle.

The edge calculator 126 calculates a pixel having a larger threshold value than the preset threshold value by comparing the calculated moving average filter result value of each pixel value with a preset threshold value. The edge calculator 126 calculates a moving average filter result value between 0 and 1 when the calculated change rate of each pixel value uses a moving average filter result value. Judging by When the moving average filter result value is used, the edge calculating unit 126 may be a candidate group of the edges of the vehicle, where a result value is 0 and 1. The result value 0 may be a background candidate group. Since the edge calculator 126 is the most ideal environment if the margin of error is continuously 1, the moving average filter result is 1. The edge calculating unit 126 is a value that converges to 0 while repeating 1 and 0 when the edge is not an error frame. In other words, the smaller the rate of change, the closer the moving average filter result value is to 1, and the larger the change rate, the closer the moving average filter result value is to 0. Therefore, the edge calculator 126 may set a threshold value and determine a value close to 1 in the moving average filter result value as the own vehicle edge.

Referring to FIG. 5, the rotation estimator 130 includes a Hough transform unit 132, a variance calculator 134, a correlation unit 136, and a rotation calculator 138.

The hough transform unit 132 performs a Hough transform on all pixel points included in the inputted error frame information and all pixel points with respect to the error frame information of the pre-stored reference image. The Hough transform unit 132 may Hough transform coordinate information (x, y) of the pixel point to convert the angle information θ and the distance information ρ.

The variance calculation unit 134 calculates variance information S ² of the angle information θ of the HV transform value of the inputted error order information and the error order information of the reference image.

The correlation unit 136 calculates a cross correlation of the variance information S ² with respect to the angle information θ of the HV transform value of the inputted error frame information and the error frame information of the reference image.

Here, the cross-correlation for estimating the rotational rotation relationship is used as a measure for measuring the similarity between two signals. The two signals to be compared are expressed in polar coordinates, and the two signals are expressed as R1 and R2, respectively. Cross-Correlation is defined in Equation 1 below.

Equation 1

C [ω] = ∑ _{φ = 1} ³⁶⁰ R1 [ω] R2 [ω + φ]

Here, the signal R is a polar coordinate value of the variance information S ² , ω is an angle of the polar coordinate value, and φ is a range for detecting correlation. The signal R means a value obtained by converting the dispersion information S ² of the angle information θ into the polar coordinate system as shown in Equation 2 below.

Equation 2

R [ω] = re ^jφ

 Where r and φ are calculated by

r = S ² (0≤ φ <180), S ² (180≤ φ <360)

φ = θ (0≤ φ <180), 2θ (180≤ φ <360)

The rotation calculator 138 calculates rotation angle information using the calculated cross correlation.

The rotation calculator 138 may calculate rotation angle information by calculating a delay between two signals using Equation 3 below.

Equation 3

는

를 최대로 만드는 ω값을 의미한다.Where is the angular difference between the two signals,

Is

It means the ω value that makes the maximum.

Referring to FIG. 6, the movement estimator 140 includes a rotation corrector 142 and a movement calculator 144.

The rotation correction unit 142 rotates the edge image of the vehicle extracted from the input surrounding image of the vehicle by using the calculated rotation angle information.

The movement calculating unit 144 uses a normalized cross-correlation between the edge image of the vehicle extracted from the rotationally corrected input surrounding image and the vehicle image extracted from the pre-stored reference image, using a normalized cross correlation (ΔX, Δ). Calculate Y).

7 to 13 are views for explaining a calibration method of a camera installed in a vehicle according to an embodiment of the present invention.

Referring to FIG. 7, the method for calibrating a camera installed in a vehicle according to the present invention includes a step of detecting a host vehicle in step S100, a step S200 rotation estimation step, and a step S300 movement estimation step in the calibration device 100 of a camera installed in a vehicle. . It will be described in more detail below.

The host vehicle detection step of step S100 extracts a host vehicle edge image from continuously captured surrounding images of the vehicle.

In the step S200 rotation estimation, rotation angle information is calculated using cross-correlation and dispersion values of transformed values obtained by Huff transforming the edge image of the vehicle extracted from the input surrounding image of the vehicle and the vehicle edge image of the previously stored reference image. do.

In step S300, the moving estimation step estimates a moving value by using a normalized cross correlation between the edge image of the vehicle extracted from the input surrounding image of the vehicle corrected using the calculated rotation angle information and the edge image of the vehicle extracted from the previously stored reference image. do.

Hereinafter, the host vehicle detecting step of step S100 will be described in detail.

In operation S110, the camera calibrating apparatus 100 inputs a continuous vehicle surrounding image.

In operation S120, the calibrating apparatus 100 of the camera detects an edge from a continuously surrounding image of the vehicle.

Referring to FIG. 8, the camera calibrating apparatus 100 detects an edge of a vehicle surrounding image by applying a Sobel edge method in a continuous frame of the input surrounding image of the vehicle.

9 is a view illustrating a frame image of a vehicle surrounding image and a vehicle surrounding image detected by applying the Sobel edge method in a continuous frame of an input vehicle surrounding image according to an exemplary embodiment. Referring to FIG. 9, the edge image of the surrounding image detected by applying the Sobel edge method in a continuous frame of the input surrounding image of the vehicle includes an edge of the vehicle and an edge of the edge of the vehicle, 910 and 920. The data may further include a portion 930 in which data is missing from the edge of the vehicle.

In operation S130, the camera calibrating apparatus 100 calculates a moving average filter result value for each pixel point of the edge images detected from the continuously input surrounding image of the vehicle.

In operation S140, the camera calibrating apparatus 100 compares a moving average filter result value for each pixel point of the calculated edge images with a preset threshold.

Referring to FIG. 8, the camera calibrating apparatus 100 may calculate a rate of change for each pixel point of the edge images calculated using a moving average filter of successive edge images.

In operation S150, the camera calibrating apparatus 100 calculates the own vehicle edge information using pixels having a moving average filter result value for each pixel point of the calculated edge images greater than a preset threshold. This is because the edge of the host vehicle does not change even when the vehicle moves in the continuously captured images of the surrounding images of the vehicle, and only the surrounding environment changes.

Next step S200 rotation estimation step will be described in detail.

In operation S210, the camera calibrating apparatus 100 inputs the calculated host vehicle edge information.

In operation S220, the camera calibrating apparatus 100 hough-transforms all pixel points included in the inputted vehicle edge information. Also, the camera calibrating apparatus 100 huff-transforms all pixel points with respect to the own vehicle edge information of the previously stored reference image.

In operation S230, the camera calibrating apparatus 100 calculates variance information S ² of the angle information θ of the Hough transform value of the inputted host vehicle edge information and the host vehicle edge information of the reference image.

Referring to FIG. 10, the calibrating apparatus 100 of the camera may include angle information θ and distance information ρ for all pixel point (x, y) coordinate information included in the input data of the deviation and the deviation of the reference image. Hough transform to the &lt; RTI ID = 0.0 &gt;), &lt; / RTI &gt;

Referring to FIG. 11, the camera calibrating apparatus 100 may convert one pixel point of an edge into one curve with respect to the host vehicle edge information 1110 and the input host vehicle edge information 1120 of the reference image (1130). 1140).

In operation S240 and step S250, the camera calibration apparatus 100 converts the dispersion information S ² of the angle information θ of the HV transform value of the input data of the deviation and the deviation of the reference image into a polar coordinate system. The cross angle is calculated by calculating cross-correlation.

Referring to FIG. 12, the camera calibrating apparatus 100 may perform dispersion information S on the angle information θ of the Hough transform value of the calibrated edge information calculated by the camera calibrating apparatus 100 and the calibrated edge information of the reference image. ² ) can be displayed and converted to the polar coordinate system and cross-correlation can be calculated to calculate rotation angle information of the inputted edge information and the own-order edge information of the reference image.

Next, the step S300 movement estimation step will be described in detail with reference to FIG. 7 again.

In operation S310, the correction apparatus 100 of the camera rotates and corrects the edge image of the vehicle extracted from the input surrounding image of the vehicle by using the calculated rotation angle information.

In steps S320 and S330, the camera correction apparatus 100 performs a two-dimensional image by using a normalized cross correlation between the edge image of the vehicle extracted from the rotationally corrected input surrounding image of the vehicle and the edge image of the vehicle extracted from a previously stored reference image. The moving values DELTA X and DELTA Y are calculated.

Referring to FIG. 13, the calibrating apparatus 100 of the camera may correct the surrounding image of the vehicle, which is input while driving the vehicle, as the reference image, using the calculated rotation angle information and the movement value.

Combinations of each block of the block diagrams and respective steps of the flowcharts attached to the present invention may be performed by computer program instructions. These computer program instructions may be embedded in an encoding processor of a general purpose computer, special purpose computer, or other programmable data processing equipment such that the instructions executed by the encoding processor of the computer or other programmable data processing equipment are not included in each block or block diagram. It will create means for performing the functions described in each step of the flowchart. These computer program instructions may be stored in a computer usable or computer readable memory that can be directed to a computer or other programmable data processing equipment to implement functionality in a particular manner, and thus the computer usable or computer readable memory. It is also possible for the instructions stored in to produce an article of manufacture containing instruction means for performing the functions described in each block or flowchart of each step of the block diagram. Computer program instructions may also be mounted on a computer or other programmable data processing equipment, such that a series of operating steps may be performed on the computer or other programmable data processing equipment to create a computer-implemented process to create a computer or other programmable data. Instructions that perform processing equipment may also provide steps for performing the functions described in each block of the block diagram and in each step of the flowchart.

In addition, each block or step may represent a portion of a module, segment or code that includes one or more executable instructions for executing a specified logical function. It should also be noted that in some alternative embodiments, the functions noted in the blocks or steps may occur out of order. For example, the two blocks or steps shown in succession may in fact be executed substantially concurrently or the blocks or steps may sometimes be performed in the reverse order, depending on the functionality involved.

As such, those skilled in the art will appreciate that the present invention can be implemented in other specific forms without changing the technical spirit or essential features thereof. Therefore, the embodiments described above are to be understood as illustrative and not restrictive in all aspects. The scope of the present invention is shown by the following claims rather than the detailed description, and all changes or modifications derived from the meaning and scope of the claims and their equivalents should be construed as being included in the scope of the present invention. .

Claims (11)

In the camera calibration device installed in a vehicle,
An image input unit configured to input continuously photographed surrounding images of the vehicle;
A self-extraction unit extracting an edge-of-vehicle edge image from continuously captured surrounding images of the vehicle;
A rotation estimating unit configured to calculate rotation angle information by using cross-correlation and dispersion values of transformed values obtained by transforming a vehicle frame image extracted from the input surrounding image of the vehicle and the vehicle frame image of the pre-stored reference image; And
The edge image of the vehicle extracted from the input surrounding image of the vehicle is rotated and corrected using the rotation angle information, and the edge image of the vehicle extracted from the input rotational correction image of the corrected vehicle is extracted from the frame image and the pre-stored reference image. And a moving estimator for estimating a moving value using a normalized cross correlation of the edge image.
The method of claim 1,
And the continuously captured surrounding image of the vehicle is input through a camera installed in the vehicle while the vehicle is driving.
The method of claim 1,
The host tea extracting unit
An edge extracting unit extracting an arbitrary edge from the input surrounding image of the vehicle through an edge extracting method;
A change rate calculator configured to calculate a moving average filter result value of each pixel value of arbitrary edges extracted from consecutively captured surrounding images of the vehicle; And
And an edge calculator configured to calculate, as an edge of the car, a pixel having a calculated moving average filter value of each pixel value greater than a preset threshold value.
The method of claim 3, wherein
The edge extracting unit,
Compensation apparatus for a camera installed in a vehicle for extracting an arbitrary edge from the surrounding image captured by the Sobel edge extraction method.
The method of claim 1,
The rotation estimation unit,
A Hough transform unit that huff-transforms all pixel points included in the inputted autonomous frame information and all pixel points with respect to the pre-stored autonomous frame information;
A variance calculator configured to calculate variance information of angle information of the HV transform value of the inputted host vehicle edge information and the host vehicle edge information of the reference image;
A correlation unit configured to calculate a cross correlation between the variance information on the angle information of the inputted error range information and the Hough transform value of the error range information of the reference image; And
And a rotation calculator configured to calculate rotation angle information using the calculated cross correlation.
The method of claim 1,
The movement estimating unit
A rotation correction unit configured to rotate and correct the edge image of the vehicle extracted from the input surrounding image of the vehicle using the calculated rotation angle information; And
Correction apparatus for a camera installed in a vehicle including a moving calculation unit for calculating a moving value by using normalized cross correlation between the edge image of the vehicle extracted from the rotationally corrected input of the surrounding image of the vehicle and the frame image extracted from the pre-stored reference image. .
In the camera calibration method performed in the camera calibration device installed in a vehicle,
Extracting an edge of an edge of a vehicle from continuously captured surrounding images of the vehicle;
Calculating rotation angle information by using cross-correlation and variance values of transform values obtained by Hough transforming the edge image of the vehicle extracted from the input surrounding image of the vehicle and the vehicle edge image of the previously stored reference image; And
The edge image of the vehicle extracted from the input surrounding image of the vehicle is rotated and corrected using the rotation angle information, and the edge image of the vehicle extracted from the input rotational correction image of the corrected vehicle is extracted from the frame image and the pre-stored reference image. Estimating a moving value using a normalized cross correlation of the edge image.
The method of claim 7, wherein
Extracting the host vehicle edge image,
Detecting an edge of the surrounding image of the vehicle by applying the Sobel edge method to a continuous frame of the input surrounding image of the vehicle;
Calculating a moving average filter result value for each pixel point of edge images detected from continuously inputted surrounding images of the vehicle;
Comparing a moving average filter result value for each pixel point of the calculated edge images with a preset threshold value; And
Comprising a comparison result, the method of calibrating the camera installed in the vehicle comprising the step of calculating the own vehicle edge information using pixels whose moving average filter result value for each pixel point of the edge image is larger than a predetermined threshold value.
The method of claim 7, wherein
Computing the rotation angle information,
Hough transforming all the pixel points included in the inputted vehicle edge information;
Calculating variance information of angle information of the HV transform value of the inputted host vehicle edge information and the host vehicle edge information of the reference image;
Compensating the camera installed in the vehicle comprising the step of converting the variance information of the angle information of the HV transform value of the input of the vehicle edge information and the reference image of the reference image to the polar coordinate system, and calculating the cross-correlation Way.
The method of claim 7, wherein
Estimating the shift value,
Rotating correction of the edge image of the vehicle extracted from the input surrounding image of the vehicle by using the calculated rotation angle information;
And calculating a moving value using a normalized cross-correlation of the edge image of the vehicle extracted from the rotation-corrected input surrounding image of the vehicle and the frame image of the vehicle extracted from the pre-stored reference image.
A non-transitory computer-readable recording medium storing a computer program for executing the method of any one of claims 7 to 10.

Images