KR20140136810A - Image correction apparatus and method - Google Patents

Abstract

An image correction device according to an embodiment of the present invention comprises: an image acquisition unit to acquire an image by photographing a lattice shape which is formed on the ground and includes a plurality of horizontal lines parallel to an X-axis of a rectangular coordinate system and a plurality of vertical lines perpendicular to the horizontal lines; a line detection unit to extract a linear component from the image; a feature extraction unit to extract a plurality of feature parts from the linear component; a correction amount calculation unit to calculate a correction amount based on the feature parts; and an image conversion unit to perform correction of the image based on the correction amount.

Description

IMAGE CORRECTION APPARATUS AND METHOD BACKGROUND OF THE INVENTION 1. Field of the Invention [0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a video correction apparatus and a video correction method, and more particularly, to an apparatus and method for automatically correcting an image obtained by a camera in an AVM (Around View Monitoring) system.

Recently, the automobile technology has been developed to improve driver convenience and safety. Various facilities and apparatus and systems for operating and controlling the same have been developed. In particular, a video camera system is used in a wide range of automobiles. For example, by installing a camera outside the automobile and using it, a variety of systems are available, such as a driver checking an external image around the vehicle, recognizing objects and lanes around the vehicle, and alerting the driver through the display of the driver's instrument panel Developed and applied.

Among these, the surrounding view monitoring system installs a plurality of cameras on the outside of the vehicle and photographs the surroundings of the vehicle, thereby performing a function of assisting the driver in driving and parking. The output image of the surround view monitoring system is synthesized into a single Bird View image as seen from the top of the vehicle through the image processing process and finally displayed.

In order for the output image of the surround view monitoring system to be used for the purpose of accurately grasping the parking support or the vehicle surroundings information to the driver, each camera image must be synthesized with high precision. For this purpose, Should have. Therefore, it is indispensable to physically correct the installation position of the camera in the production process of the vehicle or to correct the conversion parameter value for image synthesis by software.

As a prior art for solving such a correction work by software, there have been proposed camera calibration methods for converting an image by performing image processing in an ECU so that a corrected image is generated even if the image is misaligned due to a camera installation tolerance. These methods sequentially calculate an error corresponding to a roll, a pitch, and a yaw corresponding to a rotation about three axes of a three-dimensional Cartesian coordinate system of the camera, The projection parameters are obtained and the rotation matrix of the corresponding camera is calculated to transform the image, thereby correcting the misalignment.

However, in this case, there is a problem that the correction work is required to be performed manually by the operator while viewing the display screen, or requires additional tasks such as installing a separate target apparatus, mark, or equipment for the calibration work at a predetermined position around the vehicle . These additional requirements may require a significant amount of time and labor to perform calibration work on a vehicle assembly line or in a garage and, as a result, may increase the cost of vehicle service.

The patent document described in the following prior art documents discloses a technique for providing a calibration method and a calibration program for a camera mounted on a vehicle and discloses a technical matter of the present invention for obtaining a grid image and calculating an error correction amount in each direction It is not.

Korea Published Patent 2011-0116243

According to an aspect of the present invention, there is provided a video correction apparatus and a video correction method, which solve the above-described problems.

It is an object of one embodiment of the present invention to provide a method for performing calibration of a camera image relatively quickly, simply and automatically in a vehicle assembly line or a vehicle repair shop.

The solution to the problem of the present invention is not limited to those mentioned above, and other solutions not mentioned can be clearly understood by those skilled in the art from the following description.

The image correcting apparatus according to an embodiment of the present invention includes a plurality of horizontal lines formed on the ground and parallel to the X axis of the rectangular coordinate system and a plurality of vertical lines orthogonal to the plurality of horizontal lines, A straight line detecting unit for extracting a straight line component from the image, a feature extracting unit for extracting a plurality of features from the straight line component, a correction amount calculating unit for calculating a correction amount based on the plurality of features, And an image converting unit for performing the correction of the image based on the correction amount.

Wherein the feature extracting unit includes a roll feature extracting unit for extracting a first line and a second line from the components corresponding to the plurality of horizontal lines among the straight line components and a feature extracting unit for extracting, from a component corresponding to the plurality of vertical lines, And a pitch characteristic extracting unit for extracting the third line and the fourth line.

Wherein the correction amount calculating unit includes a roll correction amount calculating unit for calculating a correction amount in the roll direction based on the first line and the second line, a roll correction amount calculating unit for calculating, based on the third line and the fourth line, A pitch correction amount calculation unit for calculating a correction amount in the direction of the pitch; And a correction amount calculating unit for calculating a correction amount in the Yaw direction based on an angle formed by the roll correction amount calculating unit and the pitch correction amount calculating unit and an angle formed by the fifth line included in the linear component corrected by the pitch correction amount calculating unit and the X axis of the rectangular coordinate system, And the corrected straight line component may be a straight line component corrected by the roll correction amount calculating section and the pitch correction amount calculating section.

The roll correction amount calculating unit calculates a correction amount in the roll direction by a difference between the length of the first line and the length of the second line, and the pitch correction amount calculating unit calculates the correction amount of the roll, It is also possible to calculate the amount of correction in the pitch direction by the difference between the length of the first line and the length of the fourth line.

According to another aspect of the present invention, there is provided a method of correcting an image, comprising the steps of: capturing a grid shape including a plurality of horizontal lines parallel to the X-axis of a rectangular coordinate system and a plurality of vertical lines orthogonal to the plurality of horizontal lines, A straight line detecting step of extracting a straight line component from a contour of the image, a roll correcting step of calculating a correction amount of a roll direction based on the straight line component, A pitch correction step of calculating a correction amount in the pitch direction based on the amount of correction in the yaw direction based on the linear component, the correction amount in the roll direction, and the correction amount in the pitch direction, And a yaw calibration step of calculating a yaw calibration value.

The roll calibration step may include a roll direction feature extraction step of extracting a first line and a second line from the components corresponding to the plurality of horizontal lines in the straight line component, It is possible to include a roll correction amount calculating step of calculating a correction amount of the roll direction based on the line.

It is preferable that the roll correction amount calculating step calculates the correction amount in the roll direction by the difference between the length of the first line and the length of the second line.

The pitch calibration step may include a pitch direction feature extraction step of extracting a third line and a fourth line from the components corresponding to the plurality of horizontal lines in the straight line component, It is possible to include a pitch correction amount calculating step of calculating a correction amount of the pitch direction based on the line.

It is preferable that the pitch correction amount calculating step calculates the correction amount in the pitch direction by the difference between the length of the third line and the length of the fourth line.

The yaw correction step may be based on an angle formed by the roll correction amount calculating unit and the fifth line included in the straight line component corrected by the pitch correction amount calculating unit and the X axis of the rectangular coordinate system. It is preferable to calculate the correction amount in the yaw direction.

An image conversion determination step of comparing the correction amount of the roll direction, the correction amount of the pitch direction, and the correction amount of the yaw direction with the preset reference values after the yaw correction step, Further comprising an image conversion step of outputting a converted image based on the correction amount when at least one of a correction amount in a roll direction, a correction amount in a pitch direction, and a correction amount in a yaw direction is equal to or greater than the reference value It is possible to do.

The present invention is capable of correcting image distortion due to camera misalignment relatively quickly and precisely by image conversion in an automobile assembly line or a car garage. In addition, there is no need to install a separate mark or facility around the automobile in a fixed position, and the user does not have to perform passive tasks such as selecting or setting correction coordinates on the output screen for calibration. Therefore, the cost of vehicle service can be reduced since no professional manpower for calibration is required.

Further, even if there is a change in the camera mounting position due to a vehicle aging or an accident, it is possible to easily calibrate the output image only if there is a space for parking a vehicle having a grid-like straight line at a car repair shop.

The effects of the present invention are not limited to those mentioned above, and other effects not mentioned may be clearly understood by those skilled in the art from the following description.

1 is a block diagram of a video correction apparatus according to an embodiment of the present invention.
2 is a diagram showing a coordinate system for explaining the present invention.
3 is a diagram for explaining a method of acquiring a grid image in the image acquiring unit of the present invention.
4A to 4C are diagrams for explaining the functions of the feature extraction unit and the correction amount calculation unit of the present invention.
5 is a flowchart illustrating a method of correcting an image according to another embodiment of the present invention.

It is noted that the technical terms used herein are used only to describe specific embodiments and are not intended to limit the invention. It is also to be understood that the technical terms used herein are to be interpreted in a sense generally understood by a person skilled in the art to which the present invention belongs, Should not be construed to mean, or be interpreted in an excessively reduced sense. Further, when a technical term used herein is an erroneous technical term that does not accurately express the spirit of the present invention, it should be understood that technical terms that can be understood by a person skilled in the art are replaced. In addition, the general terms used in the present invention should be interpreted according to a predefined or prior context, and should not be construed as being excessively reduced.

Also, the singular forms "as used herein include plural referents unless the context clearly dictates otherwise. In the present application, the term "comprising" or "comprising" or the like should not be construed as necessarily including the various elements or steps described in the specification, Or may be further comprised of additional components or steps.

Also, suffixes "module", "unit" and "part" for the components used in the present specification are given or mixed in consideration of ease of specification, and each component having its own distinct meaning or role no.

Furthermore, terms including ordinals such as first, second, etc. used in this specification can be used to describe various elements, but the elements should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings, wherein like reference numerals refer to like or similar elements throughout the several views, and redundant description thereof will be omitted.

In the following description, well-known functions or constructions are not described in detail since they would obscure the invention in unnecessary detail. It is to be noted that the accompanying drawings are only for the purpose of facilitating understanding of the present invention, and should not be construed as limiting the scope of the present invention with reference to the accompanying drawings.

Hereinafter, a video correction apparatus according to an embodiment of the present invention will be described in detail with reference to FIGS. 1 to 4. FIG. 1, the image correction apparatus includes an image acquisition unit 100, a linear detection unit 200, a feature extraction unit 300, a correction amount calculation unit 400, and an image conversion unit 500 do.

Before discussing each configuration in detail, the orientation of the camera defined in the present invention will be described. As shown in Fig. 2, the orientation of the camera can generally be obtained by rotating the camera about an axis of a three-dimensional Cartesian coordinate system. The displacements of the rotational angles about the X, Y, and Z axes of the Cartesian coordinate system are represented by Roll, Pitch, and Yaw, respectively. With respect to the orientation of the camera mounted on the present invention, as shown in FIG. 3, the direction in which the camera looks is perpendicular to the X-axis and the X-axis of the orthogonal coordinate system, and the direction parallel to the ground is defined as the Y- The direction perpendicular to the Y axis is defined as the Z axis.

The image acquiring unit 100 acquires an image by photographing a grid shape including a plurality of horizontal lines parallel to the X-axis of the orthogonal coordinate system and a plurality of vertical lines orthogonal to the plurality of horizontal lines. Specifically, after the camera for the Around View Monitoring (AVM) system is mounted on the vehicle assembly line, the vehicle is aligned to the position where the grid shape is painted for tolerance correction. At this time, if the camera is arranged in the right and the forward direction, the synthesized image is output, otherwise, the misaligned or distorted image will be outputted according to the error axis.

The straight line detecting unit 200 performs an image processing process of extracting a straight line component from an image by the image obtaining unit 100. At this time, it is also possible to detect the outline of the image by applying the Canny Edge Detection algorithm. A method for extracting a straight line from an image can be performed by Hough Transform. Since the algorithms described above are commonly used image processing algorithms, a detailed description thereof will be omitted herein.

The feature extraction unit (300) extracts a plurality of feature portions for calculating correction amounts among straight line components. The feature extraction unit 300 specifically includes a roll feature extraction unit 310 for extracting a first line L1 and a second line L2 from components corresponding to the plurality of horizontal lines among the linear components, And a pitch feature extraction unit 320 for extracting a third line L3 and a fourth line L4 from the components corresponding to the plurality of vertical lines among the straight line components.

Referring to FIG. 4A, the roll feature extraction unit 310 extracts, from the linear component detected from the image, an outermost line of a straight line corresponding to a plurality of vertical lines parallel to the Y-axis of the rectangular coordinate system Lt; RTI ID = 0.0 > ly1 < / RTI > Thereafter, ly1 and ly2 are connected and the first line L1 and the second line L2 corresponding to the horizontal line of the lattice shape are extracted to the feature for error correction in the roll direction.

Referring to FIG. 4 (b), the pitch feature extracting unit 320 extracts a plurality of horizontal lines corresponding to a plurality of horizontal lines parallel to the X axis of the rectangular coordinate system, Lt; RTI ID = 0.0 > lx1 < / RTI > Then, the third line L3 and the fourth line L4 corresponding to the lattice-shaped vertical lines are connected to the feature for error correction in the pitch direction.

The correction amount calculating unit 400 calculates a correction amount based on the plurality of features. The correction amount calculating unit 400 specifically includes a roll correction amount calculating unit 410, a pitch correction amount calculating unit 420 and a yaw correction amount calculating unit 430. [

The roll correction amount calculating unit 410 calculates the amount of correction in the roll direction based on the first line L1 and the second line L2 extracted by the roll feature extracting unit 310 . When a camera mounting tolerance occurs in a roll, which is a rotation centered on the X axis of the orthogonal coordinate system, a change occurs in the ground gradient of the input image due to a change in the angle at which the camera looks at the ground. Therefore, a difference occurs in the lengths of the first line L1 and the second line L2. By the difference between the length of the first line L1 and the length of the second line L2, Can be calculated.

The pitch correction amount calculation unit 420 calculates a correction amount in the pitch direction based on the third line L3 and the fourth line L4 extracted by the pitch feature extraction unit 320 . When a camera mounting tolerance occurs at a pitch that is a rotation centered on the Y axis of the Cartesian coordinate system, difference in the lengths of the third and fourth lines L3 and L4, like the feature in the roll direction, And the amount of correction in the pitch direction can be calculated by the difference between the length of the third line L3 and the length of the fourth line L4.

The yaw correction amount calculating unit 430 calculates the yaw correction amount of the fifth line L5 included in the straight line component corrected by the roll correction amount calculating unit 410 and the pitch correction amount calculating unit 420 . Therefore, the correction amount calculation in the yaw direction is performed after calculating the correction amount in the roll direction and after calculating the correction amount in the pitch direction. Then, as shown in Fig. 4 (c), the amount of correction in the yaw direction is calculated based on the angle [theta] formed by the fifth line L5 and the X axis of the orthogonal coordinate system. In some cases, it is also possible to calculate the correction amount in the yaw direction based on the angle formed by the fifth line L5 and the Y-axis of the orthogonal coordinate system.

The image converting unit 500 performs the correction of the image based on the correction amount calculated from the correction amount calculating unit 400. [ Specifically, after setting the rotation parameter based on the correction amount, the image is converted based on the rotation parameter.

In the case of correcting the roll error, if the difference between the first line L1 and the second line L2 is positive, it is determined that the roll rotation angular displacement has occurred in the positive direction, Lt; RTI ID = 0.0 > parameter < / RTI > On the contrary, if the difference between the first line L1 and the second line L2 is negative, it is determined that the roll rotation angle displacement has a tolerance in the negative direction, and a positive rotation parameter value for the X axis is set . It is also possible to set in advance the amount of change of the first line L1 and the second line L2 in accordance with the change of 1 DEG of the roll rotation angle in order to speed up the calibration time.

When calibrating the pitch error, set the rotation parameter value for the Y axis as well as correcting the roll error.

In the case of correcting the yaw error, the rotation parameter value is not set as a rotation parameter value by a separate operation as in the case of correcting the error of the roll and the pitch, The output of the correction amount calculating section can be used as it is. That is, if the yaw correction amount? Calculated by the yaw correction amount calculating unit is out of the tolerance range based on 0 degree, it is determined that the mounting tolerance has occurred. If the calculated yaw correction amount [theta] is 1 [deg.] To 90 [deg.], A negative parameter value is set for the Z axis, and if it is 90 [deg.] To 180 [

Hereinafter, an image correction method according to another embodiment of the present invention will be described with reference to FIG.

The image correction method according to another exemplary embodiment of the present invention includes an image acquisition step S100, a straight line detection step S200, a roll correction step S300, a pitch correction step S400, Yaw) correction step S500, an image conversion determination step S600, and an image conversion step S700.

The image acquiring step S100 is a step of acquiring an image by photographing a grid shape formed on the ground and including a plurality of horizontal lines parallel to the X axis of the rectangular coordinate system and a plurality of vertical lines orthogonal to the plurality of horizontal lines to be. The linear detection step S200 is a step of extracting a linear component from the contour of the image. The details of the image acquiring step (S100) and the straight line detecting step (S200) are the same as those of the image acquiring unit (100) and the straight line detecting unit (200).

The details of the roll calibration step S300, the pitch calibration step S400 and the yaw calibration step S500 are the same as those of the feature extraction of the image correction device according to the embodiment of the present invention (300) and the correction amount calculation unit (400). 5, the image correction method according to another exemplary embodiment of the present invention includes a roll calibration step S300, a pitch calibration step S400, and a yaw calibration step S500 Progresses sequentially.

After the Yaw calibration step S500, an image conversion determination step S600 for comparing the correction amount of the roll direction, the correction amount of the pitch direction, and the correction amount of the yaw direction with preset reference values, ). If all of the correction amounts in the three directions are smaller than a preset reference value, it is determined that there is no mounting error of the camera, and the image correction is terminated. Conversely, when at least one of the correction amounts in the three directions is equal to or greater than the reference value, the image is converted based on the correction amount. The image conversion step S700 is performed by the image conversion unit 500 described above, and the details of the image conversion step S700 are the same as those of the image conversion unit 500 described above. However, the converted image is fed back to the image input unit 100 again to enter the image acquiring step (S100) to calculate the correction amount so that the image correction is continuously performed until the correction amount for all the axes satisfies the tolerance range It is also possible.

The embodiments and the accompanying drawings described in the present specification are merely illustrative of some of the technical ideas included in the present invention. Therefore, it is to be understood that the embodiments disclosed herein are not intended to limit the scope of the present invention but to limit the scope of the present invention.

It will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims and their equivalents. It should be interpreted.

100: image acquisition unit 200: linear detection unit
300: Feature extraction unit 310: Roll feature extraction unit
320: Pitch feature extraction unit 400:
410: Roll correction amount calculating unit 420: Pitch correction amount calculating unit
430 Yaw correction amount calculating unit 500:
L1: first line L2: second line
L3: third line L4: fourth line
L5: fifth line [theta]: Yaw correction amount
S100: Image acquisition step
S200: Straight line detection step
S300: Roll calibration step
S310: Roll direction feature extraction step
S320: Roll correction amount calculating step
S400: Pitch calibration step
S410: Pitch direction feature extraction step
S420: Pitch correction amount calculating step
S500: Yaw calibration step
S600: image conversion judgment step
S700: Image conversion step

Claims (11)

An image acquiring unit which is formed on the ground and photographs a grid shape including a plurality of horizontal lines parallel to the X axis of the orthogonal coordinate system and a plurality of vertical lines orthogonal to the plurality of horizontal lines to obtain an image;
A linear detector for extracting a linear component from the image;
A feature extraction unit for extracting a plurality of features from the linear components;
A correction amount calculating unit for calculating a correction amount based on the plurality of features; And
An image converting unit for performing the correction of the image based on the correction amount;
The image correction device comprising:
The method according to claim 1,
The feature extraction unit may extract,
A roll feature extracting unit for extracting a first line and a second line from the components corresponding to the plurality of horizontal lines among the linear components; And
A pitch feature extraction unit for extracting a third line and a fourth line from the components corresponding to the plurality of vertical lines among the linear components;
The image correction device comprising:
3. The method of claim 2,
Wherein the correction-
A roll correction amount calculating unit for calculating a correction amount in a roll direction based on the first line and the second line;
A pitch correction amount calculating unit for calculating a correction amount in a pitch direction based on the third line and the fourth line; And
A correction amount in the Yaw direction based on an angle formed by the roll correction amount calculating section and the fifth line included in the straight line component corrected by the pitch correction amount calculating section and the X axis of the rectangular coordinate system, A yaw correction amount calculating unit for calculating a yaw correction amount;
The image correction device comprising:
The method of claim 3,
The roll correction amount calculating unit may calculate,
A correction amount in the roll direction is calculated by a difference between the length of the first line and the length of the second line,
The pitch correction amount calculating unit may calculate,
And calculates a correction amount in a pitch direction by a difference between the length of the third line and the length of the fourth line.
An image acquiring step of acquiring an image by photographing a grid shape formed on the ground and including a plurality of horizontal lines parallel to the X-axis of the rectangular coordinate system and a plurality of vertical lines orthogonal to the plurality of horizontal lines;
A straight line detecting step of extracting a straight line component from an outline of the image;
A roll calibration step of calculating a correction amount in a roll direction based on the linear component;
A pitch correction step of calculating a correction amount in a pitch direction based on the linear component; And
A yaw calibration step of calculating a correction amount in a yaw direction based on the straight line component, the correction amount in the roll direction, and the correction amount in the pitch direction;
The method comprising the steps of:
6. The method of claim 5,
The roll calibration step may include:
A roll direction feature extraction step of extracting a first line and a second line from the components corresponding to the plurality of horizontal lines among the linear components; And
A roll correction amount calculating step of calculating a correction amount in a roll direction based on the first line and the second line;
The method comprising the steps of:
The method according to claim 6,
The roll correction amount calculating step may include:
And a correction amount in a roll direction is calculated by a difference between a length of the first line and a length of the second line.
6. The method of claim 5,
The pitch calibration step may include:
Extracting a third line and a fourth line from the components corresponding to the plurality of horizontal lines among the linear components; And
A pitch correction amount calculating step of calculating a correction amount in a pitch direction based on the third line and the fourth line;
The method comprising the steps of:
9. The method of claim 8,
Wherein the step of calculating the pitch correction amount includes:
The correction amount of the pitch direction is calculated by the difference between the length of the third line and the length of the fourth line.
6. The method of claim 5,
The yaw calibration step may include:
A correction amount in the Yaw direction based on an angle formed by the roll correction amount calculating section and the fifth line included in the straight line component corrected by the pitch correction amount calculating section and the X axis of the rectangular coordinate system, Producing video correction method.
6. The method of claim 5,
After the Yaw calibration step,
An image conversion determining step of comparing the correction amount of the roll direction, the correction amount of the pitch direction, and the correction amount of the yaw direction with preset reference values, respectively; And
An image converting step of outputting a converted image based on the correction amount when at least one of a correction amount in the roll direction, a correction amount in the pitch direction, and a correction amount in the yaw direction is equal to or greater than the reference value;
The method further comprising:

Images