KR102119388B1 - AVM system and camera calibration method - Google Patents

Abstract

An AVM system and camera tolerance correction method are disclosed. The AVM system installed in the vehicle includes a plurality of cameras respectively installed at the front, right, left, and rear of the vehicle, and photographs surroundings while the vehicle is moving so as to pass between a plurality of circular patterns disposed on the ground. An image generator that generates camera image data; And extracting the outline of the detected circular pattern by detecting a circular pattern from each of the camera image data generated while the vehicle is moving, and using the correlation between the extracted outline and the circular pattern disposed on the ground to each of the cameras. And a tolerance correction unit calculating a homography matrix between the corresponding camera image data and the top view image.

Description

AVM system and camera calibration method

The present invention relates to an AVM system and a camera tolerance correction method.

In general, the driver's watch on the inside of the vehicle mainly faces forward, and the driver's left and right and rear watches have a very limited watch because they are largely obscured by the vehicle body.

In order to solve this problem, a clock assisting means such as a side mirror is used, and recently, technologies using camera means for photographing the exterior of the vehicle and providing it to the driver are also being applied to the vehicle in various ways.

For example, there is an Around View Monitoring (AVM) system that shows a 360-degree omnidirectional image around a vehicle by mounting a plurality of cameras on the vehicle. The AVM system eliminates blind spots by combining the images around the vehicle captured by the cameras provided at each location of the vehicle, and providing AVM images in the form of a Top View image that looks like the driver is looking at the vehicle from the sky. And it has the advantage that the driver can easily see the obstacles around the vehicle.

The AVM system provided in the vehicle includes a plurality of cameras respectively provided on the front, rear, right, and left sides of the vehicle, and tolerance correction is performed to eliminate mismatches between images captured by the individual cameras.

1 illustrates a method for correcting tolerances according to the prior art. As illustrated in FIG. 1, a camera (20-1, 20-2, 20-3, 20-4, hereinafter collectively referred to as 20 when distinction is unnecessary) is mounted on each of the front, right, left, and rear positions. The vehicle 10 is stopped at a predetermined position in a space in which a plurality of correction plates 50 are installed for tolerance correction.

Each of the correction plates 50 is positioned at a predetermined distance from each wheel axis of the vehicle in the front left corner, front right corner, rear left corner, and rear right corner of the vehicle 10.

The correction plate 50 is formed of a grid-like pattern to prevent an error in extraction of the feature points, and the grid-like pattern is made of a combination of colors having strong color contrast.

For tolerance correction, coordinate information for feature points of the polygon pattern in the correction plate 50 is extracted from the captured image generated by each camera 20, and the coordinate information is compared with standard coordinate information corresponding to each feature point to convert the matrix This is calculated.

At this time, the world pattern should be arbitrarily assigned to the polygonal pattern. In order to provide the world coordinates, there is a limitation in that all the relative information of the polygonal pattern located in the world coordinates, such as the distance between the feature points or the positional relationship between the feature points, has been limited. In order to generate the top-view image, these limitations require that polygonal patterns be installed according to a set standard, and there is also inconvenience in entering or grasping pattern information.

Korean Registered Patent No. 10-0948886 (tolerance correction device and method for camera installed in a vehicle) United States Patent Publication US2009/0010630 (Camera system and method of correcting camera fitting errors)

In the present invention, a top view image can be generated when a pattern satisfies a condition in which an image appears without inputting the world coordinates of the pattern using a circular pattern rather than a polygonal pattern, and a homography matrix calculated for top view generation ) To provide an AVM system that can accurately estimate the external parameters of the camera and a camera tolerance correction method.

In the present invention, it is possible to shorten the time required to install the pattern by correcting the tolerance by simply moving the vehicle to pass between the patterns, and AVM capable of synthesizing images of multiple cameras even if the relative and absolute positions of the camera and the pattern are not known. It is intended to provide a method for correcting a system and camera tolerance.

Objects other than the present invention will be easily understood through the following description.

According to an aspect of the present invention, in an AVM system installed in a vehicle, the vehicle includes a plurality of cameras respectively installed in front, right, left, and rear of the vehicle, and passes through a plurality of circular patterns disposed on the ground. An image generator that photographs surroundings while the vehicle is moving to generate camera image data; And extracting the outline of the detected circular pattern by detecting a circular pattern from each of the camera image data generated while the vehicle is moving, and using the correlation between the extracted outline and the circular pattern disposed on the ground to each of the cameras. An AVM system including a tolerance correction unit for calculating a homography matrix between corresponding camera image data and a top view image is provided.

The tolerance correction unit analyzes the area and ground of the circular pattern in the top view image to which the currently calculated homography matrix is applied, so that the homography matrix for each camera is calculated using only camera image data that satisfies a predetermined validity condition. (A) calculating an error value for each camera image data using a predetermined error value calculation technique to use a relationship between an actual area of a circular pattern disposed in; And if there is camera image data having a calculated error value greater than a predetermined threshold, calculating the homography matrix again by excluding the corresponding camera image data and replacing the currently calculated homography matrix (b). It repeats until there is no camera image data whose error value is greater than a predetermined threshold value, and if there is no camera image data whose calculated error value is larger than a predetermined threshold value, the currently calculated homograph matrix at the time is determined. It can be implemented to determine the homography matrix for the corresponding camera.

에 의해 산출될 수 있다. 여기서 상기 e는 에러값, 상기 p는 각 카메라 영상 데이터를 구분하는 식별 정보, 상기 n은 하나의 카메라 영상 데이터에서 검출된 원형 패턴의 총 갯수, k는 해당 식별 정보의 카메라 영상 데이터에서 검출된 각 원형 패턴의 구분자, M은 검출된 원형 패턴에 상응하여 지면에 배치된 원형 패턴의 실제 면적, m은 상기 식별 정보의 카메라 영상 데이터에서 검출되어 현재 산출된 호모그라피 행렬에 의해 변환된 탑뷰 영상 내의 원형 패턴의 해석 면적일 수 있다.The error value is an equation

Can be calculated by Here, e is an error value, p is identification information for distinguishing each camera image data, n is the total number of circular patterns detected from one camera image data, and k is each detected from camera image data of the corresponding identification information. The separator of the circular pattern, M is the actual area of the circular pattern disposed on the ground corresponding to the detected circular pattern, m is the circular shape in the top view image detected by the camera image data of the identification information and converted by the currently calculated homography matrix. It may be an analysis area of a pattern.

The plurality of circular patterns may be arranged to be distributed around the path of the vehicle.

In order to quickly detect a circular pattern in each camera image data, the tolerance correcting unit first detects a candidate group of a circular hough transform (CHT) using a pair of gradient vectors, and then detects the detected candidate group. A circular pattern can be detected using CHT as a target.

The AVM system generates a top view image corresponding to each camera using a homography matrix corresponding to each camera and camera image data, and matches each generated top view image to the installation position of each camera based on the vehicle. After being rotated so as to, it may further include an image synthesizing unit for synthesizing AVM image data for the entire surroundings of the vehicle.

When a common circular pattern exists in a plurality of top-view images at the same time, the image synthesizing unit may synthesize the AVM image data by matching the center point and size of the common circular pattern.

According to another aspect of the present invention, a computer program stored in a computer-readable medium for correcting a camera tolerance installed in a vehicle, the computer program causing the computer to perform the following steps, the steps comprising: (a) While the vehicle is moved such that the vehicle passes between a plurality of circular patterns disposed on the ground, a circular pattern is generated from each of the camera image data generated by a plurality of cameras respectively installed in the front, right, left and rear of the vehicle. Detecting; (b) extracting the outline of the detected circular pattern; And (c) calculating a homography matrix between the camera image data corresponding to each camera and the top view image by using the correlation between the extracted outline and a circular pattern disposed on the ground. A computer program stored in a readable medium is provided.

In order to ensure that the homography matrix for each camera is calculated using only camera image data that satisfies a predetermined validity condition, the step (c) is (c-1) in the top view image to which the currently calculated homography matrix is applied. Calculating an error value for each camera image data using a predetermined error value calculation technique to use a relationship between an analysis area of a circular pattern and an actual area of a circular pattern disposed on the ground; And (c-2) if there is camera image data in which the calculated error value is greater than a predetermined threshold, calculating the homography matrix again by excluding the camera image data and replacing the currently calculated homography matrix. It can contain. Here, the steps (c-1) and (c-2) may be repeated until there is no camera image data having a calculated error value greater than a predetermined threshold value, and the calculated error value is previously specified. If there is no camera image data larger than a threshold, it is possible to determine the currently calculated homography matrix for the corresponding camera as the homography matrix for the corresponding camera.

에 의해 산출될 수 잇다. 여기서, 상기 e는 에러값, 상기 p는 각 카메라 영상 데이터를 구분하는 식별 정보, 상기 n은 하나의 카메라 영상 데이터에서 검출된 원형 패턴의 총 갯수, k는 해당 식별 정보의 카메라 영상 데이터에서 검출된 각 원형 패턴의 구분자, M은 검출된 원형 패턴에 상응하여 지면에 배치된 원형 패턴의 실제 면적, m은 상기 식별 정보의 카메라 영상 데이터에서 검출되어 현재 산출된 호모그라피 행렬에 의해 변환된 탑뷰 영상 내의 원형 패턴의 해석 면적일 수 있다.The error value is an equation

Can be calculated by Here, e is an error value, p is identification information for distinguishing each camera image data, n is the total number of circular patterns detected from one camera image data, and k is detected from camera image data of the corresponding identification information. The separator of each circular pattern, M is the actual area of the circular pattern arranged on the ground corresponding to the detected circular pattern, m is detected in the camera image data of the identification information, and is converted into the currently calculated homography matrix in the top view image. It may be an analysis area of a circular pattern.

In order to quickly detect a circular pattern in each camera image data, in step (a), a candidate group of a circular hough transform (CHT) is first detected using a pair of gradient vectors, and then detected A circular pattern can be detected using CHT for the candidate group.

In the above steps, generating a top view image corresponding to each camera using a homography matrix corresponding to each camera and camera image data; And rotating each generated top view image to match the installation position of each camera based on the vehicle, and then synthesizing it into AVM image data for the entire surroundings of the vehicle.

If a common circular pattern exists in a plurality of top view images at the same time, the AVM image data may be generated by matching the center point and size of the common circular pattern.

Other aspects, features, and advantages than those described above will become apparent from the following drawings, claims, and detailed description of the invention.

According to an embodiment of the present invention, a top view can be generated when a pattern satisfies a condition in which an image appears without inputting the world coordinates of the pattern using a circular pattern rather than a polygonal pattern, and a homography matrix calculated for generating the top view It is possible to accurately estimate the external parameters of the camera using.

In addition, it is possible to shorten the time required to install the pattern by correcting the tolerance by simply moving the vehicle to pass between the patterns, and it is also possible to synthesize images of multiple cameras even if the relative and absolute positions of the camera and the pattern are not known. have.

1 is a view for explaining a tolerance correction method according to the prior art.
Figure 2 is a block diagram schematically showing the configuration of the AVM system according to an embodiment of the present invention.
3 to 7 are views for explaining a tolerance correction process of the AVM system according to an embodiment of the present invention.
8 is a diagram illustrating an arrangement example of a circular pattern according to another embodiment of the present invention.
9 is a flowchart illustrating a camera tolerance correction method of an AVM system according to an embodiment of the present invention.

The present invention can be applied to various changes and can have various embodiments, and specific embodiments will be illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, and should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention.

The terms used herein are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this specification, terms such as “include” or “have” are intended to indicate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, and that one or more other features are present. It should be understood that the existence or addition possibilities of fields or numbers, steps, operations, components, parts or combinations thereof are not excluded in advance.

Terms such as first and second may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from other components.

In addition, the components of the embodiments described with reference to the drawings are not limited to those embodiments, and may be implemented to be included in other embodiments within the scope of maintaining the technical spirit of the present invention. Although the description is omitted, it is natural that a plurality of embodiments may be reimplemented as one integrated embodiment.

In addition, in the description with reference to the accompanying drawings, the same components, regardless of reference numerals, are assigned the same or related reference numerals, and redundant description thereof will be omitted. In the description of the present invention, when it is determined that detailed descriptions of related known technologies may unnecessarily obscure the subject matter of the present invention, detailed descriptions thereof will be omitted.

2 is a block diagram schematically showing the configuration of an AVM system according to an embodiment of the present invention, and FIGS. 3 to 7 are views for explaining a tolerance correction process of an AVM system according to an embodiment of the present invention 8 is a diagram illustrating an arrangement example of a circular pattern according to another embodiment of the present invention.

Referring to FIG. 2, the AVM system may include an image generation unit 210, a tolerance correction unit 220, an image synthesis unit 230, a storage unit 240, and a display unit 250. Although not shown, a control unit for controlling the operation of one or more components included in the AVM system may be further included.

The image generating unit 210 includes a plurality of cameras 20 respectively installed in a plurality of locations of the vehicle 10 (for example, locations designated to photograph front, rear, left, and right) to photograph an external scene. do. The external scene photographed by each camera 20 of the image generating unit 210 is stored in the storage unit 220 as camera image data. Here, the camera 110 may be embodied as, for example, a wide-angle camera having a large angle of view, so that the surrounding environment of the vehicle can be photographed in a small quantity.

The tolerance compensator 220 analyzes the camera image data stored in the storage 240 and analyzes the circular pattern (which may have an ellipse in various shapes depending on the position of the moving vehicle), and analyzes the analyzed circular pattern Calculate the homography matrix using. At this time, in the process of calculating the homography matrix, an external parameter of each camera 20 may also be calculated, and the tolerance correction unit 220 may use this to estimate the position of each camera 20 on the world coordinate. It might be.

The image synthesizing unit 230 uses the homography matrix calculated by the tolerance compensator 220 to generate camera image data as a top view image that looks down from the top of the vehicle 10, and synthesizes these top view images to AVM Create image data.

When generating AVM image data using a plurality of top-view images, the image synthesis unit 230 considers that the circles forming the circular pattern are at the same location at the same time, and the center position and size of the circular pattern are determined. Matching methods, etc. can be used. At this time, the image synthesizing unit 230 may rotate the generated top-view images to correspond to positions where the corresponding camera 20 is installed in the vehicle.

The AVM image data generated by the image synthesizing unit 230 may be output through the display unit 250.

The storage unit 240 includes, for example, an operating program of the AVM system, camera image data generated by the image generation unit 210, a homography matrix calculated by the tolerance correction unit 220, and an image synthesis unit 240. The top view image and AVM image data generated by the user may be stored. The storage unit 220 may be divided into a permanent storage memory that permanently stores data and a temporary storage memory that temporarily stores and operates necessary data during operation.

Hereinafter, a process of generating a homography matrix by the tolerance compensator 220 and a process of generating a top view image and AVM image data by the image synthesis unit 230 will be described with reference to FIGS. 3 to 8. do.

As illustrated sequentially in (a) to (c) of FIG. 3, in the AVM system according to this embodiment, two circular patterns (300-1, 300-2, hereinafter referred to as 300) are parallel to the bottom surface. It is arranged, and after the vehicle 10 is moved by a predetermined distance to pass between the two circular patterns 300, the tolerance correction for each camera 20 is performed.

To this end, the image generating unit 210 until the vehicle 10 starts to move between the circular patterns 300 arranged side by side and ends (or is recognized as being moved by a predetermined distance). ), each camera 20 continuously photographs an external scene to store corresponding camera image data in the storage unit 240.

As illustrated in FIG. 3 and the like, the circular pattern 300 disposed on the ground for tolerance correction has a circular shape, and thus always shows the same shape regardless of an angle rotated around the center point. Therefore, unlike the conventional polygonal pattern (that is, limited to be installed at an accurate angle at a predetermined position) in the arrangement process of the circular pattern 300, it has an advantage of reducing the time required for pattern placement.

In addition, the circular pattern 300 does not have to be accurately positioned at a predetermined position as in the conventional polygonal pattern, and has the advantage that tolerance correction is possible even if it is positioned at an arbitrary position as long as the predetermined position condition is satisfied.

In addition, the circular pattern 300 may be pre-specified so that all of the diameters are the same, and accurate pixel pitch (mm/pixel) calculation may be performed when the top view is generated by the image synthesis unit 230 using pre-specified diameter information. You can make it possible. Of course, in a case where the diameters of the circular patterns 300 disposed at each position are pre-specified and the diameters of the respective circular patterns can be recognized in advance, it is natural that the diameters of all the circular patterns 300 are not limited to match.

At this time, the number of circular patterns 300 used for the correction of the tolerance of the camera 20 is not limited to two, and as illustrated in FIG. 8, the vehicle 10 is arranged to exist in plural on both sides on the moving path It is natural that it may be.

The tolerance correcting unit 220 detects the circular pattern 300 from each of the camera image data generated during the period in which the vehicle 10 moves through the circular patterns 300 and stored in the storage unit 240. .

Hereinafter, for convenience of description, it will be described on the assumption that 30 camera image data sequentially generated corresponding to each of the cameras 20 while the vehicle 10 moves by a predetermined distance.

4 illustrates an image of some of the 30 camera image data generated to correspond to the camera 20 provided on the right side of the vehicle 10 while the vehicle 10 is moving. As illustrated, the circular pattern 300 is photographed in the form of ellipses of various shapes according to the relative positions of the camera 20 and the circular pattern 300.

The tolerance compensator 220 may apply a method of using a pair of gradient vectors to quickly detect the circular pattern 300 from the camera image data generated by each camera 20. In this case, a LoG (Laplacian of Gaussian) filter may be used. A method for quickly detecting a circle in an image using a gradient pair vector is obvious to those skilled in the art, so a description thereof will be omitted.

That is, the tolerance correcting unit 220 quickly detects a candidate group of circular hough transform (CHT) using a pair of gradient vectors, and finally detects the outline of a circular pattern displayed as an ellipse in the camera image data using CHT. do. For reference, the outline of the circular pattern 300 detected from the photographed image data in FIG. 4 and the like is represented by a red line.

In order to enable the circular pattern 300 to be quickly detected using a pair of gradient vectors, the interior of the circular pattern 300 may be colored with a predetermined color, such as white or black.

The tolerance compensator 220 determines whether the circular pattern 300 is normally detected using a binarization technique (see Equation 1 below) that separates the circular pattern portion and the background portion from each camera image data to detect the outline of the circular pattern. You can also verify.

Here, I(x,y) denotes an image value of image coordinates (x,y) in the camera image data, Th denotes a threshold value, and B(x,y) denotes camera image data The image value corresponding to the coordinate (x,y) in is compared with the threshold value and means a binary value (ie, 0 or 255).

The tolerance compensator 220 may detect and verify the outline of the circular pattern 300 by finding the directionality of the outer edge with reference to the binarized result value for each image coordinate.

The tolerance correction unit 220 includes information about the circular pattern 300 previously stored (for example, one or more of shape, size, etc.) and information about the circular pattern 300 detected from 30 camera image data ( For example, a homography matrix indicating the relationship between each camera 20 and the ground may be calculated using one or more of the location, shape, and size of the center point. In general, the circular pattern 300 disposed on the ground is displayed as an ellipse in the captured image data, and the equation of the ellipse can be expressed as Equation 2 below.

Here, (α,β) represents the center point of the ellipse, a represents the radius of the long axis, and b represents the radius of the minor axis.

Equation 2 described above can be generalized to the elliptic equations Ax ² + Bxy + Cy ² + Dx + Ey + F = 0, and using each coefficient represented by A to F, a symmetric coefficient matrix (Symmetric Coefficient Matrix) Q can be expressed as Equation 3 below.

로 정의되므로, 실세계에서의 대칭적 계수 행렬 Q'는 하기 수학식 4와 같이 정의될 수 있다.The symmetric coefficient matrix Q in Equation 3 satisfies the relationship of x ^T Qx=0, and the homography matrix H representing the relationship between image coordinates (x) and real-world coordinates (x') is

Since it is defined as, the symmetric coefficient matrix Q'in the real world can be defined as in Equation 4 below.

In addition, the vanishing line (line l _p ), which is a straight line passing through the vanishing point, is a symmetric coefficient matrix Q and an intersection (point x _p ) (ie, the point where the vanishing line and the ellipse detected from the camera image data meet. ) Can be defined as in Equation 5 below.

In addition, the homography matrix H is defined using a similarity matrix S, an affinity matrix A, and a projectivity matrix A, but the influence of the similarity matrix S is negligible, so the homography matrix H is It may be defined by simplifying Equation 6 below.

Here, l ₁ , l ₂ and l ₃ mean a vanishing line that is a straight line passing through the vanishing point, and α and β are coefficients of affine transformation, respectively.

The tolerance compensator 220 uses the above equations 1 to 6 for 30 camera image data generated corresponding to each of the cameras 20 while the vehicle 10 moves by a predetermined distance. A homography matrix H corresponding to the camera 20 is generated. Each camera image data generated using the generated homography matrix H may be converted into a top view image (see FIG. 5).

Subsequently, the tolerance correcting unit 220 uses more camera image data (ie, 30 camera image data) generated while moving by a predetermined distance, so that the homography matrices generated to correspond to each camera 20 are more sophisticated. An improvement process is carried out so as to be canceled.

The improvement process repeats the process of checking the validity of each camera image data in which a circular pattern has been detected using the currently calculated homography matrix, and calculating the homography matrix again only for valid camera image data. When all of the camera image data used to calculate satisfy the validity condition, the final generated homography matrix is determined by an improved homography matrix corresponding to the corresponding camera 20.

To check the validity of each camera image data in which a circular pattern is detected, the tolerance correction unit 220 may calculate an error value of each camera image data using Equation (7) below. Of course, the error value calculation method, such as the ratio value of the actual area and the analysis area of the circular pattern 300 as an error value, may be various.

Here, e is an error value, p is identification information for classifying each camera image data, n is a total number of circular patterns detected from camera image data of the corresponding identification information, k is each detected from camera image data of the corresponding identification information The separator of the circular pattern, M _k is the actual area of the circular pattern 300 disposed on the ground corresponding to the detected circular pattern, m _k is detected from the camera image data of the corresponding identification information and converted by the current homography matrix It represents the area of the circular pattern (ie, the analysis area).

When one circular pattern is detected from the camera image data, according to Equation 7, the actual size of the circular pattern 300 coincides with the size of the camera image data and the circular pattern converted by the current homography matrix. Will have an error value of 0. Of course, when a plurality of circular patterns are detected from the camera image data, it will be calculated as an error value considering all of the circular patterns.

The tolerance compensator 220 calculates an error value for each of the 30 camera image data, determines whether there is camera image data having a calculated error value greater than a predetermined threshold value, and determines an error value greater than the threshold value. When the branch camera image data is present, the homography matrix is calculated again using only the camera image data having error values smaller than the threshold as target camera image data (see Equations 3 to 6 described above).

For example, if it is determined that the error value of the first and second camera image data among the 30 camera image data is greater than the threshold value, the homography matrix is formed by using only the third to 30th camera image data as target camera image data. Is calculated again.

The tolerance correcting unit 220 calculates an analysis area of the detected circular pattern for each of the target camera image data using the calculated homography matrix, and performs error value calculation again.

At this time, if there is no camera image data having an error value greater than a predetermined threshold, the current homography matrix is determined as a homography matrix corresponding to the camera 20. That is, when all the camera image data used to calculate the homography matrix satisfy the validity condition, the finally generated homography matrix is determined as an improved homography matrix corresponding to the corresponding camera 20.

However, if there is camera image data having an error value greater than a predetermined threshold, the tolerance correction unit 220 reselects the target camera image data excluding the corresponding camera image data and recalculates the homography matrix. The process of calculating the error value of the camera image data is performed again.

When the homography matrix for each camera 20 is determined by the above-described process, the image synthesizing unit 230 may generate AVM image data by synthesizing the top view image corresponding to each camera 20 (FIG. 7). Reference).

At this time, the image synthesizing unit 230 may rotate the top view image converted using the homography matrix of each camera 20 to correspond to the mounting position of each camera 20 for image synthesis. For example, the top view image corresponding to the camera 20 mounted on the right side of the vehicle 10 may be rotated so that the right side of the vehicle 10 is represented as a captured image, as illustrated in FIG. 6.

Also, the image synthesizing unit 230 considers that circular patterns detected in a plurality of camera image data captured by each camera at the same time are in substantially the same position, so that the common circular patterns exist in both top view images. It may be possible to quickly perform image synthesis using a method of matching the center position and size of the.

9 is a flowchart illustrating a camera tolerance correction method of an AVM system according to an embodiment of the present invention.

Referring to FIG. 9, in step 910, the AVM system is photographed by each camera 20 while the vehicle 10 moves by a predetermined distance to pass between a plurality of circular patterns 300 disposed on the ground. Create camera image data. The cameras 20 may be installed in a plurality of places of the vehicle 10 (for example, positions designated to photograph front, rear, left, and right, respectively).

In step 920, the AVM system detects a circular pattern in each camera image data. The AVM system can detect, for example, a CHT candidate group by applying a method using a pair of gradient vectors, and then, using the CHT, can detect the outline of a circular pattern represented by an ellipse in the camera image data.

In step 930, the AVM system calculates an ellipse equation based on the outline of the ellipse using the camera image data generated corresponding to each camera 20, and uses the calculated ellipse equation to homography between world coordinates and camera image data. A matrix (referred to as a raw homography matrix) is calculated for each camera 20.

In step 940, the AVM system determines whether the original homography matrix is calculated using only camera image data that satisfies a predetermined validity condition.

At this time, the AVM system is calculated using the relationship between the actual area of the circular pattern 300 known in advance and the analysis area calculated by converting the circular pattern detected from the camera image data into world coordinates using a raw homography matrix. When one or more camera image data having an error value greater than a predetermined threshold value is present, it may be determined that the predetermined validity condition is not satisfied (see Equation 7 above).

If there is camera image data having an error value greater than a predetermined threshold, the process proceeds to step 950, and the AVM system reselects the target camera image data from which the camera image data is excluded among the camera image data generated in step 910. To decide.

Subsequently, the process proceeds to step 930, and the AVM system calculates the homography matrix (referred to as the first enhanced homography matrix) for each camera 20 using the target camera image data as the target again.

Subsequently, in step 940, the AVM system determines whether the first enhanced homography matrix is calculated using only camera image data that satisfies a predetermined validity condition. In this case, the analysis area calculated by converting the circular pattern detected from the camera image data into world coordinates using the first enhanced homography matrix will be considered.

If the first improved homography matrix is not calculated using only camera image data that satisfies the validity condition, the AVM system proceeds to steps 950 and 960 to perform a homography matrix (ie, second homography) for each camera 20. Matrix).

That is, steps 930 to 950 are individually performed for each camera 20 and repeatedly performed until it is recognized that the homography matrices are calculated only with camera image data satisfying the validity condition.

However, if, as a result of the determination in step 940, all currently generated homography matrices are calculated for only camera image data that satisfies the validity condition, the AVM system proceeds to step 960 and the homo corresponding to each camera 20 A top view image corresponding to each camera image data is generated using a graphing matrix, and a plurality of top view images are synthesized to generate AVM image data.

At this time, the AVM system rotates each top view image to match the installation position of the corresponding camera, and then uses the method of synthesizing such that the center positions of the common circular patterns photographed at the same time point coincide. Data can be generated.

It is natural that the above-described method for correcting camera tolerance of the AVM system may be performed by an automated procedure according to a time series sequence by a program built in or installed in a digital processing device. The codes and code segments constituting the program can be easily deduced by a computer programmer in the field. In addition, the program is stored in a computer readable medium that can be read by a digital processing device, and is read and executed by a digital processing device to implement the method.

The above has been described with reference to embodiments of the present invention, but those skilled in the art can variously modify the present invention without departing from the spirit and scope of the present invention as set forth in the claims below. You will understand that you can change it.

10: vehicle 20-1, 20-2, 20-3, 20-4: camera 50: correction plate
50: correction plate 210: image generation unit
220: tolerance correction unit 230: image synthesis unit
240: storage unit 250: display unit
300-1, 300-2: circular pattern

Claims (13)

In the AVM system installed in the vehicle,
An image including a plurality of cameras respectively installed at the front, right, left and rear of the vehicle, and photographing the surroundings while the vehicle is moving to pass between a plurality of circular patterns disposed on the ground to generate camera image data Generation unit; And
A circular pattern is detected from each of the camera image data generated while the vehicle is moving to extract the outline of the detected circular pattern, and is correlated to each camera using a correlation between the extracted outline and the circular pattern disposed on the ground. Tolerance correction unit for calculating a homography matrix between the camera image data and the top view image,
The circular pattern included in each of the camera image data generated while the vehicle is moving is modified to correspond to the relative positions of the camera and the circular pattern disposed on the ground according to the moving position of the vehicle at the time of generation of the corresponding camera image data. AVM system, which is formed into a shape.
According to claim 1,
The tolerance compensator may be configured to calculate a homography matrix for each camera using only camera image data that satisfies a predetermined validity condition.
The error value for each camera image data is calculated by using the predefined error value calculation technique to use the relationship between the analysis area of the circular pattern in the top view image to which the currently calculated homography matrix is applied and the actual area of the circular pattern placed on the ground. Step (a); And
If there is camera image data whose calculated error value is greater than a predetermined threshold, calculating the homography matrix again by excluding the corresponding camera image data and replacing the currently calculated homography matrix (b). Repeat until there is no camera image data whose value is greater than a predetermined threshold,
If there is no camera image data having a calculated error value greater than a predetermined threshold, the AVM system is characterized in that the currently calculated homography matrix of the corresponding time point is determined as a homography matrix for the corresponding camera.
According to claim 2,
The error value is an equation

Calculated by,
The e is an error value, the p is identification information for distinguishing each camera image data, the n is the total number of circular patterns detected from one camera image data, and k is each prototype detected from camera image data of the corresponding identification information. The separator of the pattern, M is the actual area of the circular pattern disposed on the ground corresponding to the detected circular pattern, m is the circular pattern in the top view image detected by the camera image data of the identification information and converted by the currently calculated homography matrix. AVM system characterized by the interpretation area of.
According to claim 1,
AVM system, characterized in that the plurality of circular patterns are arranged to be distributed around the path of the vehicle.
According to claim 1,
In order to quickly detect a circular pattern in each camera image data, the tolerance correcting unit first detects a candidate group of a circular hough transform (CHT) using a pair of gradient vectors, and then detects the detected candidate group. AVM system characterized by detecting a circular pattern using CHT as a target.
According to claim 1,
After generating a top view image corresponding to each camera using the homography matrix and camera image data corresponding to each camera, and rotating each generated top view image to match the installation position of each camera based on the vehicle, AVM system further comprising an image synthesizer for synthesizing AVM image data for the entire periphery of the vehicle.
The method of claim 6,
The image synthesizing unit, when there is a common circular pattern in a plurality of top-view images at the same time (time), AVM system characterized by synthesizing the AVM image data by matching the center point and size of the common circular pattern.
A computer program stored in a computer-readable medium for correcting a camera tolerance installed in a vehicle, the computer program causing the computer to perform the following steps, the steps comprising:
(a) While the vehicle is moved so that the vehicle passes between a plurality of circular patterns disposed on the ground, each of the camera image data generated by a plurality of cameras respectively installed in the front, right, left and rear of the vehicle Detecting a circular pattern in;
(b) extracting the outline of the detected circular pattern; And
(c) calculating a homography matrix between camera image data corresponding to each camera and a top view image using a correlation between the extracted outline and a circular pattern disposed on the ground,
The circular pattern included in each of the camera image data generated while the vehicle is moving is modified to correspond to the relative positions of the camera and the circular pattern disposed on the ground according to the moving position of the vehicle at the time of generation of the corresponding camera image data. Computer program, characterized in that formed in a computer-readable medium.
The method of claim 8,
In order to allow the homography matrix for each camera to be calculated using only camera image data that satisfies a predetermined validity condition, the step (c) is:
(c-1) An error value calculation technique that is defined in advance to use the relationship between the analysis area of the circular pattern in the top view image to which the currently calculated homography matrix is applied and the actual area of the circular pattern disposed on the ground. Calculating an error value for; And
(c-2) if there is camera image data having a calculated error value greater than a predetermined threshold, calculating a homography matrix again by excluding the corresponding camera image data and allowing the currently calculated homography matrix to be replaced. Ha,
Steps (c-1) and (c-2) are repeated until there is no camera image data having a calculated error value greater than a predetermined threshold,
If there is no camera image data having a calculated error value greater than a predetermined threshold, a computer-readable medium, characterized in that the currently calculated homography matrix at the time is determined as a homography matrix for the corresponding camera. Computer programs stored in.
The method of claim 9,
The error value is an equation

Calculated by,
The e is an error value, the p is identification information for distinguishing each camera image data, the n is the total number of circular patterns detected from one camera image data, and k is each prototype detected from camera image data of the corresponding identification information. The separator of the pattern, M is the actual area of the circular pattern disposed on the ground corresponding to the detected circular pattern, m is the circular pattern in the top view image detected by the camera image data of the identification information and converted by the currently calculated homography matrix. A computer program stored in a computer-readable medium, characterized by an area of interpretation.
The method of claim 8,
In order to quickly detect a circular pattern in each camera image data, in step (a), a candidate group of a circular hough transform (CHT) is first detected using a pair of gradient vectors, and then detected A computer program stored in a computer-readable medium, characterized in that a circular pattern is detected using the CHT of the targeted candidate group.
The method of claim 8,
Generating a top view image corresponding to each camera using a homography matrix corresponding to each camera and camera image data; And
After rotating each generated top-view image to match the installation position of each camera relative to the vehicle, and further comprising the step of synthesizing into AVM image data for the entire periphery of the vehicle, stored in a computer-readable medium Computer program.
The method of claim 12,
When a common circular pattern exists in a plurality of top view images at the same time, the AVM image data is generated by matching the center point and size of the common circular pattern, and the computer program stored in the computer-readable medium

Images