KR101808092B1 - Apparatus and method for auto-correcting and generating avm image - Google Patents

Abstract

Discloses an apparatus and method for automatic AVM image correction and generation. The present invention relates to an image processing apparatus including an image acquiring unit disposed at a designated position of a vehicle and having a plurality of cameras each acquiring an imaged image in a designated direction with respect to a vehicle, a plurality of reference images acquired and stored in advance in each of the plurality of cameras, A plurality of reference points designated in advance as feature points of the vehicle body are extracted from each of the plurality of reference images and a plurality of reference points extracted from the plurality of reference images are compared with positions of reference points extracted from the plurality of captured images, An image correcting unit for correcting the photographed image and generating a plurality of corrected images, and an image generating unit for generating an AVM image by matching a plurality of images output from the image correcting unit.

Description

[0001] APPARATUS AND METHOD FOR AUTO-CORRECTING AND GENERATING AVM IMAGE [0002]

The present invention relates to an apparatus and method for automatically correcting and generating an image, and more particularly, to an apparatus and method for automatically correcting and generating an around view monitoring image.

Around View Monitoring (AVM) system typically uses images acquired by installing four cameras in the front, rear, and both sides of the vehicle to provide images of 360 degrees around the vehicle This makes it possible for a vehicle occupant to accurately recognize the surrounding environment of the vehicle, thereby improving safety and improving convenience. AVM is widely applied to ADAS (Advanced Driver Assistance System), and it is widely used for parking guidance system (PGS), lane keeping assistance system (LKAS), lane departure warning system (LDWS) ) And the like.

However, since the AVM system provides the surround view by merging the images acquired from a plurality of cameras installed at a designated position of the vehicle, if an error occurs in the images acquired by the respective cameras, the driver can accurately recognize But it is likely to be an obstacle. Therefore, the cameras that acquire images in the designated directions around the vehicle must be accurately positioned at the designated positions. However, as the time for using the vehicle is increased, various types of external shocks accumulate in the vehicle, which causes an error in the images gradually acquired even in the cameras arranged and arranged. That is, the image may be altered, resulting in a reduction in reliability and durability of the AVM system.

Accordingly, in the AVM system, a method of recognizing an error of an image obtained from each of a plurality of cameras and correcting the image when an error occurs is required. Conventionally, various schemes for correcting the image of the AVM system have been proposed, but the AVM image correction is performed in such a manner that the vehicle is placed in a specific place where a reference index for predicting the error is previously recorded.

Therefore, in order to correct the AVM image, not only is it troublesome to move the vehicle to a designated place, but also the user can not directly correct the AVM image, which requires professional work and causes loss in time and cost.

Korean Patent Publication No. 10-2016-0034134 (published on Mar. 29, 2016)

It is an object of the present invention to provide an automatic AVM image correction and generation apparatus capable of automatically generating AVM images by recognizing and correcting errors of AVM images.

It is another object of the present invention to provide an automatic AVM image correction and generation method capable of achieving the above objects.

According to an aspect of the present invention, there is provided an apparatus and method for automatically correcting and generating an AVM image, the apparatus comprising: a plurality of cameras, each of which is disposed at a designated position of a vehicle, part; Extracting a plurality of reference images previously acquired and stored in each of the plurality of cameras and a plurality of reference points preliminarily designated as feature points of the vehicle body in each of the plurality of shot images, An image corrector configured to generate a plurality of correction images by correcting the plurality of photographed images using an error value obtained by comparing positions of reference points extracted from the photographed image; An image generating unit for generating an AVM image by matching a plurality of images output from the image correcting unit; .

Wherein the image correction unit stores an image of a plurality of reference images obtained in each of the plurality of cameras in an initial state in which the plurality of cameras in the vehicle are determined to be normally installed in the vehicle body; A reference point extracting unit for extracting the plurality of reference points from each of the plurality of reference images and the plurality of shot images; And comparing the positions of the reference points corresponding to each other among the plurality of reference points extracted from the plurality of reference images and the plurality of shot images to obtain the error value with respect to the photographing direction of each of the plurality of cameras and the rotation angle of the camera An image comparison unit; And an error corrector configured to correct the plurality of photographed images according to the error value to generate the plurality of corrected images. And a control unit.

Wherein the image generating unit comprises: an image accumulating unit accumulating the plurality of corrected images; A reference point extracting unit for extracting points corresponding to the corners of the lane included in the corrected image as a plurality of reference points together with points corresponding to the plurality of extracted points in the plurality of corrected images stored in the image accumulating unit; And a controller for comparing the positions of the plurality of reference points commonly included in the transformed images to be arranged adjacent to each other in the AVM image among the plurality of transformed images subjected to the inverse projection transformation, A reference point matching unit for correcting the plurality of transformed images using the determined transform error value; And an image combining unit for generating the AVM image of bird's-eye view shape by arranging the plurality of converted images corrected by the reference point matching unit at positions corresponding to arrangement positions of the plurality of cameras, respectively; And a control unit.

Wherein the reference point extracting unit extracts a plurality of corrected images stored in the image accumulating unit in order to detect corners of the lane alternately displayed on both sides of the vehicle on the road, And detects the reference point with respect to the corner of the lane.

Wherein the image correction unit is configured to accumulate a plurality of reference images obtained in each of the plurality of cameras and the plurality of shot images applied in the plurality of cameras in an initial state in which it is determined that the plurality of cameras are normally installed in the vehicle, An image storage unit for storing the image data; A reference point extracting unit for extracting points corresponding to corners of the lane included in the photographed image together with the feature points of the vehicle body in each of the plurality of reference images and the plurality of photographed images as a plurality of reference points; And comparing the positions of the reference points corresponding to each other among the plurality of reference points extracted from the plurality of reference images and the plurality of shot images to obtain the error value with respect to the photographing direction of each of the plurality of cameras and the rotation angle of the camera An image comparison unit; And generating the plurality of corrected images by correcting the plurality of shot images so as to compensate for the error values, performing a back-projection transformation on the plurality of corrected images, An error correction unit for comparing the positions of the plurality of reference points commonly included in the transformed images to be disposed adjacent to each other in the transformed image to determine a transformed error value and correcting the plurality of transformed images using the determined transformed error value, ; And a control unit.

Wherein the image acquiring unit photographs the front of the vehicle and is disposed in the radiator grill such that the top edge of the front plate of the vehicle to be extracted as the reference point is included in the photographed image; A rear camera disposed between the rear lid of the vehicle and the rear bumper so as to photograph the rear of the vehicle and include a plurality of fixing screws of the rear plate of the vehicle to be extracted at the reference point in the taken image; A left camera installed on the left side mirror of the vehicle so as to capture the left side of the vehicle and include the left door handle of the vehicle to be extracted as the reference point and the front bumper; And a right camera installed on the right side mirror of the vehicle so as to photograph the right side of the vehicle and include the right door handle of the vehicle to be extracted as the reference point and the front bumper in the taken image; And a control unit.

According to an aspect of the present invention, there is provided an automatic AVM image correction and generation method for an AVM image automatic correction and generation apparatus including an image acquisition unit, an image correction unit, and an image generation unit, A plurality of cameras of the image acquiring unit are respectively disposed at designated positions of the vehicle to acquire images taken in a designated direction around the vehicle; Extracting a plurality of reference images previously acquired and stored in each of the plurality of cameras by the image correction unit and a plurality of reference points preliminarily designated as feature points of the vehicle body in each of the plurality of shot images; Comparing the reference points extracted from the plurality of reference images with the positions of reference points extracted from the plurality of shot images to obtain an error value; Generating a plurality of correction images by correcting the plurality of photographed images using the error value; And generating an AVM image using the plurality of corrected images by the image generating unit; .

Accordingly, the apparatus and method for automatically correcting and generating an AVM image of the present invention automatically compares the reference image acquired in a vehicle initial state with the currently acquired image on the basis of a specific position of the vehicle body, And the AVM image can be generated. Therefore, it is possible to provide the AVM image which is always corrected without any operation even when the vehicle is in operation, so that it is not necessary to move to a designated place for AVM image correction, to use a separate device, . Therefore, the convenience and reliability of the user can be maximized and the safety can be guaranteed.

FIG. 1 shows a configuration of an AVM image automatic correction and generation apparatus according to an embodiment of the present invention.
2 shows an example of an image obtained by a plurality of cameras of the image acquiring unit and a reference point extracted by the reference point extracting unit.
Fig. 3 shows an example of an image in which an error occurs.
Fig. 4 shows an example of an image in which an error of an image in which an error occurs is corrected.
5 shows an example in which the reference point extracting section extracts two reference points from the lane.
6 is a view for explaining an error of a corner of a lane included in adjacent transformed images.
7 is a diagram showing a comparison between a distorted AVM image and a corrected AVM image.
8 illustrates a method of generating an AVM image according to an embodiment of the present invention.

In order to fully understand the present invention, operational advantages of the present invention, and objects achieved by the practice of the present invention, reference should be made to the accompanying drawings and the accompanying drawings which illustrate preferred embodiments of the present invention.

Hereinafter, the present invention will be described in detail with reference to the preferred embodiments of the present invention with reference to the accompanying drawings. However, the present invention can be implemented in various different forms, and is not limited to the embodiments described. In order to clearly describe the present invention, parts that are not related to the description are omitted, and the same reference numerals in the drawings denote the same members.

Throughout the specification, when an element is referred to as "including" an element, it does not exclude other elements unless specifically stated to the contrary. The terms "part", "unit", "module", "block", and the like described in the specification mean units for processing at least one function or operation, And a combination of software.

FIG. 1 shows a configuration of an AVM image automatic correction and generation apparatus according to an embodiment of the present invention.

Referring to FIG. 1, the AVM image automatic correction and generation apparatus of the present invention includes an image acquisition unit 100, an image correction unit 200, and an AVM image generation unit 300.

The image acquiring unit 100 includes a plurality of cameras 110 to 140 installed at designated positions of the vehicle to acquire images in a predetermined direction. In the AVM image automatic correction and generation apparatus of the present invention, the number of cameras provided in the image acquisition unit 100 is not limited. However, according to the present invention, four cameras 110 to 140 As an example.

For example, the first camera 110 is a camera disposed in front of the vehicle, and the second camera 120 is a camera disposed in front of the vehicle. And the third and fourth cameras 130 and 140 may be cameras disposed in the left and right rooms of the vehicle, respectively.

Each of the four cameras 110 to 140 may include an ultra-wide angle lens (fisheye lens) having an angle of view of 180 degrees or more so that the AVM image generated by the AVM image generating unit 300 includes a 360-degree omni-directional image around the vehicle . Here, since the image acquisition unit 100 includes four cameras 110 to 140, the angle of view of each camera must be 180 degrees or more, but the image acquisition unit 100 may include five or more cameras The angle of view may be 180 degrees or less. That is, the limitation on the angle of view of each camera may vary depending on the number of cameras.

However, in the present invention, each of the plurality of cameras 110 to 140 should be installed so that the pre-designated feature region of the body is included in the obtained image. This is because the automatic AVM image correcting and generating apparatus of the present invention is configured to recognize and correct the error based on the feature region of the body included in the image obtained from each of the plurality of cameras.

For example, when four cameras 110 to 140 are provided, the first camera 110 for photographing the front of the vehicle may be installed on the radiator grill so that the acquired image includes the front license plate or the front bumper of the vehicle, The second camera 120 for photographing the rear of the vehicle may be installed between the rear lid and the rear bumper so that a rear license plate is included in the acquired image of the vehicle. The third and fourth cameras 130 and 140 for photographing both sides of the vehicle can be installed on both side mirrors such that the door handle of the vehicle and the front or rear bumper are included in the acquired image.

The image correction unit 200 may include an image storage unit 210, a reference point extraction unit 220, an image comparison unit 230, and an error correction unit 240.

The image storage unit 210 stores a plurality of reference images for detecting an error of an image to be imaged by the imaged image obtaining unit 100. Here, since the reference image is an image stored to provide a reference for discriminating the error of the currently obtained image, the reference image must be an object to be compared with the currently obtained image. Therefore, the reference image corresponds to the number of cameras included in the image obtaining unit 110 .

It is preferable that an image obtained by each of the plurality of cameras 110 to 140 of the image obtaining unit 100 is set as a reference image in an initial state in which no external impact is applied to the vehicle as much as possible Do. Since the AVM system is installed in a number of vehicles in the basic specifications in recent years, the reference image may be set such that the vehicle manufacturer leaves the vehicle after the reference image is set in advance before the vehicle departs from the vehicle. You can also set the image.

When an AVM system is installed in a vehicle in use, an error may be included in an image acquired by each camera at the initial stage of installation. It is also possible to set the reference image using the lane around the vehicle included in the image obtained from the plurality of cameras during the vehicle operation. A detailed description of a method of using lanes around the vehicle will be given later. Also, the reference image can be set immediately after the initial correction of the AVM system is performed in a specific place provided with facilities for correcting the AVM system as in the conventional art.

The image storage unit 210 may receive and store the currently captured images of the plurality of cameras 110 to 140 of the image acquisition unit 100. Alternatively, And the image may be directly transmitted to the reference point extracting unit 220.

The reference point extracting unit 220 receives a plurality of reference images from the image storing unit 210 and a plurality of captured images currently acquired and stored in the image obtaining unit 110 and analyzes the reference images, Extract the reference points. Here, the reference point is a reference index for analyzing the error between the photographed image and the reference image, and is a minutiae point of the vehicle body that is included in the image obtained by each of the plurality of cameras and is designated in advance. It is preferable that the reference point is set to a feature point of the vehicle body which is easily extracted from the image of the vehicle body and is determined to be easy to compare the change of the image.

2 shows an example of a captured image obtained by a plurality of cameras of the image obtaining unit and a reference point extracted by the reference point extracting unit.

2, (a) is a vehicle front image obtained by the first camera 110, (b) is a right side image obtained by the fourth camera 140, (c) It is the acquired backward image.

When four cameras 110 to 140 are installed in front and rear sides and both sides of the vehicle, reference points in the respective images are indicated by red dots in FIG. 2, such as the side edges of the front number plate, the fixing screws of the rear number plate, And the vehicle side edge of the front bumper can be extracted as reference points. Unlike the front license plate, in the case of extracting the fixing screw of the rear license plate to the reference point, unlike the electric license plate in most vehicles, the edge of the latter license plate is adhered to the car body or attached to the area embedded in the car body for mounting the license plate This is because it does not appear in the photographed image. However, as an example of the above reference points, other feature points included in the photographed image and capable of representing the vehicle body may be used as reference points.

Techniques for extracting feature points of a specific feature in a photographic image processing technique are well known and well-known techniques. For example, reference points for front and rear images include canine edge detection, contour tracing, and labeling blob labeling), and the reference point for the lateral image can be extracted by the contour tracing method. Other techniques for extracting reference points exist in various ways and are well known in the art and are not described in detail here. It is preferable that a plurality of reference points are extracted from each of the plurality of reference images and the plurality of captured images.

Fig. 3 shows an example of an image in which an error occurs.

As shown in FIG. 3 (a), the plurality of cameras 110 to 140 provided on the vehicle body are moved up or down or tilted in the left or right direction, as compared with the case where no error occurs in the captured image, the position of the reference point in the image obtained due to the bending of the camera lens does not simply move in parallel even when the photographed image moved in the X axis direction or the Y axis direction is obtained as shown in FIG. In some cases, the plurality of cameras 110 to 140 may rotate. there is a problem that an error can not be discriminated if only one reference point is set in the case of a photographed image in which an error occurs due to rotation of the camera as shown in Fig. Accordingly, in the present invention, a plurality of reference points (two in this example, for example) are extracted from each of the plurality of reference images and the plurality of captured images.

The image comparing unit 230 compares the plurality of reference images and the corresponding images among the plurality of captured images to determine an error. At this time, the image comparing unit 230 compares the reference image obtained from the same camera among the plurality of cameras 110 to 140 with the photographed image, and the reference point extracting unit 220 extracts a plurality of reference images By comparing the points, an error value of the photographed image with respect to the reference image is discriminated.

The image comparing unit 230 determines whether the photographed image has been moved in the X-axis direction or the Y-axis direction from the difference between the positions of a plurality of reference points extracted from the photographed image in comparison with the positions of the plurality of reference points extracted from the reference image, Can be determined. When it is determined that the photographed image has been moved or rotated, the image comparison unit 230 generates a straight line connecting at least two reference points among the plurality of reference points in the reference image and the photographed image, And an angle difference can be calculated to obtain an error value.

The error corrector 240 compensates the error of the photographed image by using the error value determined by the image comparator 230, thereby obtaining the corrected image. The error corrector 240 receives the error value from the image comparator 230, and moves and rotates the photographed image so that the error value can be removed. That is, the position of each of the plurality of reference points of the photographed image is arranged at the position of the corresponding reference points of the reference image. In this case, since the image distortion may occur due to the angle of view, the corrected image can not be obtained by simply moving and rotating the captured image, and the correction technique according to the characteristics of each camera can correct the captured image to the corrected image. Here, since there are various prior arts, the image correction technique is not described in detail here.

Fig. 4 shows an example of an image in which an error of an image in which an error occurs is corrected.

4 (a) and 4 (b) show the vehicle front image obtained when the front camera is diverted to the left and right directions, respectively, and FIG. 4 (c) shows the corrected image. As shown in FIG. 4, the error corrector 240 of the present invention can correct the photographed image to be an image corresponding to the reference image using the error value obtained by the image comparator 230.

As a result, the error correction unit 300 extracts reference points previously set as feature points of the vehicle body from a plurality of shot images obtained from each of the plurality of cameras 110 to 140 and a plurality of reference images stored in advance, Thereby correcting the photographed image by using the discriminated error so that the vehicle can be moved to a specific position or automatically photographed by each of the cameras 110 to 140 So that it is possible to acquire the corrected image in which the error of the image is corrected. For convenience of explanation, it is assumed that an uncorrected photographed image is also output as a corrected image from the error corrector 300 because an error is not generated.

In the above description, the reference point extracting unit 220 and the image comparing unit 230 extract reference points from all the captured images obtained by the image obtaining unit 100 and compare the images to determine the error values. However, except for a special case such as a collision accident during actual vehicle operation, it is very likely that distortion occurs in the images acquired by the plurality of cameras 110 to 140 installed in the vehicle. That is, even if the error value is discriminated for all the captured images, there is a case where the error value does not change between the actual previous captured image and the current captured image. Therefore, the reference point extracting unit 220 and the image comparing unit 230 may be configured to extract a reference point from a photographed image every predetermined period or to determine an error value. Here, the specific event may be set at the vehicle start time as an example, and the predetermined period may be set at a time interval such as daily or weekly.

The AVM image generating unit 300 includes an image accumulating unit 310, a reference point extracting unit 320, a reference point matching unit 330, and an image combining unit 340.

The image accumulation unit 310 accumulates and stores a plurality of correction images applied by the image correction unit 200. [ The image accumulating unit 310 accumulates a plurality of corrected images in order to allow the image combining unit 340, which will be described later, to extract the necessary number of reference points for combining the corrected images from the reference point extracting unit 320 to be.

The reference point extraction unit 320 extracts a plurality of reference points from each of the plurality of correction images accumulated in the image accumulation unit 310. Herein, the reference point is a reference coordinate required to perform an inverse perspective transformation (or an inverse perspective transformation) for generating an AVM image displayed in an individual's eye view.

An image captured by a plurality of cameras 110 to 140 of the image acquisition unit 100 is basically an image including a perspective. However, since the bird's-eye view, which is an output form of the AVM image, provides the same image as that of the surroundings including the vehicle at the upper part of the vehicle, the perspective of each position coordinate is removed from the image captured by each of the cameras 110 to 140, To the plane coordinates of the plane.

The inverse projection transformation is a method of performing matrix transformation on each coordinate of the input image to convert it to plane coordinates, and generating an output image using the converted plane coordinates. Equation (1) represents an example of a basic mathematical expression for projection transformation (or reverse projection transformation).

X ' and y ' are coordinates of each pixel obtained in the original image to be subjected to the projection transformation, x ' and y ' And w is a weight.

However, since the present invention applies an inverse projection transformation, x and y in Equation (1) are coordinate values of each pixel in a corrected image to be subjected to a back projection transformation, and x 'and y' And becomes the coordinate value of each pixel in the removed transformed image.

In order to derive the eight coefficients (a to h) in Equation (1), four coordinate values are required for each of the coordinates (x, y) of the input image and the coordinates (x ', y') of the output image. (X1, y1), (x2, y2), (x3, y3), (x4, y4) in the corrected image obtained by correcting the captured image obtained by the plurality of cameras 110 to 140, Four coordinates ((x1 ', y1'), (x2 ', y2'), (x3 ', y3'), (x4 ', y4')) are required in the image. In this case, since the resolution is preset in the converted image to be outputted as the AVM image, the reference point extracting unit 320 needs to set a reference point for obtaining four coordinates in the corrected image, which is the input image.

The reference point extraction unit 320 extracts two of the four reference point coordinates to be acquired from the corrected image using the reference point extracted from the reference point extraction unit 220. [ In the above example, since the reference point extracting unit 220 extracts two reference points from each of the plurality of shot images, the reference point extracting unit 320 can also extract two reference points from each of the plurality of correction images.

In addition, the reference point extracting unit 320 can detect and extract a lane in the corrected image obtained during traveling of the vehicle with respect to the remaining two reference points. A lane mark is a road mark frequently included in a photographed image of a vehicle during road running, and is displayed relatively regularly in the vicinity of the vehicle. Therefore, in the present invention, the reference point extracting unit 320 can detect the remaining two reference points other than the two reference points obtained from the vehicle body, from the lane edge. Lane edges can be detected using blob labeling and Harris corner detection techniques.

5 shows an example in which the reference point extracting section extracts two reference points from the lane.

As shown in FIGS. 5A and 5B, the lanes on the road generally alternate with each other, rather than the left and right lanes appearing at the same time around the vehicle body. Therefore, a plurality of shot images acquired at a specific time often include only one lane. In other words, it is mostly the case that only one reference point can be obtained. In order to solve this problem, as described above, the image accumulating unit 310 accumulates a plurality of corrected images in a cumulative manner, thereby obtaining all corrected images for both lanes, and the reference point extracting unit 320 accumulates And extracts the edges of both lanes from the corrected image as the remaining two reference points. At this time, the reference point extracting unit 320 extracts a predetermined number (for example, two) of consecutive acquisitions (for example, two) that are possible in each of the cameras 110 to 140 in order to minimize a time difference of an image for detecting the edges of both lanes. Extract the reference point from the image.

The reference point matching unit 330 obtains eight coefficients a to h for the back projection transformation using the reference points extracted from the respective correction images by the reference point extraction unit 320, To obtain transformed images. The coordinates of the same reference points included in the transformed images adjacent to each other in the transformed images of the captured images obtained by the plurality of cameras 110 to 140 are compared to determine whether they represent the same position.

If it is determined that they do not represent the same position, the reference point matching unit 330 compensates for the error of the transformed image to be arranged adjacent to each other in the AVM image using the position difference of each reference point. This is for further correcting the error that may occur when combining the corrected images to generate the AVM image even though the respective images are compared with the reference image by the image correcting unit 200 to correct the errors.

6 is a view for explaining an error of a corner of a lane included in adjacent transformed images.

FIG. 6 is an image of the left rear of the vehicle, which is an image obtained by combining the transformed images after the back projection transformation of the photographed images obtained by the left camera 130 and the rear camera 120. FIG. In FIG. 6, (a) is an image in which a lane is displayed normally, (b) is a result of an error between adjacent transformed images, and the lane of the transformed image obtained by the left camera 130 in the bird's- Can be confirmed. And (c) shows the hidden lane by comparing the image with the error and the normal image. (c), the corner of the lane indicated on the left should be changed to the lane indicated on the right.

The image combining unit 340 receives the error-compensated transformed images from the reference point matching unit 330 and generates AMV images by performing image stitching in the same manner as a conventional AVM image generating method.

7 is a diagram showing a comparison between a distorted AVM image and a corrected AVM image.

(B), (e), (h), and (k) represent the AVM images to be acquired in the steady state, 110, the left camera 130, the right camera 140, and the rear camera 120 are turned off. And (c), (f), (i), and (l) illustrate an AVM image corrected by the AVM image automatic correction and generation apparatus of the present invention.

As shown in FIG. 7, the AVM image automatic correction and generation apparatus of the present invention automatically detects any camera direction error among a plurality of cameras 110 to 140 of the image obtaining unit 100 And correct AVM images can be provided to the user.

The image generating unit 300 includes the image accumulating unit 310, the reference point extracting unit 320, and the reference point matching unit 330 in addition to the image correcting unit 200. However, if the image storage unit 210 is configured to accumulate a plurality of photographed images, the image accumulation unit 310 may be omitted, and the reference point extraction unit 320 may extract the reference points from the reference point extraction unit 220, As an additional reference point. Also, the error corrector 240 may perform the operation of the reference point matching unit 330 together. That is, the image correcting unit 200 does not compensate only the image distortion of the camera by using the reference point of the vehicle body, but also corrects the coordinate error at the time of the inverse projection conversion by further using the corners of the lane , The image generating unit 300 may merely merge the corrected images to generate the AVM image.

8 illustrates an automatic AVM image correction and generation method according to an embodiment of the present invention.

Referring to FIGS. 1 to 7, a method of automatically correcting and generating an AVM image of FIG. 8 will be described. First, a plurality of cameras 110 to 140 installed in a predetermined position on a vehicle body acquire a captured image in a designated direction (S110) . The photographed image is applied to the image correction unit 200 and the reference point extraction unit 220 of the image correction unit extracts reference points from the reference image and the photographed image stored in the image storage unit 210 at step S120. As described above, the reference image is stored in the image storage unit 210 in advance as an image captured by each of the plurality of cameras 110 to 140 in an initial state in which no external impact is applied to the vehicle, These are the minutiae points of the body which are designated in advance.

When the reference point extracting unit 220 extracts a plurality of reference points from the reference image and the captured image, the image comparing unit 230 compares the positions of the reference points extracted from the reference image and the captured image (S130) .

As a result of the comparison, it is determined whether an error occurs in the position of the reference points (S140). If it is determined that an error has occurred in the position of the reference points, the error value is calculated and the photographed image is corrected according to the calculated error value (S150).

Thereafter, the image accumulation unit 310 of the image generation unit 300 accumulates and stores the corrected images output from the error correction unit 240 (S160), and the reference point extraction unit 320 outputs the corrected images to the image accumulation unit 310 A plurality of reference points are extracted from the stored correction image. At this time, the reference point extracting unit 320 extracts a plurality of reference points from the corrected image as priority reference points, and detects the corners of the lane in the corrected image to extract as additional reference points. The reference point extracting unit 320 detects the edges of the lane in a predetermined number of continuously accumulated accumulated images in consideration of the fact that both lanes of the vehicle are alternately displayed and extracts them as reference points.

On the other hand, the reference point matching unit 330 performs a back-projection transformation based on the extracted reference point to remove the perspective from the corrected image (S180). The reference point matching unit 330 compares coordinates of reference points commonly displayed on the transformed images to be arranged adjacent to each other according to the arrangement positions of the cameras 110 to 140 in the transformed images from which the perspective is removed (S190 ).

The reference point matching unit 330 determines whether an error occurs due to a difference between the coordinates of the reference points (S200). If it is determined that an error has occurred, the reference point matching unit 330 corrects the transformed image so that the coordinates of the converted image match each other (S210 ). Then, the image combining unit 340 matches the corrected transformed images and generates an AVM image representing the frontal direction of the vehicle in the form of a bird's-eye view (S220).

As described above, in the automatic AVM image correction and generation method, the step of accumulating and storing the corrected images (S160) to the step of correcting the transformed images 210 is performed in the body reference point extracting step (S120) to the photographed image correcting step May be merged and configured.

The method according to the present invention can be implemented as a computer-readable code on a computer-readable recording medium. A computer-readable recording medium includes all kinds of recording apparatuses in which data that can be read by a computer system is stored. Examples of the recording medium include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage, and the like. The computer-readable recording medium may also be distributed over a networked computer system so that computer readable code can be stored and executed in a distributed manner.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is evident that many alternatives, modifications and variations will be apparent to those skilled in the art.

Accordingly, the true scope of the present invention should be determined by the technical idea of the appended claims.

Claims (15)

An image acquiring unit disposed at a designated position of the vehicle, the image acquiring unit including a plurality of cameras each acquiring an image captured in a designated direction around the vehicle;
Extracting a plurality of reference images previously acquired and stored in each of the plurality of cameras and a plurality of reference points preliminarily designated as feature points of the vehicle body in each of the plurality of shot images, An image corrector configured to generate a plurality of correction images by correcting the plurality of photographed images using an error value obtained by comparing positions of reference points extracted from the photographed image; And
An image generating unit for generating an AVM image by matching a plurality of images output from the image correcting unit; And an automatic AVM image correction and generation device. The apparatus of claim 1, wherein the image correction unit
An image storage unit for storing a plurality of reference images obtained in each of the plurality of cameras in an initial state in which it is determined that the plurality of cameras are normally installed in the vehicle;
A reference point extracting unit for extracting the plurality of reference points from each of the plurality of reference images and the plurality of shot images;
And comparing the positions of the reference points corresponding to each other among the plurality of reference points extracted from the plurality of reference images and the plurality of shot images to obtain the error value with respect to the photographing direction of each of the plurality of cameras and the rotation angle of the camera An image comparison unit; And
An error corrector for correcting the plurality of photographed images according to the error value to generate the plurality of corrected images; And an automatic AVM image correction and generation device. The apparatus of claim 1, wherein the image generating unit
An image accumulating unit accumulating and storing the plurality of corrected images;
A reference point extracting unit for extracting points corresponding to the corners of the lane included in the corrected image as a plurality of reference points together with points corresponding to the plurality of extracted points in the plurality of corrected images stored in the image accumulating unit; And
A position of the reference points commonly included in the transformed images to be arranged adjacent to each other in the AVM image among the plurality of transformed images subjected to the inverse projection transformation is compared with each other, A reference point matching unit that corrects the plurality of transformed images using the determined transform error value; And
An image combining unit for generating the AVM image of bird's-eye view shape by arranging the plurality of converted images corrected by the reference point matching unit at positions corresponding to the arrangement positions of the plurality of cameras; And an automatic AVM image correction and generation device. 4. The apparatus of claim 3, wherein the reference point extracting unit
A method for detecting a corner of a lane alternately displayed on both sides of the vehicle on the road, comprising the steps of: detecting a corner of the lane in a corrected image continuously obtained within a predetermined time of the plurality of corrected images accumulated in the image accumulating unit; And the reference point for the AVM image is automatically detected. The apparatus of claim 1, wherein the image correction unit
Wherein the plurality of reference images obtained in each of the plurality of cameras and the plurality of shot images applied in the plurality of cameras are accumulated in an initial state in which it is determined that the plurality of cameras are normally installed in the vehicle, A storage unit;
A reference point extracting unit for extracting points corresponding to corners of the lane included in the photographed image together with the feature points of the vehicle body in each of the plurality of reference images and the plurality of photographed images as a plurality of reference points;
And comparing the positions of the reference points corresponding to each other among the plurality of reference points extracted from the plurality of reference images and the plurality of shot images to obtain the error value with respect to the photographing direction of each of the plurality of cameras and the rotation angle of the camera An image comparison unit; And
Wherein the plurality of correction images are generated by correcting the plurality of photographed images so as to compensate for the error values, and the plurality of correction images are backprojected, and the AVM image of the plurality of back- An error corrector for comparing the positions of the plurality of reference points commonly included in the transformed images to be disposed adjacent to each other to determine a transform error value and correcting the plurality of transformed images using the determined transform error value; And an automatic AVM image correction and generation device. The apparatus of claim 1, wherein the image obtaining unit
A front camera that photographs the front of the vehicle and is disposed on the radiator grill such that the top edge of the front plate of the vehicle to be extracted at the reference point is included in the photographed image;
A rear camera disposed between the rear lid of the vehicle and the rear bumper so as to photograph the rear of the vehicle and include a plurality of fixing screws of the rear plate of the vehicle to be extracted at the reference point in the taken image;
A left camera installed on the left side mirror of the vehicle so as to photograph the left side of the vehicle and include a left door handle and a front bumper of the vehicle to be extracted as the reference point in the taken image; And
A right camera installed on a right side mirror of the vehicle so as to capture the right side of the vehicle and include the right door handle of the vehicle to be extracted as the reference point and the front bumper; And an automatic AVM image correction and generation device. The apparatus of claim 1, wherein the image correction unit
Wherein the plurality of reference points are extracted every predetermined period or at a predetermined event occurrence time to obtain the error value, and if the period is not or the event is not generated, the plurality of shot images are corrected Wherein the automatic AVM image correcting and generating apparatus comprises: An automatic AVM image correction and generation method of an AVM image automatic correction and generation apparatus having an image acquisition unit, an image correction unit, and an image generation unit,
A plurality of cameras of the image acquiring unit are disposed at respective designated positions of the vehicle to acquire images photographed in a designated direction around the vehicle;
Extracting a plurality of reference images previously acquired and stored in each of the plurality of cameras by the image correction unit and a plurality of reference points preliminarily designated as feature points of the vehicle body in each of the plurality of shot images;
Comparing the reference points extracted from the plurality of reference images with the positions of reference points extracted from the plurality of shot images to obtain an error value;
Generating a plurality of correction images by correcting the plurality of photographed images using the error value; And
Generating the AVM image using the plurality of correction images; The method comprising the steps of: 9. The method of claim 8, wherein obtaining the error value comprises:
A plurality of reference images obtained in each of the plurality of cameras in an initial state in which it is determined that the plurality of cameras are normally installed on the vehicle body, and a plurality of reference points corresponding to each other among the plurality of reference points extracted from the plurality of shot images And comparing the positions of the points with each other to obtain the error value for the photographing direction of each of the plurality of cameras and the rotational angle of the camera. 9. The method of claim 8, wherein generating the AVM image comprises:
Accumulating and storing the plurality of corrected images;
Extracting points corresponding to the corners of the lane included in the corrected image as a plurality of reference points together with points corresponding to the plurality of extracted points in the cumulative stored images;
Generating a plurality of transformed images from which the perspective is removed by backprojecting the plurality of corrected images;
Comparing the positions of the plurality of reference points commonly included in the transformed images to be arranged adjacent to each other in the AVM image among the plurality of transformed images to determine a transform error value;
Correcting the plurality of transformed images using the determined transform error values; And
Generating the AVM image in the form of a bird's-eye view by matching and arranging the corrected plurality of transformed images at positions corresponding to arrangement positions of the plurality of cameras; Wherein the automatic AVM image correction and generation method comprises: 11. The method of claim 10, wherein extracting
The reference point for the corner of the lane in the corrected image successively obtained within a predetermined time of the plurality of accumulated correction images stored cumulatively to detect the corner of the lane alternately displayed on both sides of the vehicle on the road Wherein the AVM image is automatically corrected and generated. 9. The method of claim 8, wherein extracting the plurality of reference points comprises:
Accumulating and storing a plurality of reference images obtained in each of the plurality of cameras and the plurality of shot images applied in the plurality of cameras in an initial state in which it is determined that the plurality of cameras are normally installed in the vehicle, ; And
Extracting a point corresponding to an edge of a lane included in the photographed image as a plurality of reference points together with the feature points of the vehicle body in each of the plurality of reference images and the plurality of photographed images; Wherein the automatic AVM image correction and generation method comprises: 13. The method of claim 12, wherein generating the AVM image comprises:
Generating a plurality of transformed images from which the perspective is removed by performing backprojection transformation on the plurality of corrected images;
Comparing the positions of the plurality of reference points commonly included in the transformed images to be arranged adjacent to each other in the AVM image among the plurality of transformed images to determine a transform error value;
Correcting the plurality of transformed images using the determined transform error values; And
Generating the AVM image in the form of a bird's-eye view by matching and arranging the corrected plurality of transformed images at positions corresponding to arrangement positions of the plurality of cameras; Wherein the automatic AVM image correction and generation method comprises: 9. The apparatus of claim 8, wherein the plurality of cameras include a front camera, a rear camera, a left camera, and a right camera,
Wherein the front camera photographs the front of the vehicle and is arranged on the radiator grille so that the top edge of the front plate of the vehicle to be extracted to the reference point is included in the photographed image,
Wherein the rear camera photographs the rear of the vehicle and is disposed between the rear lid and the rear bumper of the vehicle so that the photographing image includes a plurality of fixing screws of the rear plate of the vehicle to be extracted at the reference point,
The left camera captures the left side of the vehicle and is installed on the left side mirror of the vehicle so that the left door handle and the front bumper of the vehicle to be extracted as the reference point are included in the captured image,
Wherein the right camera is mounted on the right side mirror of the vehicle such that the right side door of the vehicle to be extracted as the reference point and the front bumper are included in the photographed image. Calibration and generation methods. 9. The method of claim 8, wherein extracting the plurality of reference points comprises:
Wherein the plurality of reference points are extracted every predetermined period or at a predetermined event occurrence time.

Images