KR102044098B1 - Apparatus and method for calibrating blind spot detection - Google Patents

Abstract

An object of the present invention is to provide an apparatus for calibrating blind spot detection (BSD) capable of using internal parameters of an image recognition algorithm according to a location change of a camera without tuning, and a method therefor. To this end, the apparatus comprises: a panel recognition unit configured to detect markers from a camera image in which a calibration panel including the plurality of markers and patterns around the markers is captured so as to extract coordinates of the markers; an image generation unit configured to transform a viewpoint of the camera image based on the coordinates of the markers to generate an aerial view image; a pattern correction unit configured to extract the patterns around the markers from the aerial view image, compare the extracted patterns with normal patterns of a normal aerial view image, and adjust the coordinates of the markers to minimize a difference between the extracted patterns and the normal patterns; a BSD recognition unit configured to extract a rotation angle for image transformation to be used in BSD recognition from the camera image based on the adjusted coordinates of the markers and a boundary region to be cut from the aerial view image to determine a view image region; and a storage unit configured to store determined calibration data between original images corresponding to pixel locations of a view image to be generated by applying the determined rotation angle and boundary region. Therefore, the present invention can use internal parameters of an image recognition algorithm according to a location change of a camera without tuning, thereby providing performance that is the same as or similar to that of recognition performance that is developed and well-tuned in a conventional case.

Description

BSD calibration device and method {APPARATUS AND METHOD FOR CALIBRATING BLIND SPOT DETECTION}

The present invention relates to a BSD calibration apparatus and method, and more particularly to a BSD calibration apparatus and method that can use the internal parameters of the image recognition algorithm according to the change in the position of the camera without tuning.

In general, a blind spot detection system (BSD) is known as a system that prevents an accident by monitoring a surrounding by installing a camera in a vehicle, and such a BSD system is a blind spot of a driver's rear blind spot. Ultrasonic sensors, radar sensors or cameras are used to detect the other vehicle, for example.

A BSD system using a camera receives a plurality of camera images installed in a vehicle, and generates a composite image from a virtual viewpoint by perspective transformation.

At this time, it is necessary to accurately calculate in advance which position and in which direction the camera installed in the vehicle is mounted, and this calculation is called a calibration of the camera.

As a calibration method, there is a method of photographing a feature point that already knows a coordinate position with each camera, and using data in which coordinates on the camera image of each feature point correspond to actual spatial coordinates.

Meanwhile, in the case of an aftermarket, a camera installed in a vehicle is installed after the vehicle is shipped. Accordingly, the installation position and direction of the camera may vary depending on the type or model of the vehicle. Even when installed in the installation position or direction, there is a problem that an error or the like may occur during the installation process and thus malfunction or distorted information may be output.

In addition, in the BSD system, the image recognition technology changes the area to be searched from the input image, the size, shape, and distribution of pixel values in the input image according to the installation position of the camera, the ambient lighting conditions, and the like. It is necessary to set (tune) the internal decision value, ie the parameter to an optimal value.

However, the general BSD system has a lot of internal tuning parameters, so it is almost impossible to tune these parameters individually in a short time.

Korean Registered Patent Publication No. 10-1245529 (Invention name: Camera calibration method)

In order to solve this problem, an object of the present invention is to provide a BSD calibration device and method that can use the internal parameters of the image recognition algorithm according to the change of the position of the camera without tuning.

In order to achieve the above object, the present invention includes a panel recognition unit for extracting the coordinates of the marker by detecting the marker in the camera image photographed a calibration panel including a plurality of markers and the pattern around the marker; An image generating unit generating a bird's eye view image by converting the viewpoint of the camera image based on the coordinates of the marker; A pattern beam for extracting a pattern around the marker from the bird's eye view image and comparing the extracted pattern with a normal pattern of a normal bird's eye image to adjust a coordinate value of the marker such that a difference between the extracted pattern and the normal pattern is minimized government; A BSD recognizer configured to determine a view image region by extracting a rotation angle for image conversion to be used for BSD recognition from a camera image and a boundary region to be cut out from the bird's eye view image based on the coordinate value of the marker; And a storage unit configured to store the calibration data determined between the original image corresponding to the pixel positions of the view image to be generated by applying the determined rotation angle and the boundary region.

In addition, the BSD recognition unit according to the present invention is characterized in that for calculating the position corresponding to the BSD region of the bottom surface in the bird's eye view image.

In addition, the BSD recognition unit according to the present invention is characterized by calculating the horizontal position by calculating an arbitrary distance corresponding to infinity in the bird's eye view image.

The BSD recognizing unit according to the present invention calculates a rotation angle so that the angle of the horizontal line is horizontal, and calculates a view image area in the bird's eye view image including the BSD region and the ground below the horizontal line having a certain height. It is characterized by.

In addition, the calibration data according to the invention is characterized in that it is stored as a look up table (LUT).

The method may further include a) detecting a marker by obtaining a camera image photographed by a panel including a plurality of markers and a calibration panel including a pattern around the marker, and extracting coordinates of the detected marker; b) generating a bird's eye view image by converting a viewpoint of the camera image based on the coordinates of the marker; c) extracting a pattern around a marker from the bird's eye view image, and analyzing the extracted pattern to compare it with a normal pattern of a normal bird's eye image; d) as a result of the comparison in step c), it is determined whether the difference between the extracted pattern and the normal pattern is a minimum value, and if it is not the minimum value, the pattern correction unit coordinates the marker such that the difference between the extracted pattern and the normal pattern is a minimum value. Adjusting the value; e) As a result of the comparison in step c), the difference between the extracted pattern and the normal pattern is the minimum value, and the rotation angle for image conversion to be used for BSD recognition in the camera image based on the coordinate value of the adjusted marker by the BSD recognizer. Determining a boundary area to be cropped from the bird's eye view image; And f) the BSD recognizing unit generating calibration data of the view image area to be generated by applying the determined rotation angle and the boundary area, and storing the generated calibration data in a storage unit.

In addition, the view image area according to the present invention is characterized in that it comprises a region corresponding to the BSD area of the bottom surface in the bird's eye view image.

In addition, the view image area according to the present invention is characterized in that it comprises an arbitrary horizon area in the bird's eye view image.

In addition, the view image area according to the invention is characterized in that it comprises at least a portion of the ground area below the horizontal line.

In addition, the calibration data according to the present invention is stored as a look up table (LUT) between the original image corresponding to the pixel positions of the view image area to be generated.

The present invention can use the internal parameters of the image recognition algorithm according to the change in the position of the camera without tuning, there is an advantage that can provide a performance equivalent to or similar to the previously developed well tuned recognition performance.

In addition, the present invention has an advantage of shortening the parameter tuning process of the image recognition algorithm according to the change in the position of the camera.

1 is a block diagram showing the configuration of a BSD calibration device according to the present invention;
Figure 2 is an exemplary view for explaining the operation of the BSD calibration device according to the present invention.
3 is an exemplary view showing a calibration panel of the BSD calibration device according to the present invention.
4 is an exemplary view for explaining a view image extraction process of the BSD calibration device according to the present invention.
5 is a flowchart illustrating a BSD calibration method according to the present invention.

Hereinafter, exemplary embodiments of a BSD calibration apparatus and method according to the present invention will be described in detail with reference to the accompanying drawings.

In this specification, the expression "comprising" a certain component means that it may further include other components rather than exclude other components.

In addition, the terms "... part", "... group", "... module" and the like refer to a unit for processing at least one function or operation, which may be divided into hardware, software, or a combination of the two.

1 is a block diagram showing a configuration of a BSD calibration device according to the present invention, Figure 2 is an exemplary view for explaining the operation of the BSD calibration device according to the present invention, Figure 3 is a BSD calibration device according to the present invention 4 is an exemplary diagram illustrating a calibration panel, and FIG. 4 is an exemplary diagram illustrating a view image extraction process of a BSD calibration apparatus according to the present invention.

1 to 4, the BSD calibration device 100 according to the present invention receives an input image of a recognition system in which a parameter of an image recognition algorithm is set to an optimal value, and an image converted into a similar or identical image as an input. As a configuration that can be used, the camera 200 is calibrated through a camera image of the calibration panel 400 placed on the floor (or the ground) to be similar or identical to the input image of the recognition system. The panel recognition unit 110, the image generating unit 120, the pattern correcting unit 130, and the BSD recognition unit 140 are configured to provide the converted image.

The panel recognition unit 110 photographs the calibration panel 400 including a plurality of markers 410, 411, 420, 421 and patterns around the markers 410, 411, 420, 421 through the camera 200. The markers 410, 411, 420, and 421 are detected by receiving the received camera image, and the coordinates of the detected markers 410, 411, 420, and 421 are extracted.

That is, the panel recognition unit 110 searches for the markers 410, 411, 420, and 421 of the calibration panel 400 existing in the camera image to extract coordinate values, and the extracted coordinate values are an image generator ( The bird's eye view generated at 120 may be used for adjustment of the image.

Here, the calibration panel 400 is a pattern in which a plurality of markers 410, 411, 420, 421 having concentric patterns and a plurality of square lattice patterns are disposed at equal intervals around the markers 410, 411, 420, 421. And a first panel region P1 including a second panel region and a second panel region P2 on which a first guide portion 440 and a second guide portion 441 are disposed to perform an optimal calibration. The second panel region P2 allows the first guide part 440 to be positioned on the rear rear wheel 310 of the vehicle 300.

In addition, the markers 410, 411, 420, and 421 may include markers 410 and 420 having a predetermined size and an auxiliary marker 411 having a concentric pattern that is relatively smaller than the concentric circles of the markers 410 and 420. 421, the markers 410 and 420 located far from the camera 200 are designed to be large so that the markers can be easily recognized.

In addition, the calibration panel 400 may perform precise position recognition by using the intersection points of patterns in which square grids are arranged at equal intervals as checkers 430, 431, 432, and 433.

When the arrangement of the calibration panel 400 is completed, photographing is performed through the camera 200, and a camera image including the vehicle 300 and the calibration panel 400 is photographed through the camera 200.

In addition, the calibration panel 400 is made of different colors, for example, black and yellow, black and white so that the marker and the pattern can be easily distinguished from the bottom surface.

In addition, when there is lens distortion, the panel recognition unit 110 may perform distortion correction on the camera image.

The image generating unit 120 generates a bird's eye view image by converting the camera image based on the coordinates of the extracted markers 410, 411, 420, and 421. The bird's eye view image is a storage unit 150. The lookup table stored in the LUT may be generated through perspective transformation.

That is, the image generating unit 120 performs perspective conversion in such a manner that the position data of the image on the bottom surface is projected onto the screen plane having the focal length from the position of the camera.

The pattern correction unit 130 extracts a pattern around the markers 410, 411, 420, and 421 from the bird's-eye view image, and extracts the extracted pattern from a normal bird's-eye view image (for example, an image recognition algorithm parameter is optimal). And a coordinate value of the markers 410, 411, 420, and 421 so as to minimize the difference between the extracted pattern and the normal pattern in comparison with the normal pattern of the input image of the recognition system set as the value. Camera image including the calibration panel 400 acquired at a predetermined position from the position where the camera, lens, camera installation angle, installation direction, etc. are installed at different positions according to the Adjust the coordinate values to

In addition, the pattern correction unit 130 corrects the bird's eye view image by adjusting the coordinates of the markers 410, 411, 420, and 421 in subpixel units.

In addition, the pattern correction unit 130 may be performed a plurality of times the coordinates and pattern comparison process per one marker, iteratively performed until the optimum coordinate value of the marker through the pattern comparison.

The BSD recognizer 140 extracts a rotation angle for image conversion to be used for BSD recognition and a boundary region to be cut out from the bird's eye view image, based on the coordinate values of the adjusted markers 410, 411, 420, and 421. Determine the view image area.

In addition, the BSD recognizer 140 recognizes the first BSD area 510, which is a dangerous area, and the second BSD area 520, which is a warning area, from the bird's eye view image through recognition of a calibration panel. Image positions corresponding to the BSD region 510 and the second BSD region 520 are extracted through calculation.

In addition, the BSD recognizer 140 extracts the horizontal line 530 by calculating an arbitrary distance corresponding to infinity.

That is, if the arbitrary position is far away from infinity, it corresponds to the horizontal line, and the position of the horizontal line is extracted by calculating the position on the image while moving x with respect to the infinitely far y in the bird's eye view image.

In addition, the BSD recognition unit 140 extracts the horizontal line, calculates a rotation angle so that the angle of the horizontal line is horizontal, and includes the first and second BSD regions 510 and 520 and the extracted horizontal line The view image area 540 is calculated from the bird's eye view image such that the bottom area below the area 530 includes a portion.

That is, the BSD recognizer 140 provides a view image obtained by cutting a part of the bird's eye view image as an image converted into a similar or identical image to an input image of a recognition system in which a parameter of an image recognition algorithm is set to an optimal value. The image is such that the image region l in the y-axis direction constitutes at least two thirds of the bottom region below the horizontal line.

In addition, the BSD recognition unit 140 includes the first and second BSD regions 510 and 520 in the x-axis direction to determine the rotation angle for image conversion to be used for BSD recognition and the boundary region to be cropped. The view image area 540 to which the rotation angle and the boundary area are applied is configured.

In addition, the BSD recognizer 140 may look up a table for the determined calibration data between the determined boundary area and the original image corresponding to the pack cell positions of the image to be generated by applying the rotation value. ), And the generated lookup table is stored in the storage 150.

The lookup table may be newly generated and stored, or the information of the already stored lookup table may be updated and stored.

Next, a BSD calibration method according to the present invention will be described with reference to FIGS. 1 to 5.

The panel recognition unit 110 captures the calibration panel 400 including the plurality of markers 410, 411, 420, 421 and patterns around the markers 410, 411, 420, 421 through the camera 200. The markers 410, 411, 420, and 421 are detected by acquiring a camera image, and the coordinates of the detected markers 410, 411, 420, and 421 are extracted (S100).

The image generator 120 generates a bird's-eye view image of the camera image based on the coordinates of the markers 410, 411, 420, and 421 extracted by the panel recognition unit 110 (S110).

The pattern corrector 130 extracts and analyzes a pattern around the markers 410, 411, 420, and 421 from the bird's-eye view image generated by the image generator 120 (S120), and analyzes the extracted pattern in a normal bird's-eye view. The comparison with the normal pattern of the image (S130) determines whether it is an optimal value.

As a result of the comparison in step S130, if the difference between the extracted pattern and the normal pattern is not an optimal minimum value, the pattern correction unit 130 may display the markers 410, such that the difference between the extracted pattern and the normal pattern becomes a minimum value. The coordinate values 411, 420, and 421 are adjusted (S140).

In addition, as a result of the comparison of step S130, the difference between the extracted pattern and the normal pattern is the minimum value, and the BSD recognizer 140 adjusts the markers 410, 411, 420, and 421 adjusted by the pattern corrector 130. On the basis of the coordinate value of, a rotation angle for image conversion to be used for BSD recognition in the camera image and a boundary region to be cut out from the bird's eye view image are determined to extract the view image region (S150).

After performing step S150, the BSD recognizer 140 generates calibration data of the view image area 540 to be generated by applying the determined rotation angle and the boundary area, and the generated calibration data is stored in the storage unit ( In operation S160, the image is stored as a look up table (LUT) between the original image corresponding to the pixel positions of the view image area to be generated.

Therefore, an internal parameter of the image recognition algorithm according to a change in the position of the camera may be provided by configuring the input image of the recognition system in which the parameter of the image recognition algorithm is set to an optimal value and an image converted into a similar or identical image. Can be used without tuning.

As described above, it has been described with reference to a preferred embodiment of the present invention, but those skilled in the art various modifications and changes of the present invention without departing from the spirit and scope of the present invention described in the claims below I can understand that you can.

In addition, the reference numerals described in the claims of the present invention are not limited to the above described for clarity and convenience of the description, but are not limited thereto, the thickness of the lines shown in the drawings, the size of the components, etc. May be exaggerated for clarity and convenience of the description, and the above terms are terms defined in consideration of functions in the present invention, and may vary according to the intention or custom of the user or operator, and thus interpretation of these terms Will be made based on the contents throughout the specification.

100: BSD calibration device 110: panel recognition unit
120: image generating unit 130: pattern correction unit
140: BSD recognition unit 200: camera
300: vehicle 310: rear wheel
400: calibration panel 410, 420: marker
411, 421: auxiliary markers 430, 431, 432, 433: checkers
440: first guide portion 441: second guide portion
500: camera image 510: first BSD region
520: second BSD area 530: the horizontal line
540: view image area

Claims (10)

The markers 410, 411, 420, and 421 in the camera image in which the calibration panel 400 including the plurality of markers 410, 411, 420, and 421 and a pattern around the markers 410, 411, 420, and 421 is captured. A panel recognition unit 110 for detecting the coordinates of the markers 410, 411, 420, and 421;
An image generator 120 generating a bird's eye view image by converting the camera image from the viewpoint based on the coordinates of the markers 410, 411, 420, and 421;
Extracting a pattern around the markers 410, 411, 420, and 421 from the bird's eye view image, and comparing the extracted pattern with a normal pattern of a normal bird's-eye view image to minimize the difference between the extracted pattern and the normal pattern. A pattern corrector 130 that adjusts coordinate values of the markers 410, 411, 420, and 421;
BSD recognition to determine a view image area by extracting a rotation angle for image conversion to be used for BSD recognition and a boundary region to be cut out from the bird's eye view image based on coordinate values of the adjusted markers 410, 411, 420, and 421. Unit 140; And
And a storage unit (150) for storing the determined calibration data between the original image corresponding to the pixel positions of the view image to be generated by applying the determined rotation angle and the boundary region. The method of claim 1,
The BSD recognizing unit 140 calculates a position corresponding to the BSD region (510, 520) of the bottom surface in the bird's eye view image. The method of claim 2,
The BSD recognizing unit 140 calculates the position of the horizontal line 530 by calculating an arbitrary distance corresponding to infinity in the bird's eye view image. The method of claim 3, wherein
The BSD recognizer 140 calculates a rotation angle so that the angle of the horizontal line is horizontal, and includes the BSD regions 510 and 520 and the view image in the bird's eye view image so that the bottom surface below the horizontal line has a certain height. BSD calibration device, characterized in that it calculates an area (540). The method of claim 4, wherein
And the calibration data is stored as a look up table (LUT). a) The panel recognition unit 110 acquires a camera image in which the calibration panel 400 including the plurality of markers 410, 411, 420, 421 and a pattern around the markers 410, 411, 420, 421 is photographed Detecting the markers (410, 411, 420, 421) and extracting coordinates of the detected markers (410, 411, 420, 421);
b) the image generating unit 120 converts the camera image based on the coordinates of the markers 410, 411, 420, and 421 to generate a bird's eye view image;
c) extracting, by the pattern corrector 130, patterns around the markers 410, 411, 420, and 421 from the bird's-eye view image, analyzing the extracted patterns, and comparing the extracted patterns with normal patterns of the normal bird's-eye view image;
d) As a result of the comparison in step c), it is determined whether the difference between the extracted pattern and the normal pattern is a minimum value, so that the difference between the extracted pattern and the normal pattern is the minimum value if the difference is not the minimum value. Adjusting coordinate values of the markers (410, 411, 420, 421);
e) As a result of the comparison in step c), the difference between the extracted pattern and the normal pattern is a minimum value, and the BSD recognizer 140 based on the coordinate values of the adjusted markers 410, 411, 420, and 421. Determining a rotation angle for image conversion to be used for BSD recognition in a camera image and a boundary area to be cropped from the bird's eye view image; And
f) generating, by the BSD recognizing unit 140, calibration data of the view image area 540 to be generated by applying the determined rotation angle and the boundary area, and storing the generated calibration data in the storage unit 150. BSD calibration method comprising a. The method of claim 6,
And the view image area (540) includes an area corresponding to the BSD area (510, 520) of the bottom surface in the bird's eye view image. The method of claim 7, wherein
And the view image area (540) comprises an arbitrary horizontal line (530) area in the bird's eye view image. The method of claim 8,
The view image area (540) comprises at least a portion of the bottom surface area below the horizontal line (530). The method of claim 9,
The calibration data is stored in a look up table (LUT) between the original image corresponding to the pixel positions of the view image area to be generated, BSD calibration method.

Images