KR101464489B1 - Method and system for detecting an approaching obstacle based on image recognition - Google Patents

Abstract

The present invention relates to a method and a system for detecting an obstacle approaching a vehicle based on image recognition. According to the present invention, input images can be accurately read and the obstacle can be detected even when only a single pantoscopic camera of an around-view system is used. The method for detecting an obstacle approaching a vehicle based on image recognition according to the present invention includes a first step for correcting a radial distortion in an image that is captured by a camera of a vehicle; a second step for setting an area of the corrected image where a vehicle collision risk is present as an attention area; a third step for extracting a road sign in at least the attention area and performing weakening processing thereon; a fourth step for extracting a boundary (hereinafter, referred to as an ″obstacle lower end boundary″) between a lower end of the obstacle and a ground surface in the image in which the road sign is weakening-processed; a fifth step for detecting an obstacle having a movement (hereinafter, referred to as a ″moving obstacle″) by using the obstacle lower end boundary; a sixth step for determining whether a lower end boundary of the moving obstacle intrudes into the attention area; and a seventh step for issuing a warning event when the intrusion is considered to occur as a result of the determination in the sixth step.

Description

[0001] METHOD AND SYSTEM FOR DETECTING AN APPROACHING OBSTACLE BASED ON IMAGE RECOGNITION [0002]

The present invention relates to a method and system for detecting a vehicle obstacle, and more particularly, to a method and system for detecting an obstacle approaching a vehicle by reading an image captured through a camera (in particular, And more particularly, to a method and system for detecting vehicle access obstacles based on image recognition.

Various types of obstacles exist around the vehicle and attempt to detect obstacles to protect them from potential risk factors.

2. Description of the Related Art In general, a vehicle access obstacle sensing device or system detects an obstacle at a very close distance from a vehicle through a sensor such as an ultrasonic wave or a laser, and informs the driver of the obstacle.

However, in order to detect a vehicle access obstacle by using an image captured through a camera installed in a vehicle instead of the ultrasonic sensor or the laser sensor, a method different from the above sensor is required.

Camera-based obstacle detection can provide details about obstacles as well as detection of obstacles as opposed to ultrasonic or laser sensors.

In particular, when the camera is an Around View system, it is common to use a wide angle camera capable of securing a wide field of view.

However, in the wide-angle camera, radial distortion in which the degree of distortion is determined by the refractive index of the convex lens inevitably arises, and consequently, the visual distortion of the display image as well as the image recognition of the image processing apparatus It may cause an error.

In the case of using the wide-angle camera, since the feature varies depending on the position of the camera, a predetermined technique is required to recognize the obstacle from the image obtained from the wide-angle camera.

A conventional method and system for detecting obstacles through image recognition include a method of measuring distances using two sets of cameras, i.e., stereo vision, as in the prior arts 1 to 3, and detecting obstacles based on the distance It was common.

However, it is common and efficient that the AroundView system is composed of four cameras installed at the front and rear of the vehicle and one at the bottom of the left and right side mirrors. In order to recognize the image for obstacle detection, It is very inefficient to install a total of eight cameras, and it is necessary to use a single wide-angle camera to recognize images and detect obstacles.

Prior Art 1: Korean Utility Model No. 20-1999-0026799 (Published on July 15, 1999) Prior Art 2: Korean Patent No. 10-0882670 (Registered Date: February 02, 2009) Prior Patent 3: Korean Patent Laid-Open No. 10-2000-0037604 (Disclosure Date: July 05, 2000)

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and it is an object of the present invention to provide an image recognition system capable of accurately reading an input image and detecting an obstacle, And a method and system for detecting a vehicle access obstacle.

According to an aspect of the present invention, there is provided a method for detecting a vehicle-access obstacle based on image recognition, the method comprising: a first step of correcting radial distortion in an image taken through a camera of a vehicle; A second step of setting an area of interest in the corrected image as a region of interest with a vehicle; Extracting a road mark in at least the ROI and weakening it; A fourth step of extracting a boundary between the bottom of the obstacle and the ground (hereinafter, referred to as an 'obstacle bottom boundary') in the image in which the road marking is weakened; A fifth step of detecting a moving obstacle (hereinafter referred to as a moving obstacle) using the lower boundary of the obstacle; A sixth step of determining whether a lower boundary of the moving obstacle has invaded into the ROI; And a seventh step of generating a warning event when the invasion is detected as a result of the sixth step.

According to the method and system for detecting a vehicle access obstacle based on image recognition according to the present invention, an input image can be accurately read and an obstacle can be detected by using only a single wide-angle camera of the surround view system. Thus, the monetary and spatial efficiency And it is possible to provide detailed information on obstacles as well as detection of the presence of obstacles, thereby preventing crashes caused by insufficient driving or carelessness during low-speed operation.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is an overall block diagram of a vehicle access obstacle detection system based on image recognition according to the present invention; FIG.
FIG. 2 is a block diagram showing a detailed configuration of the image recognition processing unit shown in FIG. 1. FIG.
3 is a block flow diagram illustrating a flow of a method for detecting a vehicle access obstacle based on image recognition according to the present invention.
4 is a conceptual diagram for explaining an image correction algorithm according to the present invention.
FIG. 5 (a) is an example showing an image distortion step of the image correction step of the present invention.
5 (b) is an example showing a corrected image by the image correction step of the present invention.
6 illustrates an example of a region of interest set according to a preferred embodiment of the present invention.
7 (a) is an example of the first input image of the photographed image according to the present invention.
7 (b) is an example showing an image in which a road mark is weakened according to the present invention.
8 is an example of a horizontal component size image processed according to the obstacle bottom boundary detection method of the present invention.
9 is an example of a binarized image showing a binarized lower boundary region of an obstacle according to the present invention.
10 is an example of the detection of the lower boundary of the obstacle according to the present invention.
11 is a diagram showing a result of detection of a moving obstacle during a vehicle stop according to the present invention.
12 is an example of an entropy-based obstacle candidate region extraction image according to the present invention.
13 is an example of an absolute value difference image according to the present invention.
FIG. 14 is an example of the detection result of a mobile obstacle during driving of a vehicle access obstacle detection system based on the image recognition of the present invention.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, the terms "comprises" or "having" and the like refer to the presence of stated features, integers, steps, operations, elements, components, or combinations thereof, But do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or combinations thereof.

Also, in this specification, when an element is referred to as being "connected" or "connected" with another element, the element may be directly connected or directly connected to the other element, It should be understood that, unless an opposite description is present, it may be connected or connected via another element in the middle.

In the following, preferred embodiments, advantages and features of the present invention will be described in detail with reference to the accompanying drawings.

FIG. 1 is an overall block diagram of a vehicle access obstacle detection system based on image recognition according to the present invention, and FIG. 2 is a block diagram illustrating a detailed configuration of an image recognition processor shown in FIG.

1 and 2, the vehicle-based obstacle detection system based on image recognition according to the present invention is configured to include a camera 10, an image recognition processing unit 30, and a warning output unit 20.

The camera 10 of the present invention is a component mounted on a vehicle and photographing a running road around the vehicle to acquire an image. Preferably, a CMOS camera or a CCD camera can be employed.

The camera 10 of the present invention may in particular be an Around View system, and such an Around View system typically uses a wide angle camera capable of securing a wide field of view. Here, the surround view system includes at least four cameras installed at least one at the front and rear of the vehicle and at the bottom of the left and right side mirrors.

The image recognition processing unit 30 of the present invention recognizes an obstacle in an image acquired by the camera 10 and performs arithmetic processing for determining the risk of collision with the obstacle, .

The road marking processing unit 33, the obstacle bottom boundary extracting unit 34, the moving obstacle detecting unit 35 (see FIG. 3) ), And a collision risk discrimination section (36).

The image correcting unit 31 of the present invention corrects the radiation distortion in the image input from the camera 10. [

According to the preferred embodiment, the image correcting unit 31 is configured to correct the radial distortion using the horizontal and vertical lengths of the photographed image.

The image partitioning unit 32 of the present invention divides an area having a risk of collision with a vehicle into an area of interest in the image corrected through the image correction unit 31 and sets the area.

According to a preferred embodiment, the image partition 32 sets the region of interest as a trapezoidal shape, with the upper side of the trapezoid being located in the lower region of the missing line in the corrected image, As shown in FIG.

The road marking processing unit 33 of the present invention extracts road markings at least in the area of interest and performs a function of weakening the extracted road markings.

According to a preferred embodiment, the road marking processing unit 33 is configured to perform the weakening process by re-adjusting or replacing pixel values of the extracted road markings.

The road marking processing unit 33 extracts road markings using the brightness difference between the ground and the road markers, divides the area of interest into a plurality of areas, sets different threshold values for the respective areas, and extracts road markers .

The obstacle bottom boundary extracting unit 34 of the present invention has a function of detecting a boundary between the bottom of the obstacle and the ground (hereinafter, referred to as an 'obstacle bottom boundary') in the image in which the road marking is weakened through the road marking processor 33 do.

According to the preferred embodiment, the obstacle bottom boundary extracting unit 34 is configured to extract the road mark using the horizontal component calculated through the Sobel operator in the image subjected to the weakened road marking.

In addition, the obstacle bottom boundary extracting unit 34 is configured to finally extract the obstacle bottom boundary from which the residual noise due to the road marking is removed based on the horizontal component of the obstacle bottom boundary extracted through the Sobel operator.

Here, the residual noise refers to the fact that even if the road markers are removed (that is, weakened) through the road marking processor 33, the corresponding road markers are not completely removed and still function as noise. Therefore, since the boundary of the landmark area is still low in the area of interest, it acts as noise, and it is desirable to further perform the operation of removing residual noise due to such road markings.

At this time, the residual noise removal is configured to be performed through smoothing and a morphology operator.

The moving obstacle detecting unit 35 of the present invention has a function of detecting an obstacle having motion or movement (hereinafter referred to as a moving obstacle) by using the lower boundary of the obstacle extracted through the obstacle lower boundary extracting unit 34 do.

According to a preferred embodiment, the moving obstacle detecting unit 35 uses the lower boundary of the obstacle extracted through the obstacle lower boundary extracting unit 34, and detects the upper pixel of the lower boundary of the obstacle within an arbitrary section of the region of interest And is configured to detect whether the obstacle has movement (movement).

Here, if the vehicle is at rest, it is configured to use the moving average image and the absolute value difference image of the current image together to detect the moving obstacle.

If the vehicle is in motion, texture information extracted by local entropy calculation for the upper region of the obstacle lower boundary, absolute image difference image of the current image of the image and the previous image, And to detect the moving obstacle by using motion information calculated from the difference image.

The collision risk judging unit 36 of the present invention determines whether or not the lower boundary of the moving obstacle detected through the moving obstacle detecting unit 35 has invaded into the area of interest.

According to a preferred embodiment, the collision risk determiner 36 is configured to determine whether or not the collision is detected using the vertical axis position information of the lower boundary of the moving obstacle detected through the moving obstacle detector 35.

The distance between the moving obstacle and the vehicle, which is located in the area of interest, is calculated by calculating the width of the area of interest corresponding to the height of the vertical axis of the detected moving obstacle The degree of approach of the obstacle).

The alarm output unit 20 of the present invention functions to output a visual or audible alarm to the outside according to the signal sent by the collision risk discrimination unit 36 so that the driver can recognize the fact.

According to a preferred embodiment, the warning output unit 20 preliminarily warns that a collision danger obstacle has appeared when the vehicle has entered an upper area of the area of interest, and when the vehicle is moving beyond the upper area of the area of interest to the lower area, The final warning is issued. Here, it is preferable that the external output forms corresponding to the preliminary warning and the final warning are formed to be different from each other so that the operator can grasp the current situation accurately.

Hereinafter, a method of detecting a vehicle access obstacle in an input image based on the image recognition processing unit 30 as described above and a method of alarming a danger to a driver of the vehicle will be described in detail.

3 is a block flow diagram illustrating a flow of a method for detecting a vehicle access obstacle based on image recognition according to the present invention.

Referring to FIG. 3, the method for detecting a vehicle approaching obstacle based on image recognition according to the present invention includes: (1) correcting radial distortion in an image taken through a camera of a vehicle; Setting a region of interest in the corrected image as a region of concern with the vehicle (Step 2); Extracting at least the road markings in the ROI and weakening the extracted road markings (step 3); Extracting a boundary between the bottom of the obstacle and the ground in the weakened image (step 4); Detecting an obstacle with movement using the obstacle bottom boundary (step 5); Determining whether the lower boundary of the moving obstacle has invaded into the area of interest (step 6); And a seventh step of generating a warning event when it is determined that the invasion is detected as a result of the sixth step.

Advantageously, said step 5 is configured to remove noise from said road markings in said weakened image, and to detect said obstacle lower boundary in said noise-removed image.

Preferably, the attenuation of the road mark in step 3 is configured to process the extracted road mark pixel value in a manner such that it is readjusted or replaced.

Hereinafter, specific embodiments of the respective steps will be described in detail.

≪ Step 1 (S10)

The camera of the present invention may be an Around View system in particular, and such an Around View system generally uses a wide angle camera capable of securing a wide field of view. However, in the wide-angle camera, radial distortion in which the degree of distortion is determined by the refractive index of the convex lens inevitably arises, and consequently, the visual distortion of the display image as well as the image recognition of the image processing apparatus It may cause an error.

Step 1 of the present invention is a step of correcting radial distortion in an image photographed through a camera of the vehicle, in particular an arousal view system, so that the image recognition processor 30 can recognize the photographed image more accurately.

Conventional image distortion correction is generally performed by assigning a measured parameter to a distortion correction polynomial using a known pattern such as a chess board, which is disadvantageous to image noise.

On the other hand, the image distortion correction of the present invention is characterized in that it is configured to follow an image correction algorithm for correcting a distorted area by radial distortion using the horizontal and vertical lengths of the photographed image.

4 is a conceptual diagram for explaining an image correction algorithm according to the present invention. The relationship between the coordinates (x, y) of the distorted image of FIG. 4 and the coordinates (x ', y') of the image whose distortion has been corrected according to the image correction algorithm of the present invention is expressed by the following equations (1) and same. On the other hand, since the ratio of the horizontal length (horizontal width) to the vertical length (vertical width) of an image is not necessarily 1: 1, a different proportional expression is applied to each of the X and Y axes.

(In equations (1) and (2)

x, y: distortion image coordinates,

x ', y': Image coordinates (corrected image coordinates) in which the distortion is corrected,

x _u , y _u : image center coordinate,

W, H: width and height of the image,

L _X , L _Y : Three-dimensional distance in the x and y directions between the image center coordinate and the distortion image coordinate)

Preferably, a memory for storing the correction coordinates calculated by the equations (1) and (2) is allocated to ensure real-time processing of the image correction for step 1, and the corrected image coordinates corresponding to the respective distorted image coordinates are stored in a lookup And stores it as a lookup table.

FIG. 5A is an example showing an image distortion in the image correction step of the present invention, and FIG. 5B is an example showing an image corrected by the image correction step of the present invention.

Referring to FIG. 5, it can be seen that the image photographed by the Around View system is getting worse as the image is farther from the center of the image due to the radial distortion, and the image corrected according to the image correction algorithm of the present invention The horizontal and vertical structures can be maintained even if the distance from the center is far different from that of the distorted image, and it is possible to maintain the horizontal and vertical structure, and it is judged from the road surface to the ground (hereinafter referred to as the ground) It is possible to detect an unspecified obstacle existing on the ground.

≪ Step 2 (S20)

Step 2 of the present invention is performed by the image dividing section 32, by dividing a predetermined region in the corrected image obtained through Step 1 and setting it as a region of interest. Here, the region of interest (ROI), which is defined in the correction image, is a partial area arbitrarily partitioned in the entire area of the image photographed by the camera. This area is an area in which a collision may occur , A region where there is a risk of collision with the vehicle).

According to a preferred embodiment, the upper boundary of the ROI can be set in the lower region of the disappearance line, and the boundary between the ground and the vehicle front end (or the rear end) (hereinafter, referred to as 'vehicle boundary') can be set as the lower boundary of the ROI . Herein, the disappearance line means a line corresponding to a point (vanishing point) which meets with the horizon among a plurality of lines which are divided in the horizontal direction along the vertical axis in the whole image, and the disappearance line in the image is the height of the camera from the ground, Angle, and focal length.

This is because the lower boundary surface of all the obstacles existing on the ground on the image obtained through photographing is present in the lower region of the disappearance line and the upper region of the vehicle boundary (for example, the vehicle hood) The distance is very long, so it is not necessary to detect it in the surrounding view system which is a low-speed driving environment.

FIG. 6 is an example showing an area of interest set according to a preferred embodiment of the present invention. In the embodiment of FIG. 6, the region of interest 40 is configured in a trapezoidal shape, with the upper side of the sadrilok (i.e., the upper boundary of the region of interest 40) ) Is configured to be set at the boundary or near the front end (or the rear end) of the vehicle.

The area of interest 40 is also divided into two areas consisting of an upper area 41 and a lower area 42 to alert the vehicle that a dangerous obstacle has appeared on entry into the upper area 41, (41) to warn that the risk of collision is imminent when invading the lower region (42).

Preferably, the upper region 41 and the lower region 42 define an upper region 41 as a region close to the vanishing line with respect to a half point of the trapezoidal height in the trapezoidal shape of the region of interest 40 , And a region close to the vehicle boundary can be set as the lower region 42. [

In other words, the approaching degree of the obstacle to the occupant driving vehicle, that is, the inflow of the upper region 41 and the inflow of the lower region 42, is determined by the vehicle width corresponding to the height of the vertical coordinates of the detected obstacle in the ROI 40 (I. E., The width of the area of interest 40) to determine how far the obstacle has approached the location.

≪ Step 3 (S30) >

Step 3 of the present invention is a step of extracting a road sign 50 (e.g., a lane, a direction guide, etc.) and performing a process of weakening the road mark 50 on the image, The road markings existing in the ROI 40 may be targeted, but all of the road markings appearing in the entire image may be extracted. However, this case is not excluded.

Step 3 is performed by the road marking processing unit 33, and in particular, the weakening processing of the road marking 50 is performed by re-adjusting or replacing the pixel values thereof.

Since the road mark is located on the ground (road surface) in the area of interest set in step 2, the height can be assumed to be '0', and the road markers in the ROI can be extracted can do.

That is, the road mark 50 always exists in the lower area of the disappearance line. Also, in the surround-view system, the road marking is not always directed toward the disappearance line (vanishing point), and the color of the road marking is displayed in various colors. In addition, the road marking often has a bright value in comparison with the road surface. Also, the boundary between the obstacle and the road surface always has a very dark value.

Therefore, the road markers are extracted using the difference in brightness between the ground and the road markers, and the road markers are divided into a plurality of road markers (preferably 3 And a road mark candidate area is extracted by setting a different threshold value for each area.

In more detail, since the darkest portion of the road markers is assumed to be a yellow road mark, the pixel values of the yellow road markers are extracted from the photographed image through experiments to secure various threshold values. This can vary depending on the performance of the camera sensor.

Also, the reason why different regions of interest are divided into plural regions and different threshold values are applied to each region is that the wide-angle camera has different pixel values according to coordinates appearing even if they are the same obstacle.

When the extraction of the road marking is completed according to the above method, the process of weakening the extracted road marking is performed by manipulating the pixel values of the extracted road marking. That is, the manipulation of the pixel value is to readjust or replace the initial pixel value of the road marking with another pixel value. The pixel value to be substituted for the extracted road marking refers to the closest pixel in the road marking candidate region pixel, .

FIG. 7 (a) is an initial input image of the photographed image according to the present invention, and FIG. 7 (b) is an example showing an image in which a road mark is weakened according to the present invention. As can be seen from FIG. 7, the first image inputted through the camera is displayed clearly and strongly on the ground. However, according to the present invention, the road marking 51 has a weak road mark It can be seen that it is removed.

≪ Step 4 (S40)

Step 4 of the present invention is a step of extracting a boundary between the bottom of the obstacle and the ground (hereinafter, referred to as 'obstacle bottom boundary') in the image in which the pixel values of the road markers are readjusted through step 3.

According to a preferred embodiment, the obstacle bottom boundary detection method of step 4 is carried out through the step 3 through the Sobel operator in the image in which the road marking is weakened, thereby generating a magnitude image of the horizontal component in the image And the horizontal component thus generated is used to extract the lower boundary of the obstacle.

Here, the Sobel operator for calculating the horizontal component is expressed by the following equation (3).

(In Equation 3,

f: horizontal component size image,

Gx: Horizontal Sobel image)

Sobel operators can use masks of various sizes and shapes, and inaccuracies exist because they satisfy or approximate real-time properties. Thus, according to a preferred embodiment, the mask for the Sobel operation is used as follows.

First, using a large mask has a disadvantage that it can obtain more accurate results because it uses a large number of pixels, but it also reappears to the area where the road mark is removed. Here, a mask having a large size means a mask of 5x5 or more.

Next, the mask of small size is sensitive to noise, so that even a small discontinuous area existing on the road surface is extracted, but it is not remarkable, so it is preferable to use a mask of small size. Here, a small size mask means a mask having a size of '1 × 3' or '3 × 3'.

8 is an example of a horizontal component size image processed according to the above-described obstacle bottom boundary detection method. The horizontal component has a high value at the lower boundary 60 portion of the obstacle within the horizontal interest region. Thus, as shown in FIG. 8, the magnitude of the horizontal component can be extracted from the image of the step 3 through the operation by the Sobel operator in step 4.

In this way, the horizontal component extracted through the Sobel operation in Step 4 may be directly processed to the obstacle lower boundary 60, but in this case, there is an inaccuracy due to the noise 55 due to the application marker.

Therefore, in order to more accurately recognize the image and detect the lower boundary of the obstacle, it is preferable to perform the removal after the residual noise is removed. Here, the residual noise 55 refers to that the road mark is not completely removed even if the road mark is removed (i.e., weakened) through step 3, and still acts as noise.

In the region of interest, since the boundary of the landmark region still exists at a low value, it acts as noise, and it is preferable to further include a task of removing residual noise due to such road markings. Here, the step of removing the residual noise will be referred to as 'step 4-1'.

≪ Step 4-1 (S41) >

Step 4-1 of the present invention is a step of finally extracting the obstacle bottom boundary from the image obtained by removing the noise 55 from the road marking based on the horizontal component of the obstacle bottom boundary extracted through the Sobel operator in step 4 .

According to a preferred embodiment, the noise cancellation in step 4-1 is performed particularly through smoothing and a morphology operator, and a specific processing flow thereof is as follows.

(1) A pixel value of a region of interest is flattened using a simple blur mask that calculates an average value in a mask. In this case, since the overall pixel value of the image is lowered, discontinuous points having lower values are eliminated.

(2) Morphology operation removes the road marking boundary and road surface noise components. Here, since the horizontal component of the road marking area is not maintained constant and can not be completely removed only by simple blur, a region with small connectivity is removed through a closing operation.

(3) Through the Gaussian Blur, the obstacle bottom border area is preserved and other areas are removed. Gaussian blur has a large effect of associating neighboring pixels, so that the size of the mask is set to be large and the edge portion having a small correlation with the surrounding values is made flat.

Here, since the Gaussian blur appears so that the effect is noticeable from a size of 5 × 5 or more, it is preferable to use a mask having a size of 5 × 5 or more.

(4) A cumulative histogram is generated for the horizontal component image of the region of interest and binarization is performed. The pixel value corresponding to any one of the values selected from the lower 70% to 80% range (preferably 75%) in the cumulative histogram of '0' to '255' is calculated. Is used as a threshold value for generation.

(5) Inspect the lowest value among vertical pixels having a value other than '0' on the basis of each horizontal pixel of interest. That is, the horizontal pixel is shifted from the left side to the right side (preferably from the upper left corner to the upper right corner), and a lowermost value having a value other than '0' is detected with respect to the vertical axis. And stores the detected lowermost values in a buffer. At this time, if the value does not exist or the lowermost value is detected in the region above the disappearance line (vanishing point), it is not displayed.

The value stored in the buffer through the above process corresponds to the lower boundary of the obstacle to be extracted in step 4, and is displayed in an image form as shown in FIG. 9 is an example of a binarized image showing an obstacle lower boundary region 61 according to the present invention. The black color shown in FIG. 9 corresponds to a pixel value '0' The region corresponds to the pixel value '255'.

10 is an example of the detection of the lower boundary of the obstacle according to the present invention. The obstacle bottom boundary 62 extracted through the above-described steps is applied not only to the detection of the stationary obstacle, but also to the step of detecting the moving obstacle. That is, if an obstacle exists in an arbitrarily designated period (preferably, the region of interest), a predetermined event is generated and used to determine the degree of risk of collision later.

Hereinafter, the moving obstacle detection step (step 5, S50) will be described in detail.

≪ Step 5 (S50)

Step 5 of the present invention is a step of detecting a moving obstacle (hereinafter referred to as a moving obstacle) by using the obstacle bottom boundary. The moving obstacle detection step uses the extracted lower boundary of the obstacle. Specifically, it detects whether or not the obstacle moves in the upper pixel of the lower boundary of the obstacle within an arbitrary section within the area of interest. At this time, if the vehicle is in a stopped state, a moving average image and an absolute value difference image are used. If the vehicle is running, a moving obstacle is detected using texture, motion information, and absolute difference image of the image .

(1) Detect moving obstacles when the vehicle is stopped

Moving obstacle detection at the time of stopping the vehicle operates based on the difference image. Gaussian blur is performed to minimize the change of the image obtained in the outdoor environment because the previous image and the current image may be different due to environmental factors such as illumination change and weather.

And, moving averages are used to operate robustly against moving background elements like shaking branches. The moving average image is defined by the following equation (4).

(In Equation 4,

S _t : moving average image,

S _{t -1} : average image accumulated up to t-1,

I _{t -1} : t-1 th image

α: weight for the t-th image)

Here, 'α' is a parameter for adjusting the influence of the average image accumulated up to the previous frame, so that the recent value has a larger weight than the previous value, so that it has a larger average value than the general average image.

A difference image is generated by calculating the absolute difference value of each pixel of the moving average image S _t and the current image I _t . Subtracts the current image from the moving average image, and extracts the portion where the large change exists in the foreground. Here, the foreground includes moving obstacles with motion.

Then, the noise component is removed from the extracted foreground region using the dilation of the morphology operation, the Gaussian Blur, and the median filter, and the motion of the obstacle is detected.

FIG. 11 shows an example of the results of detection of moving obstacles at the time of vehicle stop according to the present invention. In FIG. 11, the obstacle caught in the square block 65 is a moving obstacle.

(2) Detect moving obstacles when driving

Motion detection, motion information, texture information, and absolute difference images are used to detect moving obstacles when driving. In the case of vehicle driving, when the vehicle stopping method is applied, a region serving as a background is also expressed. Therefore, the candidate region is extracted using the texture information, and the absolute difference image is applied to the motion information as the weight.

Local entropy is calculated for the region beyond the lower boundary region of the obstacle in order to extract the obstacle candidate region using the texture. Entropy is a high value in a non-uniform region, and entropy is calculated within a block of arbitrary size because it can be assumed that obstacles existing on the road mostly include non-uniform regions.

Here, the block having the arbitrary size may be changed according to the image size, and preferably, a common divisor between the horizontal length and the vertical length of the image may be calculated and set. For example, if the image size is '320 × 240', the size of the block may be '16 × 16', '8 × 8', '4 × 4', or the like.

The method for calculating the entropy in the block is expressed by Equation (5).

(In equation (5)

P _i : probability of i-th gray level,

n _i : frequency of i-th gray level,

M, N: the horizontal and vertical size of the block,

e: entropy in the block)

FIG. 12 shows an example of an entropy-based obstacle candidate region extraction image according to the present invention. The image is divided into blocks 70 of an arbitrary size and an area having a high entropy is extracted and displayed. Referring to FIG. 12, entropy is recalculated in a divided block 70 by dividing the block into small blocks having a high entropy so as to have robust characteristics with respect to the size change of the obstacle. The region having a high entropy is set as an obstacle candidate region, and motion information is extracted only for a set candidate region.

Lucas-Kanade Optical Flow is used to extract motion information. Lucas-Kanade Optical Flow computes the velocity vector only for pixels whose errors appear below the minimum error in consecutive frames, and excludes otherwise.

In the real vehicle image, Lucas-Kanade Optical Flow violates the assumption of small and consistent motion, so motion information for independent moving obstacles is extracted using the absolute difference image with low accuracy.

13 is an example of an absolute value difference image according to the present invention, which is an absolute value difference image which does not use a moving average unlike the case where the vehicle is stopped. In the difference image of FIG. 13, only a pixel value for an obstacle candidate region using entropy is extracted and a moving object is detected and displayed. In FIG. 13, this corresponds to a portion indicated by a white line 80.

In this case, since the motion of the image exists even in the area corresponding to the background when the vehicle is moving, and information of accumulated previous frames is rather noise, only the difference between the current frame (image) and the previous frame The equation for applying the weight is used as the weight of the motion information, as shown in Equation (6).

(In Equation 6,

w (x, y): weight of motion information with respect to (x, y)

d (x, y) is the (x, y) coordinate pixel value of the absolute difference image,

α: constant)

Then, the motion vector is calculated by multiplying the velocity vector for each pixel by the weight calculated by Equation (6) as shown in the following Equation (7). The motion information is calculated as a result of increasing the value of the velocity vector of the moving obstacle and restricting the velocity vector to other background areas.

14, if a speed vector of a speed vector equal to or higher than a specific threshold value is extracted from the speed vectors calculated through the above-described method, a moving obstacle at the time of driving the vehicle is detected as shown in FIG. 14, It becomes an obstacle.

Here, extracting only the velocity vector exceeding the specific threshold value may mean extracting only a vector having a velocity vector of '1' or more. Since the velocity vector of '1' or more is obtained by the weighting formula of Equation (6) when the obstacle moves.

≪ Step 6 (S60)

Step 6 of the present invention is a step of determining whether or not the lower boundary of the moving obstacle detected through step 5 has invaded into the ROI, and more specifically, determining whether the invasion has occurred using the vertical axis position information of the detected lower boundary of the obstacle And further to generate a detailed warning event according to the distance between the obstacle and the vehicle.

According to a preferred embodiment, the distance between the obstacle and the vehicle may be calculated through a vehicle width calculation method as follows. The distance between the obstacle and the vehicle is calculated by calculating the size of the width of the area of interest (i.e., width of the vehicle) corresponding to the point corresponding to the height of the vertical axis of the detected obstacle in the area of interest, The relative change in the width of the area of interest calculated thereby can be used to determine how far the obstacle has approached the location.

≪ Step 7 (S70)

Step 7 of the present invention is a step of audibly or visually informing a driver of the vehicle that the lower boundary of the obstacle has entered the ROI through step 6.

In the case of a stationary obstacle, since there is no risk of collision if the lower boundary of the obstacle does not move to the bottom of the image, only an edge of the road surface at the bottom of the obstacle is displayed and an event indicating that an obstacle exists at a specific distance can be performed.

And is configured to perform a warning event when a moving obstacle bottom boundary has entered an arbitrarily defined area (for example, a region of interest) due to the large displacement of motion in the case of a moving obstacle.

According to a preferred embodiment, the region of interest 40 is divided into two regions consisting of an upper region 41 and a lower region 42, as shown in FIG. 6, so that when the vehicle enters the upper region 41, And to warn that the risk of collision is imminent when the vehicle goes over the upper region 41 to the lower region 42. [ Here, whether or not the obstacle has entered the upper area or the lower area of the ROI can be determined through the vehicle width calculation method described above.

On the other hand, since it is determined that the risk of collision is small in the case of an object moving at the same speed, it is possible to configure not to perform the warning event when it is determined that the object is moving toward the top using motion information.

While the preferred embodiments of the present invention have been described and illustrated above using specific terms, such terms are used only for the purpose of clarifying the invention, and it is to be understood that the embodiment It will be obvious that various changes and modifications can be made without departing from the spirit and scope of the invention. Such modified embodiments should not be understood individually from the spirit and scope of the present invention, but should be regarded as being within the scope of the claims of the present invention.

10: camera, 20: alarm output unit
30: image recognition processing unit 31: image correction unit
32: image dividing unit 33: road marking processing unit
34: obstacle bottom boundary extracting unit 35: moving obstacle detecting unit
36: Collision risk discrimination part 40: Interest area
50,51: Road marking 62: Obstacle bottom boundary

Claims (14)

A first step of correcting the radiation distortion in the image photographed through the camera of the vehicle;
A second step of setting an area in which there is a risk of collision with a vehicle in the corrected image as an area of interest;
A third step of extracting road markings in at least the area of interest and weakening the extracted road markings;
A fourth step of extracting a boundary between the bottom of the obstacle and the ground (hereinafter referred to as an 'obstacle bottom boundary') in the image in which the road marking is weakened;
A fifth step of detecting a moving obstacle (hereinafter referred to as a moving obstacle) using the lower boundary of the obstacle;
A sixth step of determining whether a lower boundary of the moving obstacle has invaded into the ROI; And
And a seventh step of generating a warning event when it is detected as a result of the determination of the sixth step.
The method according to claim 1,
Wherein the weakening process of the third step is configured to process a pixel value of the extracted road marking by a method of rearranging or replacing the pixel value of the extracted road marking.
The method according to claim 1,
The road markers of the third step are extracted by using the brightness difference between the ground and the road markers, and the road markers are extracted by dividing the ROI into a plurality of regions and setting different threshold values for the respective regions Wherein the obstacle detection means detects the vehicle obstacle based on the image recognition.
The method according to claim 1,
Wherein the step of extracting the lower boundary of the obstacle in the fourth step is performed using the horizontal component calculated through the Sobel operator in the image obtained by weakening the road marking through the third step. Based vehicle access obstacle detection method.
5. The method of claim 4,
In the fourth step,
After calculating the horizontal component through the Sobel operator,
(4-1) planarizing pixel values of a region of interest using a simple blur mask;
(4-2) removing a road marking boundary and a road surface noise component through a morphology operation;
A fourth step of preserving the lower boundary region of the obstacle through the Gaussian Blur and removing the other regions;
A fourth step of generating an accumulated histogram for the horizontal component and performing binarization; And
And a fourth step of detecting a lowest value among vertical pixels having a value other than '0' based on each horizontal pixel of the ROI, and storing the lowest value in a buffer. Detection method.
6. The method of claim 5,
In the step 4-4, a pixel value corresponding to at least one selected from the range of 70% to 80% in a cumulative histogram of '0' to '255' is calculated, and the calculated pixel value is binarized Wherein the threshold value is used as a threshold value for generating an image.
The method according to claim 1,
In the fifth step,
Wherein the controller is configured to detect whether or not the obstacle is moved on an upper pixel of the obstacle bottom boundary within an arbitrary section of the ROI using the lower boundary of the obstacle extracted through the fourth step (Method of detecting vehicle access obstacle based on image recognition).
8. The method of claim 7,
Detecting a moving obstacle using a moving average image and an absolute value difference image of the current image if the vehicle is in a stopped state,
If the vehicle is running, texture information extracted by local entropy calculation for the upper region of the obstacle lower boundary, absolute difference image of the current image and the previous image, And detecting the moving obstacle based on the calculated motion information.
The method according to claim 1,
In the sixth step,
Wherein the step of determining whether the vehicle is approaching is based on the vertical axis position information of the lower boundary of the moving obstacle detected through the fifth step.
10. The method of claim 9,
And calculates a distance between the moving obstacle and the vehicle that has entered the area of interest using the distance between the moving obstacle and the vehicle, The method comprising the steps of: (a) detecting an obstacle in a vehicle;
The method according to claim 1,
Wherein the first step is configured to correct the radial distortion using the horizontal and vertical lengths of the photographed image.
The method according to claim 1,
Wherein the area of interest in the second step is set in a trapezoidal shape, wherein the upper side of the trapezoid is set to be located in a lower area of the missing line in the corrected image, and the trapezoid is divided into two areas of an upper area and a lower area Wherein the obstacle detection means detects the vehicle obstacle based on the detected obstacle.
13. The method of claim 12,
In the seventh step,
Wherein the vehicle is configured to warn that a collision hazard obstacle has appeared when the vehicle has entered the upper region, and to warn that the risk of a collision is imminent when the vehicle goes beyond the upper region to the lower region. Way.
A camera for photographing the vicinity of the vehicle to acquire an image;
An image correcting unit correcting a radiation distortion in an image input from the camera;
An image dividing unit for dividing and setting an area where there is a risk of collision with a vehicle in the corrected image as a region of interest;
A road marking processor for extracting road markings in the ROI and attenuating the road markings;
An obstacle bottom boundary extracting unit for detecting a boundary between the bottom of the obstacle and the ground (hereinafter, referred to as 'obstacle bottom edge') in the image in which the road marking is weakened;
A moving obstacle detector for detecting an obstacle with movement (hereinafter referred to as a 'moving obstacle') using the obstacle lower boundary;
A collision risk judging unit for judging whether or not the lower boundary of the moving obstacle has invaded into the region of interest; And
And a warning output unit for generating a warning event when the collision risk discrimination unit discriminates that the collision is detected.

Images