KR102372296B1 - Apparatus and method for recognize driving lane on image - Google Patents

Abstract

The present invention provides an image capturing unit that captures an original image around a vehicle, an AVM system that synthesizes the images of the image capturing unit and converts it into a top-view image, a navigation unit that provides lane information in the driving direction, and A lane detection unit that recognizes and detects information, a vehicle/motorcycle detection unit that detects the wheels of a vehicle or motorcycle around the driving vehicle and estimates the distance between the driving vehicle and the vehicle or motorcycle, a controller that controls the overall device, the driving lane is recognized The present invention relates to an apparatus and method for determining an image-based driving lane including an output unit for outputting information informing the user when the vehicle is detected.

Description

Image-based driving lane determination apparatus and method {Apparatus and method for recognize driving lane on image}

The present invention relates to a method of detecting and determining a driving lane, and more particularly, to an image-based driving lane determining apparatus and method for detecting all road lanes in a driving direction to determine a currently driving lane and provide the same.

Recently, a technology that detects a driving lane and warns a driver when the vehicle deviates from a lane has been adopted in a vehicle. The prior art is a method of detecting a driving lane by using a front image photographing, and an object of the prior art is to determine a driving lane from an image.

However, the prior art has a problem in that it is difficult to estimate in which lane the vehicle runs because it is configured to detect only the existence of a driving lane.

In order to solve the problems of the prior art as described above, the image-based driving lane determining apparatus and method according to an embodiment of the present invention determine all lanes in the driving direction using not only the front image of the vehicle but also the side image capturing And you can see which lane you are currently driving. An object of the present invention is to detect a driving lane by using the present invention, and to develop an Advanced Driver Assist System (ADAS) or unmanned autonomous driving technology.

The problems of the present invention are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

In order to achieve the above object, an image-based driving lane determining apparatus according to an embodiment of the present invention includes an image capturing unit that captures an original image with cameras on the front, left and right sides of a vehicle while driving, and the original image captured by the image capturing unit. AVM system that synthesizes and converts into a top-view image that is an image of the vehicle surroundings, recognizes a driving lane using the original image in front of the image capturing unit, estimates a lane interval from this, and uses the estimated lane interval to predict the position of the side lane, which is the left and right lane, and detect the side lane using the original side image of the imaging unit based on the predicted position of the side lane, and the number of lanes between the driving lane and the side road and a lane detector for determining all road lanes in the driving direction by adding , and an output unit for providing information on the driving lane and the side lane.

An apparatus and method for determining an image-based driving lane according to an embodiment of the present invention

Detecting the driving lane using the front original image of the imaging unit or the Top-View image of the AVM system, judging the side road lane using the side image, the side road through the detection of the vehicle or motorcycle wheel using the side image and determining a lane, and determining a driving lane through a navigation unit.

The specific details of other embodiments are included in the detailed description and drawings.

According to the image-based driving lane determining apparatus and method of the present invention, there are one or more of the following effects.

First, there is an advantage of accurately determining the driving lane by estimating the location of the driving lane according to the driving lane recognition.

Second, there is also the advantage of improving the performance of ADAS (Advanced Driver Assist System) or unmanned autonomous driving system by recognizing driving lanes and judging the location.

Effects of the present invention are not limited to the above-mentioned effects, and other effects not mentioned will be clearly understood by those skilled in the art from the description of the claims.

1 is a block diagram illustrating a configuration of an image-based driving lane determining apparatus according to an embodiment of the present invention.
2 is a block diagram illustrating a configuration of a driving lane recognition method based on a forward original image according to an embodiment of the present invention.
3 is an exemplary diagram of a method for recognizing a driving lane based on a Top-View image according to an embodiment of the present invention.
4 is an exemplary diagram of an image-based driving lane recognition method according to an embodiment of the present invention.
5 is an exemplary diagram of a method for detecting a wheel of a vehicle or motorcycle based on a side image according to an embodiment of the present invention.
6 is an exemplary diagram of a method for estimating a side image-based vehicle or motorcycle wheel position according to an embodiment of the present invention.
7 is an exemplary diagram of a method for determining a road lane using a navigation system according to an embodiment of the present invention.
8 is a flowchart illustrating a flow of a method for detecting and determining a driving lane according to an embodiment of the present invention.

Advantages and features of the present invention, and a method of achieving them, will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, and only these embodiments allow the disclosure of the present invention to be complete, and common knowledge in the art to which the present invention pertains It is provided to fully inform those who have the scope of the invention, and the present invention is only defined by the scope of the claims. Like reference numerals refer to like elements throughout.

Hereinafter, the present invention will be described with reference to drawings for explaining an apparatus and method for determining an image-based driving lane according to embodiments of the present invention.

1 is a block diagram illustrating a configuration of an image-based driving lane determining apparatus according to an embodiment of the present invention. 1 , the image-based driving lane determining apparatus according to an embodiment of the present invention includes an image capturing unit 110 , an AVM system 120 , a navigation unit 130 , a lane detection unit 140 , and a vehicle/motorcycle detection unit 150 , a control unit 160 , and an output unit 170 may be included.

The image capturing unit 110 captures original images of the front, left and right sides of the vehicle while driving. The image capturing unit 110 may include a front camera for photographing the front of the vehicle, a left camera for photographing an image on the left side of the vehicle, and a right camera for photographing an image on the left side of the driving vehicle.

The front camera may be installed in the center of the vehicle to photograph the road surface in front and the center of the vehicle while the vehicle is driving. The left camera may be installed on the left side of the vehicle to photograph the left lane and the center line when the vehicle is driving. The right camera may be installed on the right side of the vehicle to photograph the right lane while the vehicle is driving.

The AVM system 120 may output a Top-View image by synthesizing the original image captured by the image capturing unit 110 . The Top-View image is an image of the surroundings of the vehicle viewed from the top to the bottom of the vehicle.

The navigation unit 130 may detect a position of the driving vehicle and provide a position detection result that provides lane information in a driving direction of the driving vehicle.

The lane detection unit 140 may select a region in which a lane may exist in the original image of the image capturing unit 110 as a region of interest. According to the installation angle and the viewing angle of the image capturing unit 110, a region in which a lane may exist in the original image captured may be set as the region of interest.

The lane detection unit 140 may detect a driving lane of the driving vehicle, a side lane and a center line that are left or right sides of the driving lane, using the original image. Also, it is possible to detect a driving lane, a side lane, and a center line of the driving vehicle from the top-view image.

The vehicle/motorcycle detection unit 150 may detect the wheels of the vehicle or motorcycle from the left and right original images. In addition, the detected position of the wheel of the vehicle or motorcycle may be estimated.

The controller 160 controls overall devices such as the image capturing unit, the AVM system, the lane detecting unit, and the output unit.

The output unit 170 may provide the driving lane information to the driver. The output unit 170 may output the lane information including at least one of a display installed in the vehicle and an alarm.

2 is a block diagram showing the configuration of a driving lane recognition method based on an original image according to an embodiment of the present invention.

The lane detection unit 140 acquires an original image from the image capturing unit 110 (S210). The lane detection unit 140 selects a region of interest from the original image (S220). The lane detection unit 140 designates a region in the original image in which a lane may exist as a region of interest. A lane estimation component of the image is extracted from the region of interest using a morphology operation (S230). The lane estimation component is region-labeled to display the shape of the lane estimation component (S240). By using the morphology operation and region labeling, missing information may be filled with respect to the lane estimation components included in the region of interest, and image noise components generated in the background region may be removed.

A lane candidate group is detected among the lane estimation components (S250). A driving lane is detected in the lane candidate group using a line fitting process (S260).

3 is an exemplary diagram of a method for recognizing a driving lane based on a Top-View image according to an embodiment of the present invention.

Referring to FIG. 3A , the lane detection unit 140 receives a top-view image from the AVM system. A gray scale image is output from the Top-View image, and a lane center point is extracted using a Top-Hat filter. The Top-Hat filter may determine whether an amount of change in luminance between adjacent pixels in the gray scale image is greater than or equal to a threshold at which a minimum value is set. If the amount of change in luminance between adjacent pixels in the gray scale image is equal to or greater than a set threshold, the feature points 310 may be extracted on the left/right side of the lane based on the center point of the lane.

Referring to FIG. 3B , the lane detection unit 140 performs a line fitting process using the extracted feature points. The line fitting process extracts a line component by searching a 360-degree direction based on the feature points on the left/right side of the lane. The line component validity check is performed by comparing the angular error of the extracted line components. In this case, the constraint condition of the angle error is within 5 degrees. When the extracted line component satisfies the validity check, a lane candidate group is displayed based on the center point of the lane. The vehicle reference left/right representative lane is detected and determined as the driving lane 420 .

Referring to FIG. 3C , the lane detection unit 140 may detect a center line from the left image of the driving vehicle by using a double Top-Hat filter. The double top-hat filter may determine whether the amount of change in luminance between adjacent pixels in the gray scale image is greater than or equal to a threshold at which a minimum value is set. If the amount of change in luminance between adjacent pixels in the gray scale image is equal to or greater than a set threshold, the feature point 330 may be extracted on the left side of the lane based on the center point of the lane.

Referring to FIG. 3(d) , a representative line may be detected using the line fitting process. When a plurality of feature points exist, it is determined as the center line 340 .

4 is an exemplary diagram of an image-based driving lane recognition method according to an embodiment of the present invention.

Referring to FIG. 4A , a driving lane may be recognized by detecting a lane in the original image of the image capturing unit 110 . The image capturing unit 110 outputs the front original image to the lane detecting unit 140 through the front camera.

The lane detection unit 140 detects a lane 410 in the front original image according to the original image-based driving lane recognition method. A current driving lane is recognized using the detected lane.

A lane interval may be estimated using the driving lane. The lane interval may be estimated by calculating the widths of the left driving lane and the right driving lane from the front image.

Referring to FIG. 4B , the driving lane may be detected by detecting the lane in the top-view image of the AVM system 120 .

The AVM system 120 converts the original image of the image capturing unit 110 into a top-view image and outputs it to the lane detection unit 140 . The lane detection unit 140 detects a lane 420 from the top-view image according to a driving lane recognition method based on the top-view image. A current driving lane is recognized using the detected lane. A lane interval may be estimated using the driving lane.

Referring to FIG. 4C , the left road lane may be detected by detecting a lane and a center line in the left image of the vehicle of the image capturing unit 100 .

The image capturing unit 110 outputs the left original image of the driving vehicle captured by the left camera to the lane detecting unit 140 . The lane detection unit 140 designates a region in which a lane may exist in the left image of the driving vehicle as a region of interest. A center line and a left lane 430 of the vehicle may be detected in the region of interest.

The lane detection unit 140 detects a left lane 430 according to an image-based driving lane recognition method. Also, the lane detection unit 140 may detect a center line in the left lane.

A left road lane is determined using the left lane 430 .

Referring to FIG. 4D , the right road lane may be detected by detecting a lane in the image on the right side of the vehicle of the image capturing unit 100 .

The image capturing unit 110 outputs the right image of the driving vehicle captured by the right camera to the lane detecting unit 140 . The lane detector 140 designates a region in which a lane may exist in the image on the right side of the driving vehicle as a region of interest. A right lane 440 of the vehicle may be detected in the region of interest.

Referring to FIG. 4E , the lane detection unit 140 may determine the number of all road lanes in the driving direction by adding the number of side road lanes 450 that are the left and right lanes to the driving lane.

5 is an exemplary diagram of a method for detecting a wheel of a vehicle or motorcycle based on a side image according to an embodiment of the present invention.

The vehicle/motorcycle detection unit 150 may extract the characteristics of the wheels of the vehicle or motorcycle by using the Haar-like descriptor from the side images that are the left and right original images of the image capturing unit 110 . The vehicle/motorcycle detection unit 150 may detect the presence or absence of the vehicle or motorcycle wheel by using the adaboost learning algorithm from the extracted vehicle or motorcycle wheel characteristics.

The extracted vehicle or motorcycle wheel features are classified into an image feature 520 estimated as a wheel and an image feature 530 not estimated as a wheel by the classifier 510 .

When the image feature 520 estimated to be the wheel is detected according to the result of combining the classifier by searching the entire side captured image of the image capturing unit 110, it is determined that the wheel exists.

6 is an exemplary diagram of a method for estimating a side image-based vehicle or motorcycle wheel position according to an embodiment of the present invention.

The vehicle/motorcycle detection unit 150 estimates the position of the vehicle or motorcycle wheel when it is determined that the wheel of the vehicle or motorcycle is present in the side image that is the left and right original images of the image capturing unit 110 . The vehicle/motorcycle detection unit 150 detects a distance z between the image capturing unit 110 and the wheel of the vehicle or motorcycle.

The distance z is detected in consideration of the height H and the angle θ of the image capturing unit 110 . The height H means the installation height of the image photographing unit 110, the angle θ means the installation angle of the image photographing unit 110, the height H and the angle θ of the image photographing unit 110 are from the vehicle It is provided during the city and mass production process.

The vehicle/motorcycle detection unit 150 may detect the position of the vehicle or motorcycle wheel by (Equation 1) in consideration of the distance z.

(Equation 1)

In (Equation 1) used by the vehicle/motorcycle detector 150, (u0, v0) means a principal point, f means a focal length, and these internal parameters of the imaging unit 110 are , provided during mass production.

The vehicle/motorcycle detection unit 150 may estimate the position 610 of the vehicle or motorcycle wheel by calculating a distance between the detected vehicle or motorcycle wheel and the vehicle.

The vehicle/motorcycle detection unit 150 may determine a side lane of the driving vehicle by using the position 610 of the wheel.

7 is an exemplary diagram of a method for determining a road lane using a navigation system according to an embodiment of the present invention.

When added with the navigation 130 to the image-based driving lane determining device, the image-based driving lane determining device can know the number of all road lanes in the driving direction from the navigation unit, so that the road lane recognition result is based on the left and right cameras Even if some parts are not recognized, the current driving lane can be determined.

Referring to FIG. 7A , when information on all lanes of a driving road is known using the navigation device 130 , it is possible to recognize that the current driving lane is the second lane even if only the lane 710 of the left road is recognized.

Referring to FIG. 7B , when all lane information of a driving road is known using the navigation 130 , it is possible to recognize that the current driving lane is the second lane even if only the lane 720 of the right road is recognized.

8 is a flowchart illustrating a flow of a method for detecting and determining a driving lane according to an embodiment of the present invention.

The front camera of the image capturing unit 110 or the AVM system 120 outputs a front image of the driving vehicle ( 800 ).

The lane detection unit 140 detects a lane in the front image ( 805 ). The lane detection method through the front camera of the image capturing unit 110 is performed through the steps of obtaining an original image, selecting a region of interest, morphology calculation, region labeling, lane candidate group detection, and line fitting-based lane detection.

In addition, the lane detection method through the AVM system 120 is performed through the steps of extracting a feature point through a Top-Hat filter, line fitting using the feature point, detecting a lane candidate group, and recognizing a representative lane.

The lane detection unit 140 recognizes a driving lane from the detected lane ( 810 ). The representative lanes of the left and right regions of the vehicle reference may be determined as the driving lanes.

The lane spacing is estimated using the driving lane (S815). The lane interval may be estimated by calculating the widths of the left driving lane and the right driving lane from the front image.

The left and right lane positions of the driving lane are predicted ( 820 ). By using the width of the driving lane, the positions of the left and right lanes can be predicted.

The image capturing unit 110 captures an image with the left and right cameras of the vehicle. (825). The lane detector 140 selects a region of interest from the left/right images (S830). The region of interest is an area in which a lane may exist through prediction of positions of the left and right lanes of the driving lane.

The lane detection unit 140 detects a lane in the ROI ( 835 ). The lane detection unit 140 may extract a feature point from the region of interest using a top-hat filter, detect a left/right lane using the feature point, and detect a center line using a double top-hat filter. . A side road lane is determined using the detected left and right lanes and the center line ( 840 ).

The vehicle/motorcycle detection unit 150 detects wheels of the vehicle or motorcycle from the images of the left and right cameras of the vehicle of the image capturing unit 110 ( 845 ). The vehicle/motorcycle detection unit 150 extracts the characteristics of a vehicle or motorcycle wheel by using a Haar-like descriptor from the left and right images of the vehicle, and uses the adaboost learning algorithm to extract the characteristics of the wheel of the vehicle or motorcycle by using the adaboost learning algorithm. can be detected.

The vehicle/motorcycle detection unit 150 may calculate a distance between the detected vehicle or motorcycle wheel and the driving vehicle to estimate the detected vehicle or motorcycle wheel position 610 ( 850 ). The detected distance between the wheel of the vehicle or motorcycle and the driving vehicle may estimate the position of the vehicle or motorcycle wheel in consideration of the camera height and the camera installation angle of the image capturing unit 110 . The vehicle/motorcycle detection unit 150 calculates the presence and location of wheels of the vehicle or motorcycle, and determines the lane of the side road of the driving vehicle ( 855 ).

The navigation unit 130 receives information on the number of lanes on the road in the driving direction ( 860 ).

Since the lane detection unit 140 can know the number of all lanes in the driving direction through the navigation, it can determine the current driving lane even if the road lane recognition result based on the side image is not recognized.

The lane detection unit 140 determines all road lanes in the driving direction by using the driving lane and information on the left and right lanes ( 865 ). The lane detection unit 140 determines a driving lane of the driving vehicle through all the road lanes ( 870 ).

In the above, preferred embodiments of the present invention have been illustrated and described, but the present invention is not limited to the specific embodiments described above, and in the technical field to which the present invention belongs, without departing from the gist of the present invention as claimed in the claims Various modifications may be made by those of ordinary skill in the art, and these modifications should not be individually understood from the technical spirit or perspective of the present invention.

110: video recording unit
120: AVM system
130: navigation unit
140: lane detection unit
150: vehicle / motorcycle detection unit
160: control unit
170: output unit

Claims (11)

an image photographing unit for photographing an original image with cameras on the front, left and right sides of the vehicle while driving;
an AVM system for synthesizing the original image captured by the image capturing unit and converting it into a Top-View image that is an image of the vehicle surroundings;
Recognizing the driving lane using the original image in front of the imaging unit and estimating the lane interval from this,
Predicting the positions of side lanes that are left and right lanes using the estimated lane spacing,
Detecting the side lane using the original side image of the imaging unit based on the predicted position of the side lane,
a lane detection unit for determining all road lanes in a driving direction by adding the number of lanes of the driving lane and the side road; and
and an output unit providing information on the driving lane and the side lane. The method of claim 1,
An image further comprising a vehicle/motorcycle detection unit for detecting the wheels of a vehicle or motorcycle around the driving vehicle from the side images, which are the left and right original images of the image capturing unit, and estimating the distance between the driving vehicle and the vehicle or the motorcycle based driving lane judgment device. The method of claim 1,
The image-based driving lane determining device further comprising a navigation unit that provides information on a driving direction lane of the driving vehicle to the lane detection unit. detecting a lane by using the original image in front of the imaging unit;
recognizing a driving lane using the lane;
estimating a lane interval using the driving lane;
predicting positions of side lanes that are left and right lanes using the lane spacing;
selecting the predicted position of the side lane within the side image, which is the left and right original images of the imaging unit, as a region of interest;
detecting a side lane in the region of interest;
determining a lane of a side road by using the side lane;
determining all road lanes in a driving direction by adding the number of lanes of the driving lane and the side road; and
determining a driving lane using all the road lanes;
An image-based driving lane determination method comprising a. 5. The method of claim 4,
The step of detecting a lane using the original image in front of the image capturing unit comprises:
obtaining an original image of the front of the vehicle in the image capturing unit;
selecting a region of interest from the original image;
performing a morphology calculation so that only a lane is visible in the region of interest;
area labeling the lane on which the morphology operation has been performed;
detecting a lane candidate group in the region-labeled image; and
detecting a lane in the lane candidate group based on line fitting;
An image-based driving lane determination method comprising a. 5. The method of claim 4,
The image-based driving lane determination method further comprising the step of detecting a lane using the top-view image of the AVM system. 7. The method of claim 6,
The step of detecting a lane using the Top-View image of the AVM system comprises:
extracting feature points from the Top-View image using a Top-Hat filter;
Line fitting step using the feature point;
Detecting the line fitting candidate group, and
detecting a lane by recognizing a representative lane from among the candidate lanes;
An image-based driving lane determination method comprising a. 5. The method of claim 4,
The image-based driving lane determination method further comprising the step of determining a lane of a side road through detection of a vehicle or motorcycle wheel using the side image. 9. The method of claim 8,
The step of determining the lane of the side road by detecting the wheel of the vehicle or motorcycle by using the side image,
An image-based driving lane determination method for detecting a vehicle or motorcycle wheel using a Haar-like descriptor and an adaboost learning algorithm in the side image, and estimating the detected vehicle or motorcycle wheel position. 5. The method of claim 4,
The image-based driving lane determination method further comprising the step of determining the driving lane using a navigation unit. 11. The method of claim 10,
The step of determining the driving lane using the navigation unit,
An image-based driving lane determination method for acquiring information on all road lanes in the driving direction using the navigation unit to determine a current driving lane even if a side lane of the driving vehicle is not recognized.

Images