KR20160123203A - Road sign detection-based driving lane estimation method and apparatus - Google Patents

Abstract

Disclosed are a method and an apparatus for estimating a driving lane based on detection of road signs. According to an embodiment of the present invention, the method for estimating a driving lane based on detection of road sign comprises the following steps of: extracting a road sign candidate area from a predetermined area of interest of an image obtained from a camera; determining an actual road sign area by verifying the extracted road sign candidate area; determining whether the determined actual road sign area is in a left direction or a right direction; calculating a three-dimensional location of a lower end point of the actual road sign with respect to the determined direction; and estimating the location of a driving lane where a vehicle is being driven by using the calculated three-dimensional location on the lower end point. The apparatus for estimating a driving lane based on detection of road signs can guide a driving lane in a more correct and convenient manner.

Description

Technical Field [0001] The present invention relates to a road sign detection-based driving lane estimation method and apparatus,

BACKGROUND OF THE INVENTION 1. Field of the Invention [0002] The present invention relates to a road sign detection based driving lane estimation method and apparatus.

For advanced driver assistance systems and autonomous driving, it is essential to accurately estimate the position of the vehicle. Particularly, it is very important to accurately identify the lane on which the vehicle is traveling because it can be used for various applications such as precise road guidance, accurate traffic situation analysis, and forward warning through communication.

GPS and other satellite-based navigation systems are not suitable for finding out a lane because there are errors of up to several tens of meters in downtown areas where high-rise buildings and underground roads are frequent.

In addition, the IMU-based inertial navigation-based method has a limitation in that the position error continuously increases with time because the vehicle motion is continuously accumulated.

The method of using the precision map has a limitation that it is necessary to use a large-capacity precise map generated in advance by expensive equipment and it is difficult to use when the road marking is blocked by other vehicles.

In order to solve the problems of the prior art described above, the present invention proposes a driving lane estimation method and apparatus based on road sign detection that can solve the limitations of radio navigation, inertial navigation and precision map-based methods.

According to an embodiment of the present invention, there is provided a method of extracting a road sign candidate region, the method comprising: extracting a road sign candidate region within a predetermined area of interest of an image acquired from a camera; Determining an actual road sign area through verification of the extracted road sign candidate area; Determining whether the determined actual road sign area is in the left or right direction; Calculating a three-dimensional position of the lower end point of the actual road sign area with respect to the determined direction; And estimating a position of the lane in progress using the calculated three-dimensional position of the lower end point.

The step of estimating the position of the lane may include: approximating the calculated three-dimensional position of the lower end point to the end position of the roadway; And dividing a lateral distance from the camera to the lower end to a standard lane width.

The step of determining the actual road sign area may include extracting features describing characteristics of the road sign candidate area using the brightness and color information of the road sign candidate area; And determining whether the road sign candidate region is the actual road sign region using the extracted feature.

The lower end point may be a lower right end point if the actual road sign area is the right direction and a left lower end point if the actual road sign area is the left direction.

The step of calculating the three-dimensional position of the lower endpoint may include calculating a height of the road sign from the ground, a height from the ground of the camera, a focal length in pixels, an ordinate of the principal point of the camera, Calculating a distance between the camera and the actual road sign area using at least one of the vertical coordinates of the lower end point of the actual road sign area included; And at least one of a distance between the camera and the actual road sign area, an abscissa of a lower end point of the actual road sign area included in the image, an abscissa of the camera principal point, And calculating a lateral distance from a lower end point of the actual road sign area to a car on which the camera is attached.

The area of interest may be set using camera external variables indicating the installation state of the camera and installation height information of the road signs according to the installation regulations.

According to another aspect of the present invention, there is provided an image processing method comprising the steps of: extracting a road sign area from an image acquired from a camera; Determining whether the extracted road sign area is in the left or right direction; Calculating a three-dimensional position of the lower end point of the road sign area using the determined direction, a distance between the camera and the road sign, and a road sign installation rule; And estimating a position of the lane in progress using the calculated three-dimensional position of the lower end point.

According to another aspect of the present invention, there is provided a computer-readable recording medium on which a program for performing the above-described method is recorded.

According to another aspect of the present invention, there is provided a driving lane estimation apparatus comprising: a processor; And a memory coupled to the processor, wherein the memory extracts a road sign candidate region within a predetermined region of interest of the image acquired from the camera, and verifies the extracted road sign candidate region to obtain a road sign region Determining whether the determined road sign area is in the left or right direction, calculating the three-dimensional position of the lower end point with respect to the determined direction, and calculating the three-dimensional position of the ongoing lane There is provided a driving lane estimation device for storing program instructions executable by the processor to estimate a position.

According to the present invention, it is expected that the following effects will occur because it is possible to recognize a driving lane of a car in a congested urban area.

  First, from the viewpoint of navigation, it is possible to carry out more accurate and convenient route guidance based on the route to the information and the destination by the present driving vehicle. For example, in a situation where a right turn must be made from the front, if the vehicle is in the first lane, it can be induced to change the lane at an earlier point in time.

  Second, in the advanced driving support system and the autonomous driving side, it is possible to efficiently recognize the surrounding environment and support the vehicle control by using the preliminary information based on the driving lane information. That is, since the types of obstacles that may exist depending on the driving lane situation are changed, it is possible to select the object to be recognized based on the advance information, and the control of the vehicle can be made efficient based on the movement of the vehicle according to the lane.

  Third, in the aspect of intelligent transportation system, when analyzing the road situation using information gathered from individual vehicles, it can be carried out at the level of the lane instead of the road level, and more precise analysis becomes possible.

  Fourth, in terms of inter-vehicle communication, precise location information of the situation can be transmitted in front of the vehicle, and efficient information sharing becomes possible. For example, if an accident occurs in a two-lane vehicle, it is possible to provide different levels of warning to vehicles present in the lane and vehicles in the other lane.

1 is a flow chart of a driving lane estimation process according to a preferred embodiment of the present invention.
FIG. 2 is a view showing a pre-traveling environment photographing image according to the present embodiment.
3 is a diagram illustrating an example of a region of interest for road sign recognition according to the present embodiment.
4 is a view showing a result of extracting a road sign candidate region according to the present embodiment.
FIG. 5 shows an example of the result of verifying the road sign through the road sign candidate region verification according to the present embodiment.
6 shows an example of the result of classifying the left / right position of the road sign.
The blue circles according to the embodiment of FIG. 7 show examples of the results of detecting the road sign bottom end point.
8 is a view showing a structure of a road sign.
9 is a side view of the camera and the road sign.
10 (a) is a bird's-eye view of a road sign and a camera viewed from the sky, and FIG. 10 (b) is a triangle marked by a dotted line in detail on the left side.
FIG. 11 shows a result of performing the car estimation by the proposed method.
12 is a view showing a driving lane estimation apparatus according to a preferred embodiment of the present invention.

While the invention is susceptible to various modifications and alternative forms, specific embodiments thereof are shown by way of example in the drawings and will herein be described in detail.

It should be understood, however, that the invention is not intended to be limited to the particular embodiments, but includes all modifications, equivalents, and alternatives falling within the spirit and scope of the invention. Like reference numerals are used for like elements in describing each drawing.

Hereinafter, embodiments according to the present invention will be described in detail with reference to the accompanying drawings.

The present invention proposes a method and an apparatus for estimating a lane on which a car is traveling based on a road sign frequently existing in a city area.

Specifically, a camera is used to photograph the forward driving environment, and the position of the sign is recognized by detecting the road sign by using the computer vision and pattern recognition technology. Then, the distance from the end of the road to the camera is estimated by calculating the three-dimensional position of the road sign by using the installation information of the sign determined by the traffic regulations. Finally, the lateral position of the estimated car is divided by the standard car width to estimate the lane on which the car is traveling.

Unlike radio navigation, the proposed method can operate regardless of high - rise buildings and overpasses in the presence of road signs, does not have the limitation of cumulative error over time, unlike inertial navigation, It has an advantage that a high-capacity precision map is not required at all. In addition, since the road sign is different from the road surface, it is featured that most of the road sign is photographed on the front camera even in the presence of the vehicle in front.

Due to these advantages, the present invention can be easily applied to a previously installed front lane keeping camera or a black box camera.

1 is a flow chart of a driving lane estimation process according to a preferred embodiment of the present invention.

The process shown in Fig. 1 may be performed in a computing device (driving lane estimation device) connected to a lane-keeping front camera or a black box camera and having a processor and a memory.

Referring to FIG. 1, an apparatus according to the present embodiment acquires an image photographed by a camera (step 100).

FIG. 2 is a view showing a pre-traveling environment photographing image according to the present embodiment.

Then, a road sign candidate is extracted within a predetermined area of interest of the photographed image (step 102).

The area of interest can be set using camera external variables indicating the installation state of the camera and installation height information of the road signs according to the installation regulations.

FIG. 3 is a view showing an example of a region of interest for road sign recognition according to the present embodiment, and an area where a road sign can be located is set as an area of interest 300.

The road sign candidate extraction uses the technique of learning features such as Viola-Jones, HOG-SVM (Histogram of Oriented Gradient-Support Vector Machine), the technique of measuring similarity using templates such as chamfer matching, And the like can be used.

4 is a view showing a result of extracting a road sign candidate region according to the present embodiment.

As shown in FIG. 4, a plurality of road sign candidate regions 400 may be extracted within the ROI.

The road sign candidate region extraction method is designed so as to generate as few false errors as possible while including correct answers.

Next, the road sign candidate region verification process is performed (step 104).

Since the extracted road sign candidate regions include erroneous detection, the road sign candidate region verification process is performed to judge the authenticity of the road sign candidate regions again.

This candidate verification process determines the real road sign area.

The road sign candidate verification process consists of a feature extraction step and a classification step.

In the feature extraction step, features for describing characteristics of the corresponding region are extracted using the brightness and color information of the candidate region. Various feature extraction methods such as HOG and LBP (Local Binary Pattern) can be used in this step.

In the classification step, it is determined whether the candidate area is a road sign. Various classification methods such as SVM and Neural network may be used in this step.

FIG. 5 shows an example of the result of verifying the road sign through the road sign candidate region verification according to the present embodiment.

Next, the road sign left / right position classification process is performed (step 106).

In order to detect the end toward the driveway, it is necessary to recognize whether the recognized road sign is installed on the left side of the road or on the right side.

It is necessary to detect the position of the left lower end point when it is installed on the left side and the position of the lower right end point when it is installed on the right side.

However, the information inside the recognized road sign area does not include information about the installation location of the road sign.

Therefore, the left and right installation positions of the road sign area are recognized including not only the inside of the recognized road sign area but also the outside area. For example, the image of the area where the width and height of the road sign area are increased by 20% is used. The area outside the road sign provides important information for recognizing the left / right installation position, since it includes information such as signs and the like.

At this stage, various feature extraction methods and feature classification methods described in the road sign candidate verification step can be used. 6 shows an example of the result of classifying the left / right position of the road sign.

After the road sign left / right position classification process, the lower end point of the road sign area is detected (step 108).

The end of the road sign towards the driveway signifies the lower endpoint.

In the case of the sign placed on the left side, the lower left end point is detected. On the right side, the lower right end point is detected.

In order to detect the corresponding position, a method of combining straight line detection results by applying a linear estimation method such as a Hough transform or RANSAC to the edge detection result, a method of determining the similarity degree by chamfer matching using an L-shaped template Various approaches can be used.

The blue circles according to the embodiment of FIG. 7 show examples of the results of detecting the road sign bottom end point.

 Most of the road signs installed in the city center are as shown in Fig. 8, and their height is fixed to 500 cm from the road surface by the regulations. In the present invention, such a preliminary information is used to calculate the three-dimensional position of the lower end of the sign.

9 is a side view of the camera and the road sign.

9 (a) is a side view of a camera attached to a road sign and a vehicle. The assumption here is that the camera's optical axis is parallel to the ground. If the optical axis and the ground are not parallel, you can change them in parallel using camera external variables.

9, H _p represents the height of the sign, H _c represents the height of the camera, and Z represents the longitudinal distance from the camera to the sign.

FIG. 9 (b) shows the triangle indicated by the dotted line in FIG. 9 (a) in detail.

In (b) of Figure 9 v refers to the vertical position of the road sign at the bottom end point up to the image and, o _v; means the vertical coordinates of the camera liquor (pricipal point), f denotes a focal length of the pixels.

In FIG. 9 (b), H _p is fixed to 500 cm by definition, and H _c , o _v , and f can be measured in advance through the camera calibration process, and v is a method of detecting the lower end point of the road sign by the above- Able to know. Therefore, the distance Z from the camera to the road sign can be calculated by the following equation.

10 (a) is a bird's-eye view of a road sign and a camera viewed from the sky.

In FIG. 10 (a), X represents the lateral distance from the camera to the lower end of the road sign.

FIG. 10 (b) shows a triangle indicated by a dotted line in detail on the left side. In (b) of Figure 10 u denotes the horizontal coordinate of the bottom end road sign photographed in the image, and _u o refers to the abscissa of the camera pub. Therefore, the lateral distance from the end of the road sign to the car attached to the camera can be calculated by the following equation.

Through this process, it is possible to estimate the three-dimensional position of the lower end point of the road sign.

After estimating the three-dimensional position of the lower end point, the driving lane estimation process is performed (step 110)

The road sign end point is compulsorily set up within 10 to 20 cm from the end of the road as shown in Fig.

Therefore, the position of the road sign end point can be approximated to the end position of the roadway. Based on this, it is possible to estimate the position of the lane on which the car is traveling by dividing the lateral distance from the camera to the end of the road sign by the standard lane width.

For example, when the camera is installed in the center of the vehicle, the lateral distance from the camera to the end of the road sign is measured as 7m. If the width of the standard car is 3m, the position of the vehicle is 3/3, which is a raise of 7/3.

FIG. 11 shows a result of performing the car estimation by the proposed method. In the example above, the distance from the camera to the end of the road is estimated to be 10.2m. Based on this, the driving lane has been recognized from the left four lanes. In the example below, the distance from the camera to the end of the car is estimated to be 4.1m. Based on this, it is recognized that the current lane is two lanes away from the right lane.

12 is a view showing a driving lane estimation apparatus according to a preferred embodiment of the present invention.

As shown in FIG. 12, the apparatus according to the present embodiment may include a processor 1200 and a memory 1202.

Here, the processor 1200 may include a central processing unit (CPU) or other virtual machine capable of executing a computer program.

Memory 1202 may include a non-volatile storage device such as a fixed hard drive or a removable storage device. The removable storage device may include a compact flash unit, a USB memory stick, and the like. Memory 1202 may also include volatile memory, such as various random access memories.

Such memory 1202 stores program instructions that are executable by the processor 200.

According to a preferred embodiment of the present invention, the memory 1202 extracts a road sign candidate within a predetermined area of interest of the image acquired from the camera, determines a road sign area through verification of the extracted candidate, Determining whether the road sign area is in the left or right direction, calculating the three-dimensional position of the lower end point with respect to the determined direction, approximating the calculated three-dimensional position of the lower end point to the end position of the roadway, The instructions of the program executed by the processor 1200 are stored so as to divide the lateral distance to the end point by the standard lane width to estimate the position to the lane in progress.

The present invention can overcome the limitations of existing methods using GPS and IMU, and has the advantage of a practical aspect that precision maps are not used for estimating lanes. It can also be used in combination with existing methods of using GPS, IMU, and precision maps.

As described above, the present invention has been described with reference to particular embodiments, such as specific elements, and specific embodiments and drawings. However, it should be understood that the present invention is not limited to the above- And various modifications and changes may be made thereto by those skilled in the art to which the present invention pertains. Accordingly, the spirit of the present invention should not be construed as being limited to the embodiments described, and all of the equivalents or equivalents of the claims, as well as the following claims, belong to the scope of the present invention .

Claims (11)

Extracting a road sign candidate region within a predetermined area of interest of the image acquired from the camera;
Determining an actual road sign area through verification of the extracted road sign candidate area;
Determining whether the determined actual road sign area is in the left or right direction;
Calculating a three-dimensional position of the lower end point of the actual road sign area with respect to the determined direction; And
Estimating a position of the lane in progress using the calculated three-dimensional position of the lower end point. The method according to claim 1,
Wherein the step of estimating the position of the lane comprises:
Approximating the calculated three-dimensional position of the lower end point to the end position of the roadway;
And dividing the lateral distance from the camera to the lower end point by a standard lane width. The method according to claim 1,
Wherein the step of determining the actual road sign area comprises:
Extracting a feature describing characteristics of the road sign candidate region using brightness and color information of the road sign candidate region; And
And determining whether the road sign candidate region is the actual road sign region using the extracted feature. The method according to claim 1,
The lower end point,
If the actual road sign area is the right direction, it is the lower right end point,
And if the actual road sign area is the left direction, the left and lower end points. The method according to claim 1,
Calculating the three-dimensional position of the lower endpoint comprises:
The height of the road sign from the ground, the height from the ground of the camera, the focal distance in pixels, the ordinate of the principal point of the camera, and the ordinate of the lower end point of the actual road sign area included in the image Calculating a distance to the actual road sign area using at least one of the camera and the actual road sign area; And
Using at least one of a distance between the camera and the actual road sign area, an abscissa of a lower end point of the actual road sign area included in the image, an abscissa of the camera principal point, And calculating a lateral distance from a lower end point of the road sign area to a car on which the camera is attached. The method according to claim 1,
Wherein the area of interest is set using camera external variables indicating the installation state of the camera and installation height information of the road signs according to installation rules. Extracting a road sign area from the image acquired from the camera;
Determining whether the extracted road sign area is in the left or right direction;
Calculating a three-dimensional position of the lower end point of the road sign area using the determined direction, a distance between the camera and the road sign, and a road sign installation rule; And
Estimating a position of the lane in progress using the calculated three-dimensional position of the lower end point. A computer-readable recording medium on which a program for performing the method according to any one of claims 1 to 7 is recorded. As a driving lane estimation device,
A processor; And
A memory coupled to the processor,
The memory comprising:
Extracts a road sign candidate region within a predetermined region of interest of the image acquired from the camera,
A road sign area is determined through verification of the extracted road sign candidate area,
Determining whether the determined road sign area is in the left or right direction,
Calculating a three-dimensional position of the lower end point with respect to the determined direction,
And stores the executable program instructions by the processor to estimate the position of the lane in progress using the calculated three-dimensional position of the lower end point. 10. The method of claim 9,
The memory comprising:
In order to estimate the position of the lane,
Dimensional position of the calculated lower end point is approximated to the end position of the roadway,
And stores program instructions executable by the processor to divide the lateral distance to the camera and the bottom end point by a standard lane width. 10. The method of claim 9,
The memory comprising:
For the three-dimensional position calculation of the lower end point,
The height of the road sign from the ground, the height from the ground of the camera, the focal distance in pixels, the ordinate of the principal point of the camera, and the ordinate of the lower end point of the actual road sign area included in the image Calculates a distance to the actual road sign area using the camera,
Using at least one of a distance between the camera and the actual road sign area, an abscissa of a lower end point of the actual road sign area included in the image, an abscissa of the camera principal point, And stores program instructions executable by the processor to calculate a lateral distance from a lower end point of the road sign area to the car-mounted vehicle.

Images