KR102158169B1 - Lane detection apparatus - Google Patents

Abstract

The present invention relates to a road lane recognition device. The road lane recognition apparatus of the present invention recognizes the lane and determines the lane on which the vehicle is traveling even under conditions in which the lane is not visible or snow/dirt is accumulated on the road and the lane itself is not recognized. In addition, the present invention accurately extracts/updates the current location information of the vehicle by using both GPS information and information on a road sign.

Description

Lane Recognition Device {LANE DETECTION APPARATUS}

The present invention relates to a lane recognition device. More specifically, it relates to a lane recognition device that virtually recognizes a lane through a change in size of front image information photographed by at least one camera mounted in front of the vehicle so that the vehicle can drive normally even in bad weather and snowy roads.

Currently, most lane recognition methods use a camera to capture a road surface (road surface), extract image data, and recognize the relative position of a vehicle with respect to the lane. In addition, such lane information may be used to warn a driver of speeding and lane departure, or to control a vehicle or maintain a lane through longitudinal and lateral control.

However, since the camera photographs the road surface, it was practically impossible to recognize the lane under conditions in which a part of the lane was erased or the lane could not be photographed due to snow accumulation on the road surface. In this case, the driver did not know how many lanes his vehicle is currently driving, and this caused an accident with an adjacent vehicle.

In particular, in recent years, intelligent vehicles based on autonomous driving systems (ie, unmanned vehicles, autonomous vehicles, etc.) in which driving operations such as driving, stopping, turning, acceleration or deceleration are automatically performed by the system without directly manipulating the vehicle even if there is a driver. ) Is being actively developed, and driving practice has already been conducted on the actual road. The main tasks of such an intelligent vehicle include maintaining a driving lane, securing a safe distance from adjacent vehicles, detecting nearby obstacles, avoiding collisions, and controlling vehicle speed according to traffic conditions or road conditions. Among them, detection of driving lanes is one of the core technologies to solve major problems in intelligent vehicles, and many studies are actively being conducted.

Since detection of such a driving lane has a great influence on safe driving, accurate driving lanes are detected using various sensors to estimate and determine the position of the lane. For example, various sensors, such as an image sensor, a radar (RADAR), or a lidar (LIDAR) sensor, are used to implement an intelligent vehicle control system in a single or combined form for detection of a lane or object recognition in front of a vehicle. Particularly, an image-based system using an image sensor has been widely used due to the advantage of being able to extract a lot of information at low cost and to utilize various existing image processing algorithms. However, these methods also have not fundamentally solved the problem of accurately detecting a lane in a harsh driving environment in which lanes are not recognized as described above.

And it was difficult to accurately detect the current position of the vehicle in an environment in which the lane was not recognized. Of course, the vehicle location can be known through GPS, but since the error range is large, this could not be applied in the case of an unmanned vehicle that requires precise vehicle location during driving.

Korean Patent Publication No. 10-2013-0011825 (2013. 01. 30. Lane Recognition Method)

Accordingly, an object of the present invention is to provide a lane recognition device capable of accurately recognizing a driving lane of a vehicle even under a condition in which the lane on the road surface is not recognized.

In addition, the present invention provides a vehicle recognition device that accurately detects and updates a current position of a vehicle by using GPS information and road information of a road sign.

According to a feature of the present invention for achieving the above object, at least one camera installed in front of the vehicle; A map DB storing left/right images of the actual road based on the driving direction of the vehicle; And when the vehicle is driving, the lane is compared with the left/right image of the actual road provided by the map DB, and the lane is configured including a control unit that recognizes the lane and determines the driving lane of the vehicle according to the size change of the left/right image captured by the camera. It is to provide a recognition device.

Here, the lane recognition is virtually processed by comparing the left/right image of the actual road and the left/right image captured by the camera.

In addition, if the left image is the largest among the images photographed by the camera, the control unit determines that the vehicle travels on the first lane, and if the left image is the smallest, the vehicle travels on the outermost lane.

In addition, according to an embodiment of the present invention, a GPS receiver that provides current location information of the vehicle is further included, and the control unit is configured to allow the map DB to drive the vehicle from the map server according to the current location information of the vehicle provided by the GPS receiver. It controls to receive the real-time left/right image of the road that matches the direction.

In addition, an exemplary embodiment of the present invention further includes a camera for photographing a road sign, and an extraction unit for extracting text and image information from the road sign information photographed by the camera, wherein the controller includes location information provided by the GPS receiver and the The current location information of the vehicle is updated by using the location information obtained by the extraction unit.

According to the lane recognition apparatus of the present invention as described above, a camera installed in the vehicle photographs the left/right image of the driving direction of the vehicle, and the size change of the left/right image is checked while comparing it with reference information provided in advance. Are doing. In addition, according to the change in the size of the left/right image, the vehicle can virtually recognize the lane of the current road and determine which lane the vehicle is traveling along.

Therefore, the present invention is to provide an environment in which a vehicle can safely travel even in a condition in which the lane of the road is not recognized. In particular, since the driverless vehicle runs while maintaining the driving lane, there is an effect of providing conditions for the driverless vehicle to stably travel even in a place where the lane is not recognized.

In addition, since the present invention detects/updates vehicle location information by using both GPS information and road sign information, there is an effect of accurately providing reference information necessary for lane recognition.

1 is a block diagram showing a lane recognition apparatus according to a preferred embodiment of the present invention
2 is a flowchart showing a lane recognition method using the lane recognition device of the present invention
3 is an exemplary diagram of a left/right image of a road taken by a camera provided in the lane recognition apparatus of the present invention
4 is a flowchart showing a process of updating the current location information of the vehicle by the lane recognition apparatus of the present invention

The present invention virtually acquires driving lane information by using a change in the size of left/right image information captured by a camera mounted on a vehicle, through which it is possible to recognize in which lane the vehicle is currently driving. It is a technical feature that updates the current location information of the vehicle using GPS information and road sign information.

A preferred embodiment of the present invention will be described in detail with reference to the accompanying drawings. 1 is a block diagram showing a lane recognition apparatus according to an exemplary embodiment of the present invention.

As shown, the lane recognition apparatus 100 includes cameras 102 and 104. The camera is preferably installed on the left/right side of the front of the vehicle so as to photograph the front of the vehicle (ie, the road surface and the left/right front direction). In the embodiment, it will be described that the first camera (or the left camera) 102 and the second camera (or the right camera) 104 are mounted. Of course, this is only an exemplary embodiment, and two or more cameras may be mounted on the left and right sides, or only one camera may be mounted at a predetermined position in front of the vehicle. In other words, the number of cameras is irrelevant as long as the condition for confirming the size change of the left/right image in the driving direction of the vehicle based on the image information captured by the camera is satisfied. Of course, if there is only one camera, an image dividing device (not shown) that divides the left/right image from the image captured by the camera should be further provided.

A map DB 110 is provided in the lane recognition device 100. The map DB 110 stores image information about buildings, mountains, and landscapes such as rivers located around the road, and information about the lane of the road on which the vehicle is running. In this embodiment, data stored in the map DB 110 Will be described as reference information. This reference information is processed and stored in any one of 1D, 2D and 3D information. Preferably, it will be stored as three-dimensional information that can know the size and position of the image. The reason why reference information is necessary is because the left/right images of the road are different from each other.

The reference information stored in the map DB 110 is not only stored as 3D information, but also all data on the driving direction of the vehicle must be stored. This is because there must be actual image information capable of comparing information captured by the first camera 102 and the second camera 104 when the vehicle is driving. For this reason, the map DB 110 is bound to have a relatively large capacity. Of course, as another example, the present invention may provide a method of minimizing the capacity of the map DB 110. That is, a GPS receiver 130 for receiving vehicle location information and a processing unit 140 for receiving map information on the vehicle's traveling direction from the map server 10 and sequentially storing it in the map DB 110 are required. . In this case, the map DB 110 can receive and provide map information in a first-in-first-out (FIFO) method even if the capacity is reduced, so that map information can be efficiently provided to the lane recognition method. That is, the map in the front of the driving direction of the vehicle is received and transmitted, and the map in the rear of the driving direction is deleted.

A comparator 150 for comparing image information captured by the first camera 102 and the second camera 104 with the reference information is configured. The comparator 150 includes left/right image information (ie, reference information) in front of the vehicle stored in the map DB 110 and left/right image information captured by the cameras 102 and 104 according to the vehicle driving. In other words, it serves to compare the target information).

According to the comparison result of the comparator 150, the controller 160 is configured to detect a change in the size of the target information compared to the reference information, and to control a series of processes of virtually recognizing a lane on which the vehicle is traveling. The control unit 160 may add functions to a microprocessor or electronic control unit that controls operations related to the operation of the vehicle, or may be separately configured as a device for lane recognition.

The control unit 160 also selectively controls on/off driving of the processing unit 140 and the comparator 150 according to road conditions. That is, it is because the reference information need not be used under conditions in which the first camera 102 and the second camera 104 can recognize the lane. The control unit 160 only needs to recognize the lane information captured by the first camera 102 and the second camera 104.

In addition, the controller 160 may update vehicle location information using information obtained using GPS information and information on a road sign. This is because reference information on the current location of the vehicle must be accessed from the map DB 110 when driving the vehicle. To this end, according to an embodiment of the present invention, a camera 170 for photographing a road sign mounted around a road may be further included. Of course, if the first camera 102 or the second camera 104 is a condition capable of sufficiently photographing a road sign, such a camera 170 may not be necessary. However, in the present embodiment, for convenience of explanation, it is assumed that a third camera 170 that photographs only road signs is provided. The third camera 170 may be mounted on the front part of the vehicle or inside the windshield, and its position may be changed as much as possible.

In addition, an extraction unit 180 for extracting text and image information from image-processed sign information captured by the third camera 170 in relation to the road sign is configured. The extraction unit 180 can be extracted through color contrast. This is because, for example, a road sign has text or images of different colors printed on a series of background colors. Examples of text and images include place names, images representing intersections and rest areas, and distance information (Km).

Next, a method for recognizing a lane according to the present invention will be described. In an embodiment of the present invention, as shown in FIG. 2, the vehicle starts from traveling on a predetermined road.

When the vehicle is running, the controller 160 determines whether the first camera 102 and the second camera 104 can recognize the lane (S100). As a result of the determination, if lane recognition is possible, only driving the first camera 102 and the second camera 104 drives along the driving lane (S110). This applies to unmanned vehicles as well as general vehicles. In particular, since normal lane recognition is possible in an unmanned vehicle, a more stable driving will be possible while maintaining the lane.

On the other hand, there may be a case in which the first camera 102 and the second camera 104 cannot normally perform lane recognition. As described above, this corresponds to a case where the lane is not recognized because the lane is erased due to a long maintenance period, or the lane itself is not recognized because snow or dirt is accumulated on the road. If the lane is not recognized as such, the controller 160 will perform the lane recognition method provided by the embodiment of the present invention.

That is, when the vehicle is driven, the first camera 102 and the second camera 104 photograph the front of the vehicle. Preferably, the surroundings of the left/right road will be photographed based on the vehicle, and the object to be photographed includes a building or the like (see FIG. 3). Then, the information captured by the first and second cameras 101 and 104 is transmitted to the comparator 150 in real time (S120).

In addition, the control unit 160 controls the map DB 110 to access reference information on the driving direction of the vehicle based on the current location information of the vehicle and provide it to the comparator 150 (S130). This process is performed at the same time point as when the photographing information of the first camera 102 and the second camera 104 is transmitted to the comparator 150. At this time, while the reference information is stored in the map DB 110, the reference information for the current location of the vehicle may be accessed, but only reference information corresponding to the driving direction of the vehicle is transmitted from the map server 10 in real time to be provided. It may be as described above. This is possible because the current location and driving direction information of the vehicle can be known using GPS information.

The comparator 150 compares the reference information with object information photographed by the first camera 102 and the second camera 104 (S140). In addition, an example of the comparison result may be a change in the size of the left image and the right image, which are target information compared to the reference information. That is, the size of the left image and the right image of the reference information and the left image and the right image of the captured target information may be the same or different.

A comparative example of the comparator 150 is as follows. When the vehicle travels along the first lane on a two-lane one-way road (ie, four-lane round trip), the left image becomes larger and the right image becomes relatively smaller. Conversely, when driving on the second lane, the right image is larger than the left image. This means that the more lanes there are, the larger the image size change will appear. This exemplary diagram is shown in FIG. 3. 3, it can be seen that the right image B is displayed larger than the left image A. As another example, ratio information of the photographed left/right image relative to the reference information may be used. For example, if the image of the left/right reference information is 100%, it is stepped according to the number of lanes (here, three lanes). If it is less than %, it can be set in advance as driving on 3 lanes.

The comparison result of the comparator 150 is transmitted to the controller 160. In addition, the control unit 160 may detect a change in the size of the captured left/right image compared to the reference information, and determine which lane the vehicle travels while virtually recognizing the lane (S150). For example, as shown in FIG. 3, when the left image A is smaller and the right image B is larger compared to the reference information, it is recognized that the vehicle is driving along a lane outside the second lane, not at least the first lane. On the contrary, when the left image (A) becomes larger and the right image (B) becomes smaller, it is determined as a case of driving along the inner lane.

As described above, even if the lane on the road surface is not recognized, the present invention can determine whether the vehicle travels in several lanes while virtually recognizing the lane of the road by using the left/right image information of the road.

Meanwhile, in the present invention, it is important to accurately detect the current location information of the vehicle. That is, as described above, the present invention uses map information of the current location of the vehicle. Of course, according to an embodiment of the present invention, it is possible to check the current position of the vehicle by receiving GPS information using the GPS receiver 130. However, as is known, the GPS information has some errors. Therefore, it is because a case in which reference information necessary for lane recognition cannot be accurately provided, and in this case, the actual lane is not recognized. In particular, since the present invention is actively applied to an unmanned vehicle running while recognizing a lane and maintaining a driving lane, it is important to check and update information on the current location of the vehicle.

To this end, the present invention intends to use GPS information and road sign information.

Therefore, as shown in FIG. 4, when the location information needs to be updated while the vehicle is driving (S200), the control unit 160 first checks the current location of the vehicle through the location information received by the GPS receiver 130 (S120). ). Further, the controller 160 additionally acquires the location information of the vehicle by using the information of the road sign photographed by the third camera 170 and extracted by the extraction unit 180 (S220). This will give you a more accurate idea of the vehicle's current location.

The control unit 160 updates the vehicle location information obtained by using the GPS information and the road sign information in real time (S230). It is natural that such update information is much more accurate than that of updating using only GPS. In addition, the update of the vehicle location information is not performed at random, but only when a road sign is photographed and related information is extracted.

When the information obtained through such a process is used, the control unit 160 can access and provide sophisticated reference information from the map DB 110 (S240), and thus, the lane recognition method can be efficiently processed.

As described above, it can be seen that the present invention is configured to recognize a lane through a change in the size of left and right image information photographed by a camera even in an environment in which the lane cannot be recognized, and to accurately detect a vehicle position in cooperation with road sign information.

Although described with reference to the illustrated embodiments of the present invention as described above, these are only exemplary, and those of ordinary skill in the art to which the present invention belongs can use various types without departing from the gist and scope of the present invention. It will be apparent that variations, modifications and other equivalent embodiments are possible. Therefore, the true technical protection scope of the present invention should be determined by the technical spirit of the appended claims.

100: lane recognition device
102: first camera
104: second camera
110: Map DB
130: GPS receiver
140: processing unit
150: comparator
160: control unit
170: third camera
180: extraction unit

Claims (5)

At least one camera installed in front of the vehicle;
A map DB storing left/right images of the actual road based on the driving direction of the vehicle; And
Including; a control unit for recognizing a lane and determining a driving lane of the vehicle according to a change in size of the left/right image captured by the camera by comparing the left/right image of the actual road provided by the map DB when the vehicle is driving, and

A comparator for comparing the left/right image of the actual road provided by the map DB with the left/right image captured by the camera,
A processing unit that receives a map in front of the driving direction of the vehicle in real time and transmits it to the map DB, and deletes the map of the driving direction rear from the map DB;
A third camera that photographs road signs,
An extraction unit for extracting information from the image of the road sign,
Further comprising a GPS receiver providing current location information of the vehicle,

The data stored in the map DB is stored as 3D information including image size and location information,
The control unit stops driving of the comparator and processing unit if it is possible to recognize a lane in the image captured by the camera, and determines a driving lane of the vehicle through the recognized lane,
The control unit drives the comparator and processing unit when it is impossible to recognize the lane in the image captured by the camera, deletes the map in the rear of the driving direction from the map DB, and receives a map in the front of the driving direction of the vehicle in real time to the map DB. Thus, by comparing the left/right image of the road taken from the camera with the left/right image of the actual road stored in the map DB, the driving lane of the vehicle is determined through the virtually processed lane,
The virtually processed lane is determined according to the ratio of the size change of the left object and the right object of the image photographed by the camera,
The control unit updates the current location information of the vehicle using the information extracted from the road sign and the GPS information transmitted from the GPS receiver,
The lane recognition device, characterized in that the road sign is recognized through color contrast by the extraction unit. delete The method of claim 1,
The control unit determines that the vehicle travels on the first lane when the left image is the largest among the images photographed by the camera, and determines that the vehicle travels on the outermost lane when the left image is the smallest. The method of claim 1,
The control unit controls the map DB to receive real-time left/right images of the road corresponding to the driving direction of the vehicle from the map server according to the current location information of the vehicle provided by the GPS receiver. delete

Images