KR102032501B1 - Method and system for lane departure warning - Google Patents

Abstract

The present invention relates to a method for detecting lane departure, which comprises the following steps of: acquiring a plurality of images corresponding to each vehicle direction through a plurality of cameras installed to be distributed in the left, the right, the front, and the rear of a vehicle; checking whether there is lane departure in each vehicle direction based on each of the images; calculating a lane departure probability based on whether there is lane departure in each vehicle direction; and generating and outputting warning information to give a warning of lane departure when the lane departure probability is greater than or equal to a predetermined value.

Description

Lane Departure Detection Method and Apparatus {METHOD AND SYSTEM FOR LANE DEPARTURE WARNING}

The present invention relates to a lane departure detecting method and apparatus, and more particularly, to a lane departure detecting method and apparatus for stably performing a lane departure detecting operation even when at least one of a plurality of cameras malfunctions.

Recently, due to the development of the automobile industry, the spread of automobiles has been commercialized to the extent that it is called the one-family-one automobile era. As a result, truck drivers and long-distance commuters who have a lot of long-haul transportation are exposed to the risk of lane departure accidents due to fatigue, inattention, and drowsy driving.

In general, Advanced Safety Vehicles (hereinafter referred to as "ASVs") are advanced vehicles that apply advanced electronic and control technology to improve the safety of the vehicle.It reduces traffic accidents to increase traffic and save energy. And the driver's convenience.

ASV is called cruise control system (Adaptive Cruise Control, "ACC") that automatically controls the distance between lanes, and Lane Departure Warning System (LDWS), which monitors and warns of lane departure. ), And the like.

In particular, LDWS is a safety system that detects the image of the road ahead from the camera attached to the vehicle, grasps the currently running lane, and if the driver attempts to leave the lane due to carelessness or drowsy driving, an alarm sounds.

LDWS analyzes the lateral position, side speed, steering angle, lane width, and road curvature based on the images captured by the camera and informs the driver when lane departure is detected.

For example, as in Korean Patent Publication No. 10-2018-0015018, the conventional lane departure technology recognizes both lanes, and then determines the lane departure by determining and analyzing the relative position of the vehicle with respect to both lanes. do.

However, in the prior art, when any one of the vehicle cameras malfunctions, a lane departure detection operation cannot be performed normally. That is, when at least one camera is in a faulty state or when a light saturation phenomenon occurs in the camera image in a sudden change in illuminance such as a tunnel exit, the LDWS cannot recognize both lanes, resulting in a determination error. .

Domestic Publication No. 10-2018-0015018 (2018.02.12)

In order to solve the above problems, the present invention is to provide a lane departure detection method and apparatus for stably performing a lane departure detection operation even if at least one of a plurality of cameras malfunction.

The object of the present invention is not limited to the above-mentioned object, and other objects which are not mentioned will be clearly understood by those skilled in the art from the following description.

As a means for solving the above problems, according to an embodiment of the present invention, the method comprising: obtaining a plurality of images corresponding to each vehicle direction through a plurality of cameras distributed in a left / right / before / after a vehicle; Checking lane departure by vehicle direction based on each of the images; Calculating a lane departure probability based on the lane departure by vehicle direction; And generating and outputting warning information to warn the lane departure only when the lane departure probability is greater than or equal to a predetermined value.

The determining of whether the lanes deviate by vehicle direction may include: correcting camera distortion after acquiring each lane detection image from each of the images; And checking whether the lanes deviate from vehicle lanes based on the lane positions in the lane detection image.

The determining of whether the lanes deviate by vehicle direction may include: correcting camera distortion of each image, and extracting and storing each lane detection image; And checking whether the lanes deviate from vehicle lanes based on the lane positions in the lane detection image.

The determining of lane departure by vehicle direction may include determining a lane departure only when the distance is less than a predetermined value after calculating a distance from a vehicle or an edge of an image to a lane when one lane is detected. It features.

In addition, the determining of lane departure by vehicle direction may include determining first and second distances from both sides of the vehicle or the edge of the image when two lanes are detected, and then detecting deviation of the first and second distances. It is characterized in that the lane departure is determined only when it is above the set value.

The calculating of the lane departure probability may include calculating the lane departure probability by counting the number of vehicle directions in which the lane departure state is confirmed, and dividing a counting result by the total number of vehicle directions.

The present invention performs a lane departure detection operation in two stages of first checking whether lane departure is performed by vehicle direction, and then repeatedly checking these check results based on probability once again, so that at least one of the plurality of cameras malfunctions even if the lane departure occurs. This enables stable detection operation.

1 to 2 are diagrams for explaining a lane departure detection method according to an embodiment of the present invention.
3 to 5 are for explaining a lane departure confirmation step according to an embodiment of the present invention.
6 is for explaining a lane departure confirmation step according to another embodiment of the present invention.
7 is a diagram illustrating an example computing environment in which one or more embodiments disclosed herein may be implemented.

The objects and effects of the present invention and the technical configurations for achieving them will be apparent with reference to the embodiments described below in detail with the accompanying drawings. In describing the present invention, when it is determined that a detailed description of a known function or configuration may unnecessarily obscure the subject matter of the present invention, the detailed description thereof will be omitted.

Terms to be described later are terms defined in consideration of functions in the present invention, and may be changed according to intentions or customs of users or operators.

However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various forms. The present embodiments are merely provided to complete the disclosure of the present invention and to fully inform the scope of the invention to those skilled in the art, and the present invention is defined by the scope of the claims. It will be. Therefore, the definition should be made based on the contents throughout the specification.

1 to 2 are diagrams for explaining a lane departure detection method according to an embodiment of the present invention.

First, as shown in Figure 1, the method of the present invention, the image acquisition step (S1) for each vehicle direction, lane departure confirmation step (S2) for each vehicle direction, lane departure probability calculation step (S3), and lane departure warning Step S4 and the like.

First, in the image acquisition step for each vehicle direction (S1), a plurality of images corresponding to each of the left, right, front, and rear directions of the vehicle are acquired through a plurality of cameras respectively disposed at the left, right, front, and rear of the vehicle. .

In the step of checking lane departure according to vehicle directions (S2), lanes are detected from each image of each vehicle direction, and then lane departure in each direction is checked based on lane position values.

)을 산출하도록 한다. 예를 들어, 차선 이탈 상태가 4개 차량방향 중 1개 차량방향에서 확인된 경우에는 25%, 4개 차량방향 중 2개 차량방향에서 확인된 경우에는 50%, 4개 차량방향 중 3개 차량방향에서 확인된 경우에는 75%, 4개 차량방향 중 4개 차량방향에서 확인된 경우에는 100%의 값을 가지는 차선 이탈 확률이 산출될 수 있을 것이다. The lane departure probability calculating step (S3) counts the number m of vehicle directions in which the lane departure state is confirmed, and divides it by the total number of vehicle directions n to determine the lane departure probability (

). For example, 25% if the lane departure condition is found in one of four vehicle directions, 50% if it is found in two of the four vehicle directions, and three of the four vehicle directions. The lane departure probability having a value of 75% when it is identified in the direction and 100% when it is confirmed in four vehicle directions among four vehicle directions may be calculated.

In the lane departure warning step (S4), only when the lane departure probability is greater than a preset value, warning information for warning the driver that the vehicle is currently in a lane departure state is generated and output visually. In this case, the preset value is a value for determining the sensitivity of the warning operation, and may be a value which may be preset or arbitrarily changed by the driver. For example, if a lane departure is found in two or three vehicle directions among four vehicle directions by setting the preset value to 50%, 75%, or the like, the driver will confirm that the lane departure has actually occurred and promptly inform the driver. Let them be notified.

As described above, the present invention first checks lane departure by vehicle direction, and then checks these confirmation results once again based on probability, thereby finally confirming and warning lane departure.

As a result, even if some of the cameras used for lane departure detection malfunctions, the lane departure detection operation can be stably performed. That is, as shown in FIG. 2, even if a part of the camera does not provide the image itself or a light saturation phenomenon occurs in the part of the image, it is possible to finally confirm and notify whether the lane deviates.

In addition, a new camera may be separately installed for the lane departure detecting method of the present invention, but if necessary, the camera may be recycled to obtain an image for lane departure detection.

However, if the cameras already installed in the vehicle are special cameras such as fisheye cameras and wide-angle cameras, these cameras acquire and provide a distortion image for securing a wider angle of view, and the camera is calibrated to correct the distortion through the process of calibration. There is a feature of pre-acquiring and storing parameters.

Accordingly, in the lane departure confirmation step S2 of the present invention, only when the lane detection image is a distortion image, the distortion correction operation may be additionally performed according to a camera distortion correction parameter previously acquired and stored corresponding to the camera.

3 to 5 are for explaining a lane departure confirmation step according to an embodiment of the present invention, hereinafter will be described only when four cameras are installed on the left, right, front, rear of the vehicle for ease of explanation. Let's do it.

First, the left, right, front and rear images are acquired through four cameras installed on the left, right, front and rear of the vehicle, and the lanes are detected from each of them (S11). In this case, the lane detection operation of step S11 may be performed based on color or shape. That is, considering that the color and shape of the lanes are clearly different from the surrounding environment such as roads, the image characteristics corresponding to the color and shape of the lanes are acquired and stored in advance for each vehicle direction, and based on the input image of each vehicle direction To detect a lane having a corresponding video feature.

After extracting only the lane detection result of step S11 to generate a lane detection image (S12), all camera distortions included in the lane detection image are removed according to the camera distortion correction parameter (S13).

Then, the lane position is identified from the distortion-corrected lane detection image, and the lane departure is determined based on this (S14).

More specifically, when one lane is detected as in the left or right image, after calculating the distance d between the lane and the vehicle as shown in FIG. 4A, the detection distance d is a preset value. Only if the following conditions exist, the lane departure should be confirmed and notified.

When both lanes are detected as in the front or rear image, the distances d1 and d2 between the vehicle center and both lanes are calculated as shown in FIG. Only when d2 |) is above the preset value, the lane departure can be confirmed and notified.

In addition, since the vehicle camera is fixed in position, the vehicle detection position in the image is also fixed in position. As shown in FIG. 5, the distance from the lane is determined and analyzed based on the edge of the image, not the vehicle, so that the lane departure may be checked and notified. In this case, it is not necessary to perform the vehicle detection operation separately, so that the image processing load can be naturally reduced.

That is, when one lane is detected, as shown in Fig. 5A, after calculating the distance d from the edge of the image to the lane, the lane departure is confirmed and notified only when the detected distance is smaller than the preset value. Do it. When both lanes are detected, the distance d1 from the left edge of the image to the left lane and the distance d2 from the right edge of the image to the right lane are calculated as shown in FIG. Only if the deviation (| d1-d2 |) is greater than or equal to the preset value, the lane departure should be confirmed and notified.

In this way, the vehicle or the lane according to the direction of the image using the minimum information of the distance from the lane to determine whether the lane departure by vehicle direction, but by double-checking these check results based on probability once again, whether the lane departure Make it more stable and reliable.

In addition, in the above description, the distortion correction is performed by using the lane detection image instead of the original image for the image processing load according to the distortion correction. Of course it can also be performed.

6 is for explaining a lane departure confirmation step according to another embodiment of the present invention.

Referring to this, when the left, right, front and rear images are acquired through four cameras installed on the left, right, front and rear of the vehicle, the present invention corrects the camera distortion of each image according to a preset camera distortion correction parameter. (S21). After detecting lanes from each of the distortion-corrected images based on color or shape (S22), it is determined whether the lanes deviate from each of the vehicle directions based on the lane position values (S23).

FIG. 7 illustrates an example computing environment in which one or more embodiments disclosed herein may be implemented, and illustrates an example of a system 1000 that includes a computing device 1100 configured to implement one or more embodiments described above. Shows. For example, the computing device 1100 may be a personal computer, server computer, handheld or laptop device, mobile device (mobile phone, PDA, media player, etc.), multiprocessor system, consumer electronics, mini computer, mainframe computer, Distributed computing environments, including, but not limited to, any of the systems or devices described above.

The computing device 1100 may include at least one processing unit 1110 and a memory 1120. Here, the processing unit 1110 may include, for example, a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor, an application specific integrated circuit (ASIC), field programmable gate arrays (FPGA), and the like. It may have a plurality of cores. The memory 1120 may be volatile memory (eg, RAM, etc.), nonvolatile memory (eg, ROM, flash memory, etc.), or a combination thereof.

In addition, computing device 1100 may include additional storage 1130. Storage 1130 includes, but is not limited to, magnetic storage, optical storage, and the like. Storage 1130 may store computer readable instructions for implementing one or more embodiments disclosed herein, and other computer readable instructions for implementing operating systems, application programs, and the like. Computer readable instructions stored in storage 1130 may be loaded into memory 1120 for execution by processing unit 1110.

In addition, computing device 1100 may include input device (s) 1140 and output device (s) 1150. Here, the input device (s) 1140 may include, for example, a keyboard, mouse, pen, voice input device, touch input device, infrared camera, video input device, or any other input device. Also, output device (s) 1150 may include, for example, one or more displays, speakers, printers, or any other output device. In addition, computing device 1100 may use an input device or output device included in another computing device as input device (s) 1140 or output device (s) 1150.

In addition, computing device 1100 may include communication connection (s) 1160 that enable computing device 1100 to communicate with another device (eg, computing device 1300). Here, the communication connection (s) 1160 may be a modem, a network interface card (NIC), an integrated network interface, a radio frequency transmitter / receiver, an infrared port, a USB connection, or other for connecting the computing device 1100 to another computing device. It may include an interface. In addition, communication connection (s) 1160 may include a wired connection or a wireless connection.

Each component of the computing device 1100 described above may be connected by various interconnections such as a bus (eg, peripheral component interconnect (PCI), USB, firmware (IEEE 1394), optical bus structure, etc.). And interconnected by the network 1200.

As used herein, terms such as "component", "module", "system", "interface", etc. generally refer to a computer-related entity that is hardware, a combination of hardware and software, software, or running software. For example, a component may be, but is not limited to being, a process running on a processor, a processor, an object, an executable, a thread of execution, a program, and / or a computer. For example, both an application running on a controller and the controller can be a component. One or more components may reside within a thread of processes and / or execution, and the components may be localized on one computer and distributed between two or more computers.

The above description is merely illustrative of the technical idea of the present invention, and those skilled in the art to which the present invention pertains may make various modifications and variations without departing from the essential characteristics of the present invention. Therefore, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention but to describe the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments. The protection scope of the present invention should be interpreted by the following claims, and all technical ideas within the equivalent scope should be interpreted as being included in the scope of the present invention.

Claims (6)

Acquiring a plurality of images corresponding to each vehicle direction through a plurality of cameras distributed in the left / right / before / after of the vehicle;
Checking lane departure by vehicle direction based on each of the images;
Calculating a lane departure probability based on the lane departure by vehicle direction; And
And generating and outputting warning information to warn the lane departure only when the lane departure probability is greater than or equal to a predetermined value. The method of claim 1, wherein the determining of lane departure for each vehicle direction is performed.
Correcting camera distortion after acquiring each lane detection image from each of the images; And
And detecting lane departure based on the lane position in the lane detection image for each vehicle direction. The method of claim 1, wherein the determining of lane departure for each vehicle direction is performed.
Correcting camera distortion of each of the images, and extracting and storing each lane detection image; And
And detecting lane departure based on the lane position in the lane detection image for each vehicle direction. The method of claim 2 or 3, wherein the determining of lane departure based on the lane position in the lane detection image is performed.
And detecting a lane departure only when the distance is less than a predetermined value after calculating a distance from the vehicle or the edge of the image to the lane when one lane is detected. The method of claim 2 or 3, wherein the determining of lane departure based on the lane position in the lane detection image is performed.
When two lanes are detected, after the first and second distances are calculated from the vehicle or the edge of the image with both lanes, the lane departure is determined only when the deviation of the first and second distances is greater than or equal to the set value. Lane departure detection method. The method of claim 1, wherein calculating the lane departure probability
And counting the number of vehicle directions in which the lane departure condition is confirmed, and dividing a counting result by the total number of vehicle directions to calculate the lane departure probability.

Images