CN114663860A - Lane line recognition system and method for lane departure system - Google Patents

Lane line recognition system and method for lane departure system Download PDF

Info

Publication number
CN114663860A
CN114663860A CN202210349514.2A CN202210349514A CN114663860A CN 114663860 A CN114663860 A CN 114663860A CN 202210349514 A CN202210349514 A CN 202210349514A CN 114663860 A CN114663860 A CN 114663860A
Authority
CN
China
Prior art keywords
lane
image
characteristic
central line
lane line
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210349514.2A
Other languages
Chinese (zh)
Inventor
江涛
李洪达
李建芳
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Chery and Wanda Guizhou Bus Co Ltd
Original Assignee
Chery and Wanda Guizhou Bus Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Chery and Wanda Guizhou Bus Co Ltd filed Critical Chery and Wanda Guizhou Bus Co Ltd
Priority to CN202210349514.2A priority Critical patent/CN114663860A/en
Publication of CN114663860A publication Critical patent/CN114663860A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/10Complex mathematical operations
    • G06F17/11Complex mathematical operations for solving equations, e.g. nonlinear equations, general mathematical optimization problems
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/20Analysis of motion
    • G06T7/277Analysis of motion involving stochastic approaches, e.g. using Kalman filters
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30248Vehicle exterior or interior
    • G06T2207/30252Vehicle exterior; Vicinity of vehicle
    • G06T2207/30256Lane; Road marking

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Theoretical Computer Science (AREA)
  • Pure & Applied Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Mathematics (AREA)
  • Mathematical Analysis (AREA)
  • Mathematical Optimization (AREA)
  • Software Systems (AREA)
  • Databases & Information Systems (AREA)
  • Algebra (AREA)
  • General Engineering & Computer Science (AREA)
  • Operations Research (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Traffic Control Systems (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a lane line recognition system and a lane line recognition method for a lane departure system, wherein the lane line recognition system comprises a characteristic image extraction module, a lane line characteristic image extraction module and a lane line recognition module, wherein the characteristic image extraction module is used for extracting a video image captured by a camera; the characteristic point extracting and tracking module is used for extracting characteristic points of the obtained image, and tracking the characteristic points after relatively reliable characteristic point positions are obtained; acquiring a curve lane line equation; and the lane central line equation fitting calculation module calculates the characteristic points of the lane central line through the characteristic points tracked at the two sides, fits a lane central line equation under the image coordinates, obtains a transverse and longitudinal coordinate proportional relation between the inverse perspective image and the actual road through calibration, and calculates the actual lane central line equation by using the lane central line equation and the proportional relation. The method can accurately identify the lane lines in real time and accurately calculate the lane central line equation, reduce traffic accidents caused by fatigue driving or distractions of a driver and improve the driving safety.

Description

Lane line recognition system and method for lane departure system
Technical Field
The invention relates to a lane line recognition system and method for a lane departure system, and belongs to the technical field of lane line recognition.
Background
With the increasing number of motor vehicles, the traffic safety problem caused by misoperation of drivers is increasingly serious, and the intelligent automobile technology represented by a safety auxiliary driving system can effectively reduce traffic accidents caused by misoperation of the drivers.
Of course, in the automobile safety driving early warning system in the outdoor natural environment, the lane line identification is seriously influenced by the change of the environmental illumination and the view angle. If the recognition rate of the lane line is slightly low and the recognition rate of the ACC (adaptive cruise) for a vehicle traveling on a preceding line is low, there is a risk of collision. Various approaches have been taken by many scholars in recent years to eliminate the effects of these factors. For example, some scholars adopt a histogram method to enhance the identification target threshold value, and adopt various edge detection methods with correction values to improve the edge detection precision. The methods can improve the identification precision and reduce the background noise in a stable illumination environment. But does not fundamentally solve the influence of actual dynamically changing natural illumination on lane line identification. Such as light intensity changes and view angle changes, affect the recognition accuracy. Other scholars have improved recognition accuracy by changing the physical properties of the camera, such as replacing the normal camera with an infrared camera. Although the method can directly eliminate the influence of environmental illumination, the manufacturing cost is high, and the method cannot be popularized to the current automobile safe driving auxiliary system.
Disclosure of Invention
The technical problem to be solved by the invention is as follows: a lane line recognition system and method for use in a lane departure system are provided to solve the technical problems in the prior art.
The technical scheme adopted by the invention is as follows: a lane line identification system for use in a lane departure system, comprising
The characteristic image extraction module is used for extracting the lane line characteristic image of the video image captured by the camera;
the characteristic point extraction and tracking module is used for extracting the characteristic points of the image obtained by the characteristic image extraction module, and tracking the characteristic points by using a Kalman filter after acquiring relatively reliable characteristic point positions; and obtaining a lane central line equation in an image coordinate system;
the lane central line equation fitting calculation module calculates the characteristic points of the lane central line through the characteristic points tracked at the two sides, fits a lane central line equation under the image coordinates, obtains the proportional relation of horizontal and vertical coordinates between the inverse perspective image and the actual road through calibration, and calculates the actual lane central line equation by utilizing the proportional relation of the lane central line equation under the image coordinate system obtained by the characteristic point extraction and tracking module.
A lane line identification method for use in a lane departure system, the method comprising: before lane departure early warning, lane line recognition is carried out and a lane central line equation is estimated; the method comprises the following specific steps:
firstly, in an image preprocessing stage, carrying out distortion correction on an image, then carrying out inverse perspective transformation (IPM) on the corrected image to obtain a top view of a road in front of a vehicle, and carrying out bilateral filtering on the IPM image to eliminate image noise;
secondly, edge detection is carried out on the IPM image by using a Sobel operator, and extraction of a lane line characteristic image is carried out on the basis of a lane line width model and the edge detection image, so that interference of non-lane line information is eliminated; then, after obtaining the lane line characteristic image, searching and extracting lane line characteristic points, and fitting the characteristic points by using a least square method to obtain a lane line equation;
thirdly, in order to improve the extraction precision of the lane feature points, tracking the lane line feature points by adopting a Kalman filter, establishing the relation of the feature points between the front frame and the rear frame, and establishing a dynamic region of interest for the detection of the subsequent images by using the acquired lane line equation; finally, calculating lane central line characteristic points according to the lane central line characteristic points tracked by the Kalman filter, and fitting to obtain a lane central line equation under pixel coordinates;
and fourthly, after obtaining a lane center line equation under the pixel coordinates, determining a proportional relation between the inverse perspective image and an actual road coordinate space, calculating information such as a lane departure early warning model, a lane departure early warning decision algorithm and the like before the vehicle is in the actual road coordinates, and setting an early warning trigger condition.
In the first step, graying of the image is firstly carried out, and then inverse perspective transformation is carried out on the image.
The method for extracting the lane characteristic image in the second step comprises the following steps: setting g (x, y) as gradient values of the edge detection image, T as a gradient threshold value and L as lane line width; when the gradient g (x, y) of a certain point is greater than the threshold value T and the gradient value of the pixel point g (x + L, y) at the position horizontally separated from the certain point by the distance L is also greater than the threshold value T, the pixel values between the two points are all set to be 255 and the pixel value of the pixel point which does not meet the condition is set to be 0, so that the extraction from the Sobel edge detection image to the lane feature image is completed.
The invention has the beneficial effects that: compared with the prior art, the lane line identification method based on machine vision can identify the lane line in real time and calculate the lane center line equation, and can send out early warning signals when the vehicle deviates from the current lane or has the tendency of deviating from the current lane, thereby reducing traffic accidents caused by fatigue driving or distractions of a driver and improving the driving safety.
Drawings
FIG. 1 is a logic diagram of a lane line identification algorithm;
FIG. 2 is a schematic diagram of an early warning process of a lane detection and departure early warning system;
FIG. 3 is a flow chart of feature image extraction;
fig. 4 is a flowchart of a lane feature image extraction method.
Detailed Description
The invention is further described with reference to the accompanying drawings and specific embodiments.
Example 1: as shown in fig. 1 to 4, a lane line recognition system for use in a lane departure system includes
The characteristic image extraction module is used for extracting the lane line characteristic image of the video image captured by the camera; graying an image, performing inverse perspective transformation (IPM) on the image to obtain a top view image, performing edge detection on the image by using a Sobel operator, and extracting a lane line characteristic image from the edge detection image by using a lane line width model;
the characteristic point extraction and tracking module is used for extracting the characteristic points of the image obtained by the characteristic image extraction module, and tracking the characteristic points by using a Kalman filter after acquiring relatively reliable characteristic point positions; and fitting the characteristic points by using a least square method to obtain a curve lane line equation and draw a lane line. In order to further improve the lane line identification precision and speed, a dynamic interested region is established by utilizing the obtained lane lines on the two sides, the detection range of the later frame image is limited, and the detection precision is improved; dividing the left lane and the right lane by utilizing the interested regions on two sides, and distinguishing the lanes with the solid lines and the lanes with the dotted lines by counting the number of projection points;
the lane central line equation fitting calculation module calculates the characteristic points of the lane central line through the characteristic points tracked at the two sides, fits a lane central line equation under the image coordinates, obtains the proportional relation of horizontal and vertical coordinates between the inverse perspective image and the actual road through calibration, and calculates the actual lane central line equation by utilizing the proportional relation of the lane central line equation under the image coordinate system obtained by the characteristic point extraction and tracking module.
The characteristic image extraction module is a precondition for realizing lane line detection and lane departure early warning algorithm. The module comprises image preprocessing and inverse perspective lane line characteristic image extraction. The image preprocessing aims at reducing the noise interference in the image, reducing the calculation time and improving the robustness and the real-time performance of the algorithm. The system acquires video image information of a road in front of a vehicle from a camera arranged at the front part of the vehicle, and a large amount of interference exists in an image which is not processed, wherein the interference includes shadow interference, road surface reflection, fallen leaf shielding, vehicle interference, road surface non-lane mark interference, lane line missing interference and the like caused by sunlight irradiating a roadside road tree. The system needs to highlight lane line information from these disturbances. In order to achieve the purpose of highlighting lane line information and reducing interference, an image preprocessing technology needs to be applied, and the image preprocessing technology comprises image graying, image smoothing, threshold operation, edge detection and other image preprocessing algorithms.
Graying is a conversion step of realizing a three-channel color image to a single-channel gray image, and the function of the step is to reduce the complexity of image processing and improve the image processing speed.
Image smoothing, which may also be referred to as blurring, is a method used to reduce noise and distortion in an image.
The threshold operation is used for extracting the lane line information, the threshold operation is to set a threshold value firstly, the gray level of pixel points in the image is compared with the threshold value, the pixel points which are larger than the threshold value are considered as target points, and the pixel points which are smaller than the threshold value are filtered, so that the information complexity in the image is further reduced, the information in the image has only two values, the data amount of image processing is reduced, and the target information is highlighted.
The edge detection is a method for highlighting a detection target by detecting edge information in an image, and an edge detection algorithm adopted by the invention is Sobel edge detection.
Because the cost of the vision camera is relatively low, the lane departure early warning system based on vision is the current mainstream lane departure early warning form, the system can acquire the image information of the road in front through a monocular or binocular camera, then uses an algorithm program to process the image, identifies the lane line, judges the current position of the vehicle and whether the vehicle has the danger of deviating the lane, and finally feeds back information to the driver according to the judgment result, and the flow of the lane line identification method for the lane departure system provided by the invention is shown in figure 2.
Example 2: as shown in fig. 1, a lane line recognition method for use in a lane departure system, the method comprising: before lane departure early warning, lane line recognition is carried out and a lane central line equation is estimated; the method comprises the following specific steps:
firstly, distortion correction is carried out on an image in an image preprocessing stage, inverse perspective transformation (IPM) is carried out on the corrected image to obtain a top view of a road in front of a vehicle, and bilateral filtering is carried out on the IPM image to eliminate image noise;
secondly, edge detection is carried out on the IPM image by using a Sobel operator, and extraction of a lane line characteristic image is carried out on the basis of a lane line width model and the edge detection image, so that interference of non-lane line information is eliminated; then, after obtaining the lane line characteristic image, searching and extracting lane line characteristic points, and fitting the characteristic points by using a least square method to obtain a lane line equation;
thirdly, in order to improve the extraction precision of the lane feature points, tracking the lane line feature points by adopting a Kalman filter, establishing the relation of the feature points between the front frame and the rear frame, and establishing a dynamic region of interest for the detection of the subsequent images by using the acquired lane line equation; finally, calculating lane central line characteristic points according to the lane central line characteristic points tracked by the Kalman filter, and fitting to obtain a lane central line equation under pixel coordinates;
and fourthly, after obtaining a lane center line equation under the pixel coordinates, determining a proportional relation between the inverse perspective image and an actual road coordinate space, calculating information such as a front road center line equation and a curvature radius under the actual road coordinates, analyzing a lane departure early warning model, making a departure early warning decision algorithm, and setting an early warning trigger condition.
Fig. 3 shows a flow of extracting the lane line feature image according to the present invention. The method comprises the steps of directly detecting a lane line from an original image, wherein the detected lane line cannot reflect the actual road line shape due to the fact that the vision has a perspective effect, and in order to accurately detect the lane line from the image and accurately reflect the road line shape, the method comprises the steps of firstly carrying out graying on the image, then carrying out inverse perspective transformation on the image to obtain a top view, and finally carrying out edge detection on the top view by using a Sobel operator. The lane line information in the image after the edge detection is relatively obvious, but various interferences still exist, so that a lane line feature image extraction method combining an inverse perspective edge detection image and lane line width characteristics is provided on the basis of the edge detection, and the lane line feature image is extracted.
Fig. 4 is a lane feature image extraction method of the present invention, which comprises: setting g (x, y) as gradient values of the edge detection image, T as a gradient threshold value and L as lane line width; when the gradient g (x, y) of a certain point is greater than the threshold value T and the gradient value of the pixel point g (x + L, y) at the position horizontally separated from the certain point by the distance L is also greater than the threshold value T, the pixel values between the two points are all set to be 255 and the pixel value of the pixel point which does not meet the condition is set to be 0, so that the extraction from the Sobel edge detection image to the lane feature image is completed.
The above description is only an embodiment of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive of changes or substitutions within the technical scope of the present invention, and therefore, the scope of the present invention should be determined by the scope of the claims.

Claims (4)

1. A lane line identification system for use in a lane departure system, characterized by: comprises that
The characteristic image extraction module is used for extracting the lane line characteristic image of the video image captured by the camera;
the characteristic point extraction and tracking module is used for extracting the characteristic points of the image obtained by the characteristic image extraction module, and tracking the characteristic points by using a Kalman filter after acquiring relatively reliable characteristic point positions; and obtaining a lane central line equation in an image coordinate system;
the lane central line equation fitting calculation module calculates the characteristic points of the lane central line through the characteristic points tracked at the two sides, fits a lane central line equation under the image coordinates, obtains the proportional relation of horizontal and vertical coordinates between the inverse perspective image and the actual road through calibration, and calculates the actual lane central line equation by utilizing the proportional relation of the lane central line equation under the image coordinate system obtained by the characteristic point extraction and tracking module.
2. A lane line identification method for use in a lane departure system, characterized by: the method comprises the following steps: before lane departure early warning, lane line recognition is carried out and a lane central line equation is estimated; the method comprises the following specific steps:
firstly, distortion correction is carried out on an image in an image preprocessing stage, then inverse perspective transformation is carried out on the corrected image to obtain a top view of a road in front of a vehicle, and bilateral filtering is carried out on an IPM image to eliminate image noise;
secondly, performing edge detection on the IPM image by using a Sobel operator, and extracting a lane line characteristic image based on a lane line width model and the edge detection image; then, after obtaining the lane line characteristic image, searching and extracting lane line characteristic points, and fitting the characteristic points by using a least square method to obtain a lane line equation;
thirdly, tracking the characteristic points of the lane line by adopting a Kalman filter, establishing the relation of the characteristic points between the front frame and the rear frame, and establishing a dynamic region of interest by using the acquired lane line equation; finally, calculating lane central line characteristic points according to the lane central line characteristic points tracked by the Kalman filter, and fitting to obtain a lane central line equation under pixel coordinates;
and fourthly, after obtaining a lane center line equation under the pixel coordinates, determining a proportional relation between the inverse perspective image and an actual road coordinate space, calculating information such as a front road center line equation and a curvature radius under the actual road coordinates, analyzing a lane departure early warning model, making a departure early warning decision algorithm, and setting an early warning trigger condition.
3. The lane line identification method used in a lane departure system according to claim 2, characterized in that: in the first step, graying of the image is firstly carried out, and then inverse perspective transformation is carried out on the image.
4. The lane line identification method used in a lane departure system according to claim 2, characterized in that: the method for extracting the lane characteristic image in the second step comprises the following steps: setting g (x, y) as gradient values of the edge detection image, T as a gradient threshold value and L as lane line width; when the gradient g (x, y) of a certain point is greater than the threshold value T and the gradient value of the pixel point g (x + L, y) at the position horizontally separated from the certain point by the distance L is also greater than the threshold value T, the pixel values between the two points are all set to be 255 and the pixel value of the pixel point which does not meet the condition is set to be 0, so that the extraction from the Sobel edge detection image to the lane feature image is completed.
CN202210349514.2A 2022-04-02 2022-04-02 Lane line recognition system and method for lane departure system Pending CN114663860A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210349514.2A CN114663860A (en) 2022-04-02 2022-04-02 Lane line recognition system and method for lane departure system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210349514.2A CN114663860A (en) 2022-04-02 2022-04-02 Lane line recognition system and method for lane departure system

Publications (1)

Publication Number Publication Date
CN114663860A true CN114663860A (en) 2022-06-24

Family

ID=82033429

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210349514.2A Pending CN114663860A (en) 2022-04-02 2022-04-02 Lane line recognition system and method for lane departure system

Country Status (1)

Country Link
CN (1) CN114663860A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115265493A (en) * 2022-09-26 2022-11-01 四川省公路规划勘察设计研究院有限公司 Lane-level positioning method and device based on non-calibrated camera

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110443225A (en) * 2019-08-15 2019-11-12 安徽半问科技有限公司 Virtual and real lane line identification method and device based on feature pixel statistics
CN110487562A (en) * 2019-08-21 2019-11-22 北京航空航天大学 One kind being used for unpiloted road-holding ability detection system and method
CN110647850A (en) * 2019-09-27 2020-01-03 福建农林大学 Automatic lane deviation measuring method based on inverse perspective principle
CN111002990A (en) * 2019-12-05 2020-04-14 华南理工大学 Lane departure early warning method and system based on dynamic departure threshold
CN112307953A (en) * 2020-10-29 2021-02-02 无锡物联网创新中心有限公司 Clustering-based adaptive inverse perspective transformation lane line identification method and system

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110443225A (en) * 2019-08-15 2019-11-12 安徽半问科技有限公司 Virtual and real lane line identification method and device based on feature pixel statistics
CN110487562A (en) * 2019-08-21 2019-11-22 北京航空航天大学 One kind being used for unpiloted road-holding ability detection system and method
CN110647850A (en) * 2019-09-27 2020-01-03 福建农林大学 Automatic lane deviation measuring method based on inverse perspective principle
CN111002990A (en) * 2019-12-05 2020-04-14 华南理工大学 Lane departure early warning method and system based on dynamic departure threshold
CN112307953A (en) * 2020-10-29 2021-02-02 无锡物联网创新中心有限公司 Clustering-based adaptive inverse perspective transformation lane line identification method and system

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
程超: ""基于机器视觉的曲线车道线识别与车道偏离预警研究"", 《工程科技Ⅱ辑》, 5 January 2020 (2020-01-05), pages 1 - 6 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115265493A (en) * 2022-09-26 2022-11-01 四川省公路规划勘察设计研究院有限公司 Lane-level positioning method and device based on non-calibrated camera

Similar Documents

Publication Publication Date Title
CN110979321B (en) Obstacle avoidance method for unmanned vehicle
CN107330376B (en) Lane line identification method and system
CN106096525B (en) A kind of compound lane recognition system and method
CN107462223B (en) Automatic measuring device and method for sight distance of vehicle before turning on highway
EP1796043B1 (en) Object detection
CN104866823B (en) A kind of moving vehicles detection and tracking method based on monocular vision
CN104899554A (en) Vehicle ranging method based on monocular vision
Li et al. Nighttime lane markings recognition based on Canny detection and Hough transform
CN107862290A (en) Method for detecting lane lines and system
JP6723328B2 (en) Vehicle detection method, night-time vehicle detection method and system based on dynamic light intensity
CN103902985B (en) High-robustness real-time lane detection algorithm based on ROI
CN111553214B (en) Method and system for detecting smoking behavior of driver
CN107895151A (en) Method for detecting lane lines based on machine vision under a kind of high light conditions
CN202134079U (en) Unmanned vehicle lane marker line identification and alarm device
CN114758322B (en) Road quality detection system based on machine identification
CN107563301A (en) Red signal detection method based on image processing techniques
CN109285169B (en) Road rescue equipment side direction towing induction method based on wheel identification
Chen et al. A novel lane departure warning system for improving road safety
CN112927283A (en) Distance measuring method and device, storage medium and electronic equipment
Jiang et al. Lane line detection optimization algorithm based on improved Hough transform and R-least squares with dual removal
CN102831426B (en) Road environment self-adaptive straight-lane detection method
CN114663860A (en) Lane line recognition system and method for lane departure system
CN113221739B (en) Monocular vision-based vehicle distance measuring method
CN113095283A (en) Lane line extraction method based on dynamic ROI and improved firefly algorithm
CN112329694A (en) Method and device for automatically identifying lane line and readable storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination