CN111222417A - Method and device for improving lane line extraction precision based on vehicle-mounted image - Google Patents
Method and device for improving lane line extraction precision based on vehicle-mounted image Download PDFInfo
- Publication number
- CN111222417A CN111222417A CN201911350299.2A CN201911350299A CN111222417A CN 111222417 A CN111222417 A CN 111222417A CN 201911350299 A CN201911350299 A CN 201911350299A CN 111222417 A CN111222417 A CN 111222417A
- Authority
- CN
- China
- Prior art keywords
- lane line
- vehicle
- coordinate system
- image
- world coordinate
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/588—Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration by the use of more than one image, e.g. averaging, subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10032—Satellite or aerial image; Remote sensing
- G06T2207/10044—Radar image
Abstract
The invention relates to a method and a device for improving extraction precision of a lane line based on a vehicle-mounted image. Aiming at the problem that when lane line extraction is carried out based on a vehicle-mounted single front view, due to the fact that the accuracy of the lane line extraction is not high at a position far away from a vehicle due to the depth of field, the accuracy is optimized by adopting a method for splicing left and right side views in consideration of the fact that the lane lines on the left and right sides of the vehicle are relatively close to the vehicle, the method is high in accuracy, basically free of influence of hardware equipment, and high in adaptability.
Description
Technical Field
The invention relates to the technical field of automatic driving high-precision map making, in particular to a method and a device for improving lane line extraction precision based on a vehicle-mounted image.
Background
The lane lines are left and right side lines defining a lane range, and the lane range is generally defined by the lane lines printed on the ground and is roughly divided into five types, namely a single dotted line, a single solid line, a double dotted line and a virtual solid line, so that the vehicle can be ensured to run in a correct lane, safety guarantee is provided for the vehicle to run, and the lane lines are the most important road elements in the automatic driving high-precision map making. In general, in lane line extraction based on a vehicle-mounted image, a single front view is used for processing, but the depth of a front view is long, and the accuracy of lane line extraction is worse at positions in the image which are farther away from a vehicle, so that the lane line extraction is performed by using the front view.
Disclosure of Invention
The invention provides a method and a device for improving lane line extraction precision based on a vehicle-mounted image, aiming at the problem that the lane line extraction precision is not high at a place far away from a vehicle due to the depth of field when the lane line extraction is carried out based on a vehicle-mounted single front view, and considering that the lane lines at the left side and the right side of the vehicle are closer to the vehicle, a method for splicing the left side view and the right side view is adopted to optimize the precision.
The technical scheme for solving the technical problems is as follows:
in a first aspect, the present invention provides a method for improving lane line extraction accuracy based on a vehicle-mounted image, comprising the steps of:
s1, acquiring image data acquired by left and right sensors of the vehicle and laser point cloud data acquired by a vehicle-mounted laser radar under a world coordinate system, and determining an image splicing plane;
s2, splicing the left and right image data, identifying the lane line in the image data, and converting the image coordinate value of the lane line into a world coordinate value;
and S3, performing elevation correction on the lane line by using the laser point cloud data according to the XY values of the lane line in the world coordinate system.
Further, before the step S1, the method further includes calibrating the left and right sensors of the vehicle, and obtaining a homography matrix for transforming the left and right lane lines between the world coordinate system and the pixel coordinate system.
Further, the converting of the image coordinate values of the lane lines into world coordinate values in step S2 is specifically:
and converting the image coordinate values of the lane lines into world coordinate values by using the homography matrix for converting the left and right lane lines between the world coordinate system and the pixel coordinate system.
Further, the method further comprises:
and S4, denoising, thinning and smoothing the lane line image data obtained after elevation correction.
In a second aspect, the present invention further provides an apparatus for improving accuracy of lane line extraction based on a vehicle-mounted image, including:
the data acquisition module is used for acquiring image data acquired by the left and right sensors of the vehicle and laser point cloud data acquired by the vehicle-mounted laser radar under a world coordinate system and determining an image splicing plane;
the coordinate conversion module is used for splicing the left image data and the right image data, identifying a lane line in the image data and converting an image coordinate value of the lane line into a world coordinate value;
and the correcting module is used for performing elevation correction on the lane line by using the laser point cloud data according to the XY values of the lane line in the world coordinate system.
Further, the device also comprises a transformation matrix generation module which is used for acquiring a homography matrix for transforming the left and right lane lines between a world coordinate system and a pixel coordinate system according to the calibration result of the left and right sensors of the vehicle.
Further, the device also comprises an optimization module which is used for carrying out denoising, thinning and smoothing on the lane line image data obtained after elevation correction.
In a third aspect, the present invention also provides an electronic device, including:
a memory for storing a computer software program;
and the processor is used for reading and executing the computer software program stored in the memory and is used for realizing the method for improving the lane line extraction precision based on the vehicle-mounted image in the first aspect of the invention.
In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium, in which a computer software program for implementing the method for improving the accuracy of lane line extraction based on-board images according to the first aspect of the present invention is stored.
The method greatly reduces the influence of the depth of field length of the image on the extraction precision of the lane line, has strong adaptability, integrally improves the recognition precision of the lane line of the image, and indirectly reduces the equipment cost.
Drawings
FIG. 1 is a flowchart of a method for improving lane line extraction accuracy based on a vehicle-mounted image according to an embodiment of the present invention;
fig. 2 is a structural diagram of an apparatus for improving lane line extraction accuracy based on a vehicle-mounted image according to an embodiment of the present invention.
Detailed Description
The principles and features of this invention are described below in conjunction with the following drawings, which are set forth by way of illustration only and are not intended to limit the scope of the invention.
Example one
The embodiment of the invention provides a method for improving lane line extraction precision based on a vehicle-mounted image, aiming at the problem that the lane line extraction precision is not high at a place far away from a vehicle due to the depth of field when the lane line extraction is carried out based on a vehicle-mounted single front view, and considering that the lane lines on the left side and the right side of the vehicle are closer to the vehicle, a method for splicing the left side view and the right side view is adopted to optimize the precision.
The data used by the invention comprises image data and laser point cloud data, wherein the image data is left and right side views of the vehicle-mounted panoramic camera, and the laser point cloud data is point cloud data acquired by a vehicle-mounted laser radar under world coordinates.
Specifically, as shown in fig. 1, the method includes the following steps:
and S1, acquiring image data acquired by the left and right sensors of the vehicle and laser point cloud data acquired by the vehicle-mounted laser radar under a world coordinate system, and determining an image splicing plane which is usually selected as a horizontal plane.
Before that, the left and right sensors of the vehicle-mounted panoramic camera need to be calibrated so as to respectively obtain homography matrixes of the left and right lane lines which are transformed between a world coordinate system and a pixel coordinate system, and subsequent image splicing is facilitated.
And S2, splicing the left and right image data, identifying the lane lines in the image data, and carrying out lane line identification processing on the spliced image by using deep learning or traditional computer vision. Here, it is necessary to filter the recognition result to eliminate false detection as much as possible, and then convert the image coordinate values of the lane line into world coordinate values.
The coordinate mapping relationship between the pixel coordinate system in which the image data is located and the world coordinate system is as follows:
wherein u, v represent coordinates in a pixel coordinate system, s represents a scale factor, fx、fy、u0、v0Gamma represents 5 camera internal parameters, r and t represent camera external parameters, and xw、yw、zwRepresenting coordinates in a world coordinate system.
The homography matrix H is defined as:
the coordinate mapping relationship may be expressed as:
and S3, according to the XY values of the lane lines in the world coordinate system, performing plane precision optimization on the result by using the reflection intensity values of the laser point cloud, and performing elevation correction by using the Z value. And meanwhile, the false detection and elimination are carried out on the result again by utilizing the characteristics of the lane line set.
And S4, denoising, thinning and smoothing the lane line image data obtained after elevation correction.
Example two
As shown in fig. 2, an embodiment of the present invention provides an apparatus for improving accuracy of lane line extraction based on a vehicle-mounted image, and implements the method provided in the first embodiment, where the apparatus specifically includes:
and the transformation matrix generation module is used for acquiring a homography matrix for transforming the left and right lane lines between the world coordinate system and the pixel coordinate system according to the calibration result of the left and right sensors of the vehicle.
The data acquisition module is used for acquiring image data acquired by the left and right sensors of the vehicle and laser point cloud data acquired by the vehicle-mounted laser radar under a world coordinate system and determining an image splicing plane;
the coordinate conversion module is used for splicing the left image data and the right image data, identifying a lane line in the image data and converting an image coordinate value of the lane line into a world coordinate value;
and the correcting module is used for performing elevation correction on the lane line by using the laser point cloud data according to the XY values of the lane line in the world coordinate system.
And the optimization module is used for denoising, thinning and smoothing the lane line image data obtained after elevation correction.
It should be noted that the method described in the first embodiment may be implemented by a computer software program, and based on this, an embodiment of the present invention further provides an electronic device, including:
a memory for storing a computer software program;
and the processor is used for reading and executing the computer software program stored in the memory and is used for realizing a method for improving the lane line extraction precision based on the vehicle-mounted image.
It should also be noted that the logic instructions in the computer software program can be implemented in the form of software functional units and stored in a computer readable storage medium when the software functional units are sold or used as independent products. Based on such understanding, the technical solutions of the embodiments of the present invention may be essentially implemented or make a contribution to the prior art, or may be implemented in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the methods described in the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.
Claims (9)
1. A method for improving lane line extraction accuracy based on vehicle-mounted images is characterized by comprising the following steps:
s1, acquiring image data acquired by left and right sensors of the vehicle and laser point cloud data acquired by a vehicle-mounted laser radar under a world coordinate system, and determining an image splicing plane;
s2, splicing the left and right image data, identifying the lane line in the image data, and converting the image coordinate value of the lane line into a world coordinate value;
and S3, performing elevation correction on the lane line by using the laser point cloud data according to the XY values of the lane line in the world coordinate system.
2. The method of claim 1, further comprising, before the step S1, calibrating the left and right sensors of the vehicle to obtain a homography matrix for transforming the left and right lane lines between the world coordinate system and the pixel coordinate system.
3. The method according to claim 2, wherein the transforming the image coordinate values of the lane lines into world coordinate values in step S2 is specifically:
and converting the image coordinate values of the lane lines into world coordinate values by using the homography matrix for converting the left and right lane lines between the world coordinate system and the pixel coordinate system.
4. The method of claim 1, further comprising:
and S4, denoising, thinning and smoothing the lane line image data obtained after elevation correction.
5. An apparatus for improving accuracy of lane line extraction based on-vehicle images, comprising:
the data acquisition module is used for acquiring image data acquired by the left and right sensors of the vehicle and laser point cloud data acquired by the vehicle-mounted laser radar under a world coordinate system and determining an image splicing plane;
the coordinate conversion module is used for splicing the left image data and the right image data, identifying a lane line in the image data and converting an image coordinate value of the lane line into a world coordinate value;
and the correcting module is used for performing elevation correction on the lane line by using the laser point cloud data according to the XY values of the lane line in the world coordinate system.
6. The apparatus of claim 5, further comprising a transformation matrix generation module for obtaining a homography matrix for transforming the left and right lane lines between the world coordinate system and the pixel coordinate system according to the calibration result of the left and right sensors of the vehicle.
7. The apparatus of claim 5, further comprising an optimization module for performing denoising, thinning and smoothing on the image data of the lane line obtained after the elevation correction.
8. An electronic device, comprising:
a memory for storing a computer software program;
a processor for reading and executing the computer software program stored in the memory for implementing a method for improving lane line extraction accuracy based on-board images according to any one of claims 1 to 4.
9. A non-transitory computer-readable storage medium having stored thereon a computer software program for implementing a method for improving vehicle-mounted image lane line extraction accuracy according to any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911350299.2A CN111222417A (en) | 2019-12-24 | 2019-12-24 | Method and device for improving lane line extraction precision based on vehicle-mounted image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911350299.2A CN111222417A (en) | 2019-12-24 | 2019-12-24 | Method and device for improving lane line extraction precision based on vehicle-mounted image |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111222417A true CN111222417A (en) | 2020-06-02 |
Family
ID=70829163
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911350299.2A Pending CN111222417A (en) | 2019-12-24 | 2019-12-24 | Method and device for improving lane line extraction precision based on vehicle-mounted image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111222417A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111881752A (en) * | 2020-06-27 | 2020-11-03 | 武汉中海庭数据技术有限公司 | Guardrail detection and classification method and device, electronic equipment and storage medium |
| CN112837384A (en) * | 2021-03-01 | 2021-05-25 | 东软睿驰汽车技术（沈阳）有限公司 | Vehicle marking method and device and electronic equipment |
| CN113255619A (en) * | 2021-07-09 | 2021-08-13 | 禾多科技（北京）有限公司 | Lane line recognition and positioning method, electronic device, and computer-readable medium |
| WO2021253245A1 (en) * | 2020-06-16 | 2021-12-23 | 华为技术有限公司 | Method and device for identifying vehicle lane changing tendency |
| CN117152707A (en) * | 2023-10-31 | 2023-12-01 | 武汉未来幻影科技有限公司 | Calculation method and device for offset distance of vehicle and processing equipment |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104036279A (en) * | 2014-06-12 | 2014-09-10 | 北京联合大学 | Intelligent vehicle running control method and system |
| CN104766058A (en) * | 2015-03-31 | 2015-07-08 | 百度在线网络技术（北京）有限公司 | Method and device for obtaining lane line |
| CN107463918A (en) * | 2017-08-17 | 2017-12-12 | 武汉大学 | Lane line extracting method based on laser point cloud and image data fusion |
| CN110163930A (en) * | 2019-05-27 | 2019-08-23 | 北京百度网讯科技有限公司 | Lane line generation method, device, equipment, system and readable storage medium storing program for executing |
-
2019
- 2019-12-24 CN CN201911350299.2A patent/CN111222417A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104036279A (en) * | 2014-06-12 | 2014-09-10 | 北京联合大学 | Intelligent vehicle running control method and system |
| CN104766058A (en) * | 2015-03-31 | 2015-07-08 | 百度在线网络技术（北京）有限公司 | Method and device for obtaining lane line |
| CN107463918A (en) * | 2017-08-17 | 2017-12-12 | 武汉大学 | Lane line extracting method based on laser point cloud and image data fusion |
| CN110163930A (en) * | 2019-05-27 | 2019-08-23 | 北京百度网讯科技有限公司 | Lane line generation method, device, equipment, system and readable storage medium storing program for executing |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021253245A1 (en) * | 2020-06-16 | 2021-12-23 | 华为技术有限公司 | Method and device for identifying vehicle lane changing tendency |
| CN111881752A (en) * | 2020-06-27 | 2020-11-03 | 武汉中海庭数据技术有限公司 | Guardrail detection and classification method and device, electronic equipment and storage medium |
| CN112837384A (en) * | 2021-03-01 | 2021-05-25 | 东软睿驰汽车技术（沈阳）有限公司 | Vehicle marking method and device and electronic equipment |
| CN113255619A (en) * | 2021-07-09 | 2021-08-13 | 禾多科技（北京）有限公司 | Lane line recognition and positioning method, electronic device, and computer-readable medium |
| CN117152707A (en) * | 2023-10-31 | 2023-12-01 | 武汉未来幻影科技有限公司 | Calculation method and device for offset distance of vehicle and processing equipment |
| CN117152707B (en) * | 2023-10-31 | 2024-03-22 | 武汉未来幻影科技有限公司 | Calculation method and device for offset distance of vehicle and processing equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111222417A (en) | Method and device for improving lane line extraction precision based on vehicle-mounted image | |
| CN110163930B (en) | Lane line generation method, device, equipment, system and readable storage medium | |
| CN107481292B (en) | Attitude error estimation method and device for vehicle-mounted camera | |
| US10789719B2 (en) | Method and apparatus for detection of false alarm obstacle | |
| CN112348902B (en) | Method, device and system for calibrating installation deviation angle of road-end camera | |
| CN113988112B (en) | Method, device and equipment for detecting lane line and storage medium | |
| CN109741241B (en) | Fisheye image processing method, device, equipment and storage medium | |
| CN107609483A (en) | Risk object detection method, device towards drive assist system | |
| CN114067001B (en) | Vehicle-mounted camera angle calibration method, terminal and storage medium | |
| CN112348752B (en) | Lane line vanishing point compensation method and device based on parallel constraint | |
| CN114280582A (en) | Calibration and calibration method and device for laser radar, storage medium and electronic equipment | |
| CN112146848A (en) | Method and device for determining distortion parameter of camera | |
| CN110310243B (en) | Unmanned aerial vehicle photogrammetry image correction method, system and storage medium | |
| CN114919584A (en) | Motor vehicle fixed point target distance measuring method and device and computer readable storage medium | |
| CN110796604A (en) | Image correction method and device | |
| CN113297958A (en) | Automatic labeling method and device, electronic equipment and storage medium | |
| CN111191482A (en) | Brake lamp identification method and device and electronic equipment | |
| CN114140533A (en) | Method and device for calibrating external parameters of camera | |
| CN116630401A (en) | Fish-eye camera ranging method and terminal | |
| CN110827337A (en) | Method and device for determining posture of vehicle-mounted camera and electronic equipment | |
| CN114842443A (en) | Target object identification and distance measurement method, device and equipment based on machine vision and storage medium | |
| CN112529011A (en) | Target detection method and related device | |
| JP2016111585A (en) | Image processing system, system, image processing method, and program | |
| CN112654997B (en) | Lane line detection method and device | |
| CN115063772B (en) | Method for detecting vehicles after formation of vehicles, terminal equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200602 |