CN107133985B - Automatic calibration method for vehicle-mounted camera based on lane line vanishing point - Google Patents

Automatic calibration method for vehicle-mounted camera based on lane line vanishing point Download PDF

Info

Publication number
CN107133985B
CN107133985B CN201710261492.3A CN201710261492A CN107133985B CN 107133985 B CN107133985 B CN 107133985B CN 201710261492 A CN201710261492 A CN 201710261492A CN 107133985 B CN107133985 B CN 107133985B
Authority
CN
China
Prior art keywords
image
coordinate system
lane line
coordinates
line
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201710261492.3A
Other languages
Chinese (zh)
Other versions
CN107133985A (en
Inventor
薛远
陈向成
陈高灿
李兵
张勇
程腾
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Changzhou Zhixing Technology Co ltd
Original Assignee
Changzhou Zhixing Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Changzhou Zhixing Technology Co ltd filed Critical Changzhou Zhixing Technology Co ltd
Priority to CN201710261492.3A priority Critical patent/CN107133985B/en
Publication of CN107133985A publication Critical patent/CN107133985A/en
Application granted granted Critical
Publication of CN107133985B publication Critical patent/CN107133985B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)
  • Traffic Control Systems (AREA)

Abstract

The invention provides a vehicle-mounted camera automatic calibration method based on a lane line vanishing point, which comprises the following steps: acquiring an image shot by a camera when a vehicle is positioned in the middle of a lane, wherein the image comprises clear left and right lane lines; processing the image to obtain an equation of the left and right lane lines in the image, and calculating the intersection point P of the left and right lane lines in the image0And the intersection points P of the left and right lane lines and the lower edge line of the image respectively1、P2Coordinates in an image coordinate system; establishing a world coordinate system to obtain a midpoint P of the image0、P1、P2Corresponding real space point P0'、P1'、P2Coordinates in the world coordinate system; according to the point P in the image0、P1、P2Coordinates in the image coordinate system and corresponding real space points P0'、P1'、P2' coordinates in the world coordinate system, the calibration matrix is solved. The method can automatically process the shot image only by driving in the middle of the lane or stopping in the middle of the lane, obtains the calibration matrix after inquiring the width of the current lane, has high calibration precision, simple and convenient operation, does not need manual intervention, and can carry out calibration in real time.

Description

Automatic calibration method for vehicle-mounted camera based on lane line vanishing point
Technical Field
The invention relates to the field of automobile image processing, in particular to an automatic calibration method of a vehicle-mounted camera based on a lane line vanishing point.
Background
In recent years, technologies of automobile assisted driving (Advanced Driver Assistance Systems) have been rapidly developed, and a vehicle front Collision Warning (Forward Collision Warning) and a Lane departure Warning (Lane departure Warning) are important parts of the technologies. The two functions belong to computer vision application, and geometric information of an object in a three-dimensional space is calculated mainly from image information acquired by a camera to determine the distance between a vehicle and a front obstacle and whether the vehicle deviates from a lane line, and the vehicle-mounted camera needs to be accurately calibrated on the premise of realizing the functions. It is generally assumed that there is a simple linear relationship between the image captured by the camera and the object in three-dimensional space: the matrix M can be regarded as a geometric model of the camera image, the parameters in the matrix M are the camera parameters, the camera calibration process is a process of solving the matrix M, and the matrix M is a calibration matrix.
In the existing calibration methods, a tool with a specific shape and size is placed in front of a vehicle or on a road surface, and a calibration matrix is obtained after image processing. Some methods extract the inclination angle information of the camera through a GPS chip in the camera and obtain a calibration matrix through a series of complex calculations.
Disclosure of Invention
In order to solve the problems, the invention provides the automatic calibration method of the vehicle-mounted camera based on the vanishing point of the lane line, which can carry out calibration in real time without calibration reference objects with known shapes and sizes and is simple and easy to operate.
The specific technical scheme of the invention is as follows: a vehicle-mounted camera automatic calibration method based on lane line vanishing points specifically comprises the following steps:
step 1, acquiring an image shot by a camera when a vehicle is positioned in the middle of a lane, wherein the image is a scene in front of a vehicle head and comprises clear left and right lane lines;
step 2, processing the image, establishing an image coordinate system, obtaining a left lane line equation and a right lane line equation in the image, and calculating an intersection point P of the left lane line and the right lane line in the image0And the intersection points P of the left and right lane lines and the lower edge line of the image respectively1、P2Coordinates in an image coordinate system;
step 3, establishing a world coordinate system to obtain a midpoint P of the image0、P1、P2Corresponding real space point P0'、P1'、P2' coordinates in the world coordinate System, where the Point P0' is the vanishing point of the true left and right lane lines, point P1' and P2' points on the real left and right lane lines, respectively;
step 4, according to the point P in the image0、P1、P2Coordinates in the image coordinate system and corresponding real space points P0'、P1'、P2' coordinates in the world coordinate system, the calibration matrix is solved.
As a further improvement of the present invention, the step 2 specifically includes:
2.1, defining a first pixel at the upper left corner of the image as a coordinate origin, and establishing an image coordinate system taking the pixel as a unit, wherein the horizontal direction of the image is an X axis, and the longitudinal direction of the image is a Y axis;
2.2, Canny edge detection is firstly carried out on the image to obtain a binary image, Hough transformation linear detection is then carried out on the binary image, and a linear equation of a left lane line and a right lane line in the image is detected: the linear equation of the left lane line is y1＝k1x1+b1The straight line equation of the right lane line is y2＝k2x2+b2Wherein (x)1,y1) Is the coordinate of the pixel on the left lane line in the image coordinate system, (x)2,y2) Is the coordinate, k, of the pixel on the right lane line in the image coordinate system1Is the slope of the straight line, k, of the left lane line2Is the slope of the straight line of the right lane line, b1Is the straight line intercept of the left lane line, b2Is the straight line intercept, k, of the right lane line1、k2、b1、b2Obtaining in Hough transform line detection;
2.3, calculating the intersection point P of the two lane lines in the image according to the linear equation of the identified left and right lane lines0Coordinates in the image coordinate system: p0(x0,y0) Wherein
Figure BDA0001274871360000021
Calculating two vehicles in the imageIntersection point P of road line and lower edge of image1、P2Coordinates in the image coordinate system: p1(x1',y1') wherein
Figure BDA0001274871360000022
y′1＝H，P2(x′2,y′2) Wherein
Figure BDA0001274871360000023
y′2H is the number of pixels in the vertical direction of the image.
The invention has the beneficial effects that: the method of the invention uses the lane line as the calibration reference object, the vehicle only needs to run in the middle of the lane or stop in the middle of the lane, the image shot by the camera can be automatically processed, the calibration matrix is obtained after the width of the current lane is inquired, the calibration precision is high, the operation is simple and convenient, the manual intervention is not needed, and the calibration can be carried out in real time; the method selects the points where the left lane line and the right lane line respectively intersect with the vehicle head and the vanishing points of the left lane line and the right lane line as the calibration points, establishes an image coordinate system and a world coordinate system, and calculates the coordinates of the points under the image coordinate system and the world coordinate system respectively, thereby calculating a calibration matrix, and the method has the advantages of small calculation amount, quick calibration time and no calibration condition limitation.
Drawings
FIG. 1 is a flow chart of the method of the present invention.
FIG. 2 is a schematic diagram of the image coordinate system and the locations of the key points.
FIG. 3 is a schematic view of a world coordinate system corresponding to an image coordinate system established by the present invention.
Detailed Description
As shown in fig. 1, the method for automatically calibrating a vehicle-mounted camera based on a lane line vanishing point provided by the invention specifically comprises the following steps:
step 1, obtaining an image shot by a camera when a vehicle is positioned in the middle of a lane, wherein the image is a scene in front of a vehicle head and comprises clear left and right lane lines.
Step 2, processing the image to establish a graphObtaining the equation of the left and right lane lines in the image like a coordinate system, and calculating the intersection point P of the left and right lane lines in the image0And the intersection points P of the left and right lane lines and the lower edge of the image respectively1、P2The coordinates in the image coordinate system specifically include:
2.1, defining a first point at the upper left corner of the image as a coordinate origin, the horizontal direction of the image as an X axis, and the longitudinal direction as a Y axis, and establishing an image coordinate system as shown in FIG. 2;
2.2, the technology for identifying the left lane line and the right lane line is the prior art, Canny edge detection is firstly carried out on an image to obtain a binary image, Hough transformation linear detection is then carried out on the binary image, and a linear equation of the left lane line and the right lane line in the image is detected: the linear equation of the left lane line is y1＝k1x1+b1The straight line equation of the right lane line is y2＝k2x2+b2Wherein (x)1,y1) Is the coordinate of the pixel on the left lane line in the image coordinate system, (x)2,y2) Is the coordinate, k, of the pixel on the right lane line in the image coordinate system1Is the slope of the straight line, k, of the left lane line2Is the slope of the straight line of the right lane line, b1Is the straight line intercept of the left lane line, b2Is the straight line intercept, k, of the right lane line1、k2、b1、b2Obtaining in Hough transform line detection;
2.3, calculating the intersection point P of the two lane lines in the image according to the linear equation of the identified left lane line and the right lane line0Coordinates in the image coordinate system: p0(x0,y0) Wherein
Figure BDA0001274871360000031
The lane line vanishing point is represented by the intersection point of two lane lines in the image; calculating the intersection point P of two lane lines in the image and the lower edge of the image1、P2Coordinates in the image coordinate system: p1(x′1,y′1) Wherein
Figure BDA0001274871360000032
y′1＝H，P2(x′2,y′2) Wherein
Figure BDA0001274871360000033
y′2H is the number of pixels of the image in the longitudinal direction.
Step 3, establishing a world coordinate system to obtain a midpoint P of the image0、P1、P2Corresponding real space point P0'、P1'、P2' coordinates in the world coordinate system, specifically:
by P in the picture1P2The real space point corresponding to the middle point of the line segment is the original point, the real space direction opposite to the head is the positive direction of the Y axis, and P is1Corresponding real space point pointing direction P2The direction of the corresponding real space point is the positive direction of an X axis, a Z axis is vertical to a space road surface, and a world coordinate system shown in figure 3 is established;
due to point P in the image1、P2Corresponding real space point P1'、P2' the distance from the vehicle head is usually not more than 2 meters, and two points P can be approximated1'、P2' the coordinate on the Y-axis in the world coordinate system is 0. According to the principle of camera pinhole imaging, the vanishing point of two lane lines is at infinity (the parallel lines intersect at infinity), i.e. a certain point on the horizon, so point P0' the distance from the vehicle is infinite. But for camera parameters with a typical focal length of 2-8mm (a common type for tachographs and vehicle assisted driving systems), point P can be approximated0' the coordinate on the Y-axis in the world coordinate system is 10000, and at this time, the vanishing point of the real lane line can be represented by two points respectively located on the left and right lane lines. P for point correspondences in the image selected by the calibration process0'、P1'、P2' three points are points on the road surface in real space, so the coordinates of the three points on the Z axis in the world coordinate system are all 0. To sum up, obtain the real space point P0'、P1'、P2' the coordinates in the world coordinate system are respectively P0'(-D/2,10000),(D/2,10000)，P'1(-D/2,0)，P2' (D/2,0) wherein D isThe standard width of the lane in which the vehicle is located is obtained by referring to.
Step 4, according to the point P in the image0、P1、P2Coordinates in the image coordinate system and corresponding real space points P0'、P1'、P2' coordinates in the world coordinate system, the calibration matrix is solved.
The relationship between the image and the real space can be represented by:
Figure BDA0001274871360000041
wherein the content of the first and second substances,
Figure BDA0001274871360000042
the method is characterized in that (x, y) is a calibration matrix, the (x, y) is coordinates of pixel points in an image coordinate system, the (u, v) is coordinates of corresponding real space points in a world coordinate system, and t is a coefficient.
Will P0、P1、P2、P0'、P1'、P2' coordinates are substituted into the above equation, where P is substituted to avoid matrix singularities0Also using two coordinates (x)0-0.5,y0)、(x0+0.5,y0) To show, we get:
Figure BDA0001274871360000043
wherein the matrix parameter a11To a32And a coefficient t1To t4All the calibration matrixes M can be obtained by solving a linear equation, so that the calibration of the vehicle-mounted camera is completed.

Claims (1)

1. A vehicle-mounted camera automatic calibration method based on a lane line vanishing point is characterized by comprising the following steps:
step 1, acquiring an image shot by a camera when a vehicle is positioned in the middle of a lane, wherein the image is a scene in front of a vehicle head and comprises clear left and right lane lines;
step 2, processing the image, establishing an image coordinate system, obtaining a left lane line equation and a right lane line equation in the image, and calculating an intersection point P of the left lane line and the right lane line in the image0And the intersection points P of the left and right lane lines and the lower edge line of the image respectively1、P2Coordinates in an image coordinate system; the method specifically comprises the following steps:
2.1, defining a first pixel at the upper left corner of the image as a coordinate origin, and establishing an image coordinate system taking the pixel as a unit, wherein the horizontal direction of the image is an X axis, and the longitudinal direction of the image is a Y axis;
2.2, Canny edge detection is firstly carried out on the image to obtain a binary image, Hough transformation linear detection is then carried out on the binary image, and a linear equation of a left lane line and a right lane line in the image is detected: the linear equation of the left lane line is y ═ k1x+b1The straight line equation of the right lane line is y ═ k2x+b2Where (x, y) is the coordinate of the pixel on the lane line in the image coordinate system, k1Is the slope of the straight line, k, of the left lane line2Is the slope of the straight line of the right lane line, b1Is the straight line intercept of the left lane line, b2Is the straight line intercept, k, of the right lane line1、k2、b1、b2Obtaining in Hough transform line detection;
2.3, calculating the intersection point P of the two lane lines in the image according to the linear equation of the identified left and right lane lines0Coordinates in the image coordinate system: p0(x0,y0) Wherein
Figure FDA0002347336030000011
Calculating the intersection point P of two lane lines in the image and the lower edge of the image1、P2Coordinates in the image coordinate system: p1(x1,y1) Wherein
Figure FDA0002347336030000012
y1＝H，P2(x2,y2) Wherein
Figure FDA0002347336030000013
y2H is the number of pixels of the image in the longitudinal direction;
step 3, establishing a world coordinate system to obtain a midpoint P of the image0、P1、P2Corresponding real space point P0'、P1'、P2' coordinates in the world coordinate System, where the Point P0' is the vanishing point of the true left and right lane lines, P1' and P2' points on the real left and right lane lines, respectively;
step 4, according to the point P in the image0、P1、P2Coordinates in the image coordinate system and corresponding real space points P0'、P1'、P2The coordinates in the world coordinate system are used for solving a calibration matrix, and the method specifically comprises the following steps:
the relationship between the image and the real space can be represented by:
Figure FDA0002347336030000014
wherein the content of the first and second substances,
Figure FDA0002347336030000021
the method comprises the following steps of (1) obtaining a calibration matrix, (x, y) obtaining coordinates of pixel points in an image coordinate system, (u, v) obtaining coordinates of corresponding real space points in a world coordinate system, and t being a coefficient; wherein the matrix parameter a11To a32And the coefficient t can be obtained by solving a linear equation, so that a calibration matrix M is obtained, and the calibration of the vehicle-mounted camera is completed.
CN201710261492.3A 2017-04-20 2017-04-20 Automatic calibration method for vehicle-mounted camera based on lane line vanishing point Active CN107133985B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710261492.3A CN107133985B (en) 2017-04-20 2017-04-20 Automatic calibration method for vehicle-mounted camera based on lane line vanishing point

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710261492.3A CN107133985B (en) 2017-04-20 2017-04-20 Automatic calibration method for vehicle-mounted camera based on lane line vanishing point

Publications (2)

Publication Number Publication Date
CN107133985A CN107133985A (en) 2017-09-05
CN107133985B true CN107133985B (en) 2020-05-12

Family

ID=59716674

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710261492.3A Active CN107133985B (en) 2017-04-20 2017-04-20 Automatic calibration method for vehicle-mounted camera based on lane line vanishing point

Country Status (1)

Country Link
CN (1) CN107133985B (en)

Families Citing this family (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109584305A (en) * 2017-09-29 2019-04-05 宝沃汽车（中国）有限公司 Panorama system scaling method, device and vehicle
CN110555885B (en) * 2018-05-31 2023-07-04 海信集团有限公司 Calibration method and device of vehicle-mounted camera and terminal
CN110570475A (en) * 2018-06-05 2019-12-13 上海商汤智能科技有限公司 vehicle-mounted camera self-calibration method and device and vehicle driving method and device
CN110858405A (en) * 2018-08-24 2020-03-03 北京市商汤科技开发有限公司 Attitude estimation method, device and system of vehicle-mounted camera and electronic equipment
CN109572715A (en) * 2018-11-01 2019-04-05 北京海纳川汽车部件股份有限公司 Lane departure warning method, device and the automatic driving vehicle of automatic driving vehicle
CN109859278B (en) * 2019-01-24 2023-09-01 惠州市德赛西威汽车电子股份有限公司 Calibration method and calibration system for camera external parameters of vehicle-mounted camera system
CN110163176B (en) * 2019-05-28 2021-06-18 北京百度网讯科技有限公司 Lane line change position identification method, device, equipment and medium
CN112184821B (en) * 2019-07-01 2023-03-10 上海安亭地平线智能交通技术有限公司 Method and device for adjusting roll angle of camera, storage medium and electronic equipment
CN110435528A (en) * 2019-07-04 2019-11-12 华南师范大学 A kind of intelligent distance light lamp control method and system under low light environment
CN110718068B (en) * 2019-09-27 2020-12-08 华中科技大学 Road monitoring camera installation angle estimation method
CN112489136B (en) * 2020-11-30 2024-04-16 商汤集团有限公司 Calibration method, position determination device, electronic equipment and storage medium
CN112529968A (en) * 2020-12-22 2021-03-19 上海商汤临港智能科技有限公司 Camera equipment calibration method and device, electronic equipment and storage medium
CN112927309B (en) * 2021-03-26 2024-04-09 苏州欧菲光科技有限公司 Vehicle-mounted camera calibration method and device, vehicle-mounted camera and storage medium
CN113223300A (en) * 2021-04-13 2021-08-06 河北科技大学 Traffic accident speed measurement method based on vehicle-mounted video image
CN114120258B (en) * 2022-01-26 2022-05-03 深圳佑驾创新科技有限公司 Lane line identification method and device and storage medium
CN114659489A (en) * 2022-03-11 2022-06-24 苏州清研微视电子科技有限公司 Front vehicle distance detection method and device based on convex lens imaging principle

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103927748A (en) * 2014-04-09 2014-07-16 东南大学 Coordinate calibrating method based on multi-rectangle image distance transformation model
CN103927754A (en) * 2014-04-21 2014-07-16 大连理工大学 Calibration method of on-board camera
CN104063877A (en) * 2014-07-16 2014-09-24 中电海康集团有限公司 Hybrid judgment identification method for candidate lane lines

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103927748A (en) * 2014-04-09 2014-07-16 东南大学 Coordinate calibrating method based on multi-rectangle image distance transformation model
CN103927754A (en) * 2014-04-21 2014-07-16 大连理工大学 Calibration method of on-board camera
CN104063877A (en) * 2014-07-16 2014-09-24 中电海康集团有限公司 Hybrid judgment identification method for candidate lane lines

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
车载单目相机的在线自标定算法研究;刘明晴;《中国优秀硕士学位论文全文数据库-工程科技II辑》;20170315(第3期);第C035-219页第4章，第2.2.2节，第5.2节 *

Also Published As

Publication number Publication date
CN107133985A (en) 2017-09-05

Similar Documents

Publication Publication Date Title
CN107133985B (en) Automatic calibration method for vehicle-mounted camera based on lane line vanishing point
CN107738612B (en) Automatic parking space detection and identification system based on panoramic vision auxiliary system
CN107463918B (en) Lane line extraction method based on fusion of laser point cloud and image data
CN108647638B (en) Vehicle position detection method and device
US20200041284A1 (en) Map road marking and road quality collecting apparatus and method based on adas system
CN109389064B (en) Vehicle feature acquisition method and device
EP3678096B1 (en) Method for calculating a tow hitch position
Yan et al. A method of lane edge detection based on Canny algorithm
EP1394761A2 (en) Obstacle detection device and method therefor
CN110555407B (en) Pavement vehicle space identification method and electronic equipment
CN110334678A (en) A kind of pedestrian detection method of view-based access control model fusion
CN104700414A (en) Rapid distance-measuring method for pedestrian on road ahead on the basis of on-board binocular camera
Kum et al. Lane detection system with around view monitoring for intelligent vehicle
CN110197173B (en) Road edge detection method based on binocular vision
CN105206109A (en) Infrared CCD based foggy day identifying early-warning system and method for vehicle
CN109827516B (en) Method for measuring distance through wheel
CN109447908A (en) A kind of coil of strip recognition positioning method based on stereoscopic vision
Liu et al. Development of a vision-based driver assistance system with lane departure warning and forward collision warning functions
CN113205604A (en) Feasible region detection method based on camera and laser radar
CN111178150A (en) Lane line detection method, system and storage medium
CN112489106A (en) Video-based vehicle size measuring method and device, terminal and storage medium
CN114419143A (en) Depth measuring method, depth measuring apparatus, and storage medium
Petrovai et al. A stereovision based approach for detecting and tracking lane and forward obstacles on mobile devices
WO2018222122A1 (en) Methods for perspective correction, computer program products and systems
Sun Vision based lane detection for self-driving car

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant