CN112348752B - Lane line vanishing point compensation method and device based on parallel constraint - Google Patents
Lane line vanishing point compensation method and device based on parallel constraint Download PDFInfo
- Publication number
- CN112348752B CN112348752B CN202011168364.2A CN202011168364A CN112348752B CN 112348752 B CN112348752 B CN 112348752B CN 202011168364 A CN202011168364 A CN 202011168364A CN 112348752 B CN112348752 B CN 112348752B
- Authority
- CN
- China
- Prior art keywords
- lane line
- point
- characteristic
- vanishing point
- vanishing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 30
- 238000006243 chemical reaction Methods 0.000 claims abstract description 7
- 238000001514 detection method Methods 0.000 claims description 5
- 238000010586 diagram Methods 0.000 description 8
- 230000000694 effects Effects 0.000 description 7
- 230000006872 improvement Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 230000007704 transition Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
Images
Classifications
-
- G06T5/77—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30248—Vehicle exterior or interior
- G06T2207/30252—Vehicle exterior; Vicinity of vehicle
- G06T2207/30256—Lane; Road marking
Abstract
The invention discloses a lane line vanishing point compensation method and device based on parallel constraint, belonging to the field of vehicle auxiliary driving, and the method comprises the following steps: acquiring a road surface image acquired by a vehicle-mounted camera, identifying a left lane line and a right lane line of the road surface image in an image coordinate system, and respectively selecting a plurality of lane line characteristic points on each lane line; respectively solving the coordinate position of each lane line characteristic point selected on each lane line in a world coordinate system by using the initial position of the vanishing point; and if the world coordinates of the characteristic points of the lane lines on the left lane line and the world coordinates of the characteristic points of the lane lines on the right lane line do not have the parallel characteristic, compensating the positions of the vanishing points, performing coordinate conversion according to the compensated positions of the vanishing points until the world coordinates of the characteristic points of the lane lines on the left lane line and the right lane line have the parallel characteristic, and taking the positions of the vanishing points which are finally compensated as the final positions of the vanishing points. The invention improves the precision of the vanishing point compensation.
Description
Technical Field
The invention belongs to the field of vehicle auxiliary driving, and particularly relates to a lane line vanishing point compensation method and device based on parallel constraint.
Background
In Advanced Driving Assistance System (ADAS) applications of automobiles, a vehicle-mounted camera cannot be dynamically compensated due to up-and-down shake of a camera vanishing point caused by the camera itself or road reasons during curve Driving. The method is characterized in that the method is simple, but the method is not strong in universality, a special calibration plate needs to be manufactured, calibration data after calibration cannot be compensated according to the vibration of a vehicle, and the difference between real parameters of the camera and a calibration result is large due to the vibration of the vehicle in the driving process of the vehicle; the method for calibrating the parallel straight lines is characterized in that a calibration plate is not needed, camera parameters can be dynamically compensated, but the method is only limited to the straight line lane lines, and the camera parameters cannot be compensated for a curve scene.
Disclosure of Invention
Aiming at the defects or improvement requirements in the prior art, the invention provides a lane line vanishing point compensation method and device based on parallel constraint, so that the technical problem that the existing static calibration or parallel straight line calibration has limitation in curve scene compensation is solved.
To achieve the above object, according to an aspect of the present invention, there is provided a lane line vanishing point compensating method based on parallel constraint, including:
(1) acquiring a road surface image acquired by a vehicle-mounted camera, identifying a left lane line and a right lane line of the road surface image in an image coordinate system, and respectively selecting a plurality of lane line characteristic points on each lane line;
(2) respectively solving the coordinate position of each lane line characteristic point selected on each lane line in a world coordinate system by using the initial position of the vanishing point;
(3) if the world coordinates of the lane line characteristic points on the left lane line and the world coordinates of the lane line characteristic points on the right lane line do not have the parallel characteristic, compensating the positions of the vanishing points, respectively obtaining the coordinate positions of the lane line characteristic points selected on each lane line in a world coordinate system according to the compensated positions of the vanishing points until the world coordinates of the lane line characteristic points on the left lane line and the world coordinates of the lane line characteristic points on the right lane line have the parallel characteristic, and taking the positions of the vanishing points which are finally compensated as the final positions of the vanishing points.
Preferably, step (3) comprises:
and performing double-line constraint fitting on each lane line characteristic point in the world coordinate system on the left lane line and the right lane line respectively, if the fitting residual is greater than a preset threshold, compensating the position of the vanishing point, respectively obtaining the coordinate position of each lane line characteristic point selected on each lane line in the world coordinate system according to the compensated vanishing point position, and then performing double-line constraint fitting on each lane line characteristic point in the world coordinate system on each lane line until the fitting residual is not greater than the preset threshold, thereby obtaining the vanishing point position after the final compensation.
Preferably, after the double-line constraint fitting is carried out, if the position of the vanishing point is larger, a curve obtained by the constraint fitting is crossed with a world coordinate point of the corresponding lane line characteristic point, and the near end of the curve surrounds the world coordinate point; and if the position of the vanishing point is smaller, the curve obtained by constraint fitting is crossed with the world coordinate point of the corresponding lane line characteristic point, and the near end surrounds the fitted curve by the world coordinate point, wherein the near end represents the position where the curve obtained by constraint fitting is crossed with the world coordinate point.
Preferably, is composed of
Obtaining fitting Residual errors Residual, wherein right _ num is the number of the lane line characteristic points on the near-end right lane line, R _ x (i), R _ y (i) are respectively the horizontal and vertical coordinates of the ith lane line characteristic point of the right lane line, and L _ R (.) is a constraint fitting equation of the right lane line; left _ num is the number of the lane line characteristic points on the near-end left lane line, L _ x (j), L _ y (j) are respectively the horizontal and vertical coordinates of the jth characteristic point of the left lane line, and L _ L (.) is the constraint fitting equation of the left lane line.
Preferably, is composed of
Wherein Residual is greater than 0;
vanising _ Y is the Vanishing point Y-direction coordinate, Min () is the minimum value, and n is a preset value.
According to another aspect of the present invention, there is provided a lane line vanishing point compensating device based on parallel constraint, including:
the lane line detection module is used for acquiring a road image acquired by a vehicle-mounted camera, identifying a left lane line and a right lane line of the road image in an image coordinate system, and respectively selecting a plurality of lane line characteristic points on each lane line;
the coordinate conversion module is used for respectively solving the coordinate positions of the characteristic points of each lane line selected on each lane line in the world coordinate system by utilizing the initial positions of the vanishing points;
and the vanishing point compensating module is used for compensating the position of the vanishing point when the world coordinates of the characteristic points of the lane lines on the left lane line and the world coordinates of the characteristic points of the lane lines on the right lane line are not parallel, respectively calculating the coordinate position of each selected characteristic point of the lane lines on each lane line in a world coordinate system according to the compensated vanishing point position until the world coordinates of the characteristic points of the lane lines on the left lane line and the world coordinates of the characteristic points of the lane lines on the right lane line are parallel, and taking the position of the vanishing point finally compensated as the final position of the vanishing point.
Preferably, the vanishing point compensating module is specifically configured to perform double-line constraint fitting on feature points of lane lines in the world coordinate systems on the left lane line and the right lane line, compensate the vanishing point position if the fitting residual is greater than a preset threshold, respectively obtain coordinate positions of the feature points of the lane lines selected on each lane line in the world coordinate system according to the compensated vanishing point position, and then perform double-line constraint fitting on the feature points of the lane lines in the world coordinate system on each lane line until the fitting residual is not greater than the preset threshold, so as to obtain the vanishing point position after the final compensation.
Preferably, after the double-line constraint fitting is carried out, if the position of the vanishing point is larger, a curve obtained by the constraint fitting is crossed with a world coordinate point of the corresponding lane line characteristic point, and the near end of the curve surrounds the world coordinate point; and if the position of the vanishing point is smaller, the curve obtained by constraint fitting is crossed with the world coordinate point of the corresponding lane line characteristic point, and the near end surrounds the fitted curve by the world coordinate point, wherein the near end represents the position where the curve obtained by constraint fitting is crossed with the world coordinate point.
Preferably, is prepared from
Obtaining fitting Residual errors Residual, wherein right _ num is the number of the lane line characteristic points on the near-end right lane line, R _ x (i), R _ y (i) are respectively the horizontal and vertical coordinates of the ith lane line characteristic point of the right lane line, and L _ R (.) is a constraint fitting equation of the right lane line; left _ num is the number of the lane line characteristic points on the near-end left lane line, L _ x (j), L _ y (j) are respectively the horizontal and vertical coordinates of the jth characteristic point of the left lane line, and L _ L (.) is the constraint fitting equation of the left lane line.
Preferably, is prepared from
Wherein Residual is greater than 0;
vanising _ Y is the Vanishing point Y-direction coordinate, Min () is the minimum value, and n is a preset value.
In general, compared with the prior art, the above technical solution contemplated by the present invention can achieve the following beneficial effects:
compared with the parallel straight line calibration, the method has stronger universality and meets more scenes. The method not only satisfies the estimation of the vanishing point under the parallel straight line scene, but also solves the problem that the vanishing point Y direction cannot be estimated due to the vanishing point change caused by the shake of the vehicle when the parallel straight line calibration vehicle runs to the curve scene, improves the interconversion precision of the image coordinate and the world coordinate, and particularly the vanishing point compensation precision directly influences the accuracy and the effectiveness of the detection distance of the vehicle, the pedestrian and other target distances and the lane line equation.
Drawings
Fig. 1 is a schematic flowchart of a lane line vanishing point compensation method based on parallel constraint according to an embodiment of the present invention;
fig. 2 is a schematic flowchart of another lane line vanishing point compensation method based on parallel constraint according to an embodiment of the present invention;
FIG. 3 is a schematic diagram illustrating the effect of a two-line constraint fitting when the vanishing point Y-direction value is large according to an embodiment of the present invention;
FIG. 4 is a diagram illustrating the effect of a two-line constraint fit on the vanishing point Y direction values when the values are biased;
fig. 5 is a schematic diagram of the compensated effect provided by the embodiment of the present invention, in which (a) is an initial vanishing point, (b) is iteration 1, (c) is iteration 2, (d) is iteration 3, (e) is iteration 4, and (f) is iteration 5;
fig. 6 is a schematic diagram of a compensation effect after 0.15m random noise is added, where (a) is an initial vanishing point, (b) is iteration 1, (c) is iteration 2, (d) is iteration 3, (e) is iteration 4, and (f) is iteration 5;
fig. 7 is a schematic structural diagram of a lane line vanishing point compensation device based on parallel constraint according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In addition, the technical features involved in the embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.
The invention dynamically compensates the Y direction of the vanishing point by combining the parallel characteristic of the lane line and the double-line constraint fitting, and provides an accurate vanishing point position for the camera in an environment with two parallel (straight lines or curved) lane lines; the extended function can be used for coordinate conversion of targets such as vehicles, pedestrians and the like, the target distance is obtained, the image coordinate lane line is converted into a world lane line equation, a more accurate world coordinate system lane line equation is provided, and effective support is provided for follow-up LDW and LKA.
The method comprises the steps of collecting road surface images by using a vehicle-mounted camera, detecting a lane line by using a lane line detection algorithm, extracting lane line characteristic points, converting the characteristic points by combining camera internal parameters, height and vanishing point positions, according to the mutually parallel characteristics of the lane lines in the real world, when the vanishing point positions are not right, projecting and solving the world coordinate characteristic point positions to present the characteristics of 'inner eight' or 'outer eight', and re-projecting the characteristic points by adjusting the vanishing point positions until the projected characteristic points present the parallel characteristics, thereby obtaining the correct vanishing point coordinate positions.
Fig. 1 is a schematic flow chart of a lane line vanishing point compensation method based on parallel constraint according to an embodiment of the present invention, including the following steps:
s1: acquiring a road surface image acquired by a vehicle-mounted camera, identifying a left lane line and a right lane line of the road surface image in an image coordinate system, and respectively selecting a plurality of lane line characteristic points on each lane line;
s2: respectively solving the coordinate position of each lane line characteristic point selected on each lane line in a world coordinate system by using the initial position of the vanishing point;
specifically, the coordinate conversion can be performed on the lane line characteristic points by combining the internal reference of the vehicle-mounted camera, the height of the vehicle-mounted camera and the initial position of the vanishing point.
S3: if the world coordinates of the lane line characteristic points on the left lane line and the world coordinates of the lane line characteristic points on the right lane line do not have the parallel characteristic, compensating the positions of the vanishing points, respectively obtaining the coordinate positions of the lane line characteristic points selected on each lane line in a world coordinate system according to the compensated positions of the vanishing points until the world coordinates of the lane line characteristic points on the left lane line and the world coordinates of the lane line characteristic points on the right lane line have the parallel characteristic, and taking the positions of the vanishing points which are finally compensated as the final positions of the vanishing points.
Fig. 2 is a schematic flow chart of another lane line vanishing point compensation method based on parallel constraint according to an embodiment of the present invention, and in fig. 2, step (3) can be implemented by:
and performing double-line constraint fitting on each lane line characteristic point in the world coordinate system on the left lane line and the right lane line respectively, if the fitting residual is greater than a preset threshold, compensating the position of the vanishing point, respectively obtaining the coordinate position of each lane line characteristic point selected on each lane line in the world coordinate system according to the compensated vanishing point position, and then performing double-line constraint fitting on each lane line characteristic point in the world coordinate system on each lane line until the fitting residual is not greater than the preset threshold, thereby obtaining the vanishing point position after the final compensation.
In the embodiment of the invention, the converted world coordinate feature points are subjected to double-line constraint fitting, and the fitting effect is shown in fig. 3 and 4;
in the figure, "o" represents the position of a characteristic point of a lane line in a real world coordinate system, "+" represents the world coordinate position obtained by image coordinate conversion when a vanishing point is too large, and a black curve is a curve obtained by a "+" point double-line constraint fitting. It can be seen from fig. 3 that when the vanishing point is large, the transformed feature points exhibit the characteristic of "eight out", and the black curve obtained by constraint fitting thereof intersects with the "+" point transition point, and the near end surrounds the "+" point transition point; conversely, as shown in fig. 4, the characteristic of "inner eight" is exhibited, the proximal end being surrounded by a "dot" to a black curve. And subsequently, compensating the camera vanishing point according to the attribute to enable the camera vanishing point to approach to the true value of the vanishing point, and finally solving the position of the vanishing point, wherein the near end is the position where the black curve is intersected with the dot after the two lines are constrained.
In the embodiment of the present invention, the fitting Residual can be calculated by formula (1);
in the formula (1), Residual is a fitting Residual, right _ num is the number of characteristic points of a near-end right lane line, R _ x (i), R _ y (i) are horizontal and vertical coordinates of the ith characteristic point of the right lane line, and L _ R (.) is a right lane line constraint fitting equation; left _ num is the number of characteristic points of the near-end left lane line, L _ x (j), L _ y (j) are the horizontal and vertical coordinates of the jth characteristic point of the left lane line, and L _ L (.) is a constraint fitting equation of the left lane line.
The number of the characteristic points of the near-end right lane line can be determined in the following mode:
entering crossover is indicated when the magnitude of R _ x (i) -L _ R (R _ y (i)) changes from positive to negative or from negative to positive, as follows: (R _ x (i) -L _ R (R _ y (i)) <0, then the size of i is the number right _ num of near-end feature points.
The number of the near-end left lane line characteristic points can be determined in the following way:
entering crossover is indicated when the magnitude of L _ x (j) -L _ L (L _ y (j)) changes from positive to negative, or from negative to positive, as follows: (L _ x (j) -L _ L (L _ y (j)) x (L _ x +1) -L _ L (L _ y (j +1)) <0, the size of j is the number left _ num of near-end feature points.
In the embodiment of the present invention, as can be seen from fig. 3 and 4, when the vanishing point is larger, Residual is positive, and vice versa, so that the compensation direction of the vanishing point can be determined by the positive and negative of Residual, and the compensation size is determined by the value size of Residual, and when the fitting Residual is smaller than the set threshold, it indicates that the Y coordinate of the vanishing point is close to the true value, so that the value of the preset threshold can be 0.
Specifically, the vanishing point location can be iteratively optimized by equation (2):
if Residual>0, then
If Residual is less than or equal to 0, then
In the embodiment of the present invention, 10 is preferred, where the initial position of the Vanishing point is a result of static calibration when the vehicle-mounted camera is installed, and is default to the Y coordinate of the image center.
Fig. 5 is a schematic diagram of the compensated effect provided by the embodiment of the present invention, in which (a) is an initial vanishing point, (b) is iteration 1, (c) is iteration 2, (d) is iteration 3, (e) is iteration 4, and (f) is iteration 5;
fig. 6 is a schematic diagram of a compensation effect after 0.15m random noise is added, where (a) is an initial vanishing point, (b) is iteration 1, (c) is iteration 2, (d) is iteration 3, (e) is iteration 4, and (f) is iteration 5.
Fig. 7 is a schematic structural diagram of a lane line vanishing point compensating device based on parallel constraint according to an embodiment of the present invention, including:
the lane line detection module 201 is configured to acquire a road image acquired by a vehicle-mounted camera, identify two lane lines on the left and right of the road image in an image coordinate system, and select a plurality of lane line feature points on each lane line;
the coordinate conversion module 202 is configured to use the vanishing point initial position to respectively obtain a coordinate position of each lane line feature point selected on each lane line in the world coordinate system;
the vanishing point compensating module 203 is configured to compensate the vanishing point position when the world coordinates of the respective lane line feature points on the left lane line and the world coordinates of the respective lane line feature points on the right lane line do not have a parallel characteristic, and respectively obtain the coordinate position of the respective lane line feature points selected on each lane line in the world coordinate system according to the compensated vanishing point position until the world coordinates of the respective lane line feature points on the left lane line and the world coordinates of the respective lane line feature points on the right lane line have a parallel characteristic, and use the finally compensated vanishing point position as the final position of the vanishing point.
In this embodiment of the present invention, the vanishing point compensating module 203 is specifically configured to perform a double-line constraint fitting on the feature points of each lane line in the world coordinate system on the left lane line and the right lane line, compensate the vanishing point position if the fitting residual is greater than a preset threshold, obtain the coordinate position of each lane line feature point in the world coordinate system selected on each lane line according to the compensated vanishing point position, and perform the double-line constraint fitting on each lane line feature point in the world coordinate system on each lane line until the fitting residual is not greater than the preset threshold, so as to obtain the vanishing point position after the final compensation.
In the embodiment of the invention, after the double-line constraint fitting is carried out, if the position of the vanishing point is larger, the curve obtained by the constraint fitting is crossed with the world coordinate point of the corresponding lane line characteristic point, and the near end of the curve surrounds the world coordinate point; and if the position of the vanishing point is smaller, the curve obtained by constraint fitting is crossed with the world coordinate point of the corresponding lane line characteristic point, and the near end surrounds the fitted curve by the world coordinate point, wherein the near end represents the position where the curve obtained by constraint fitting is crossed with the world coordinate point.
In the embodiment of the invention, the
Obtaining fitting Residual errors Residual, wherein right _ num is the number of the lane line characteristic points on the near-end right lane line, R _ x (i), R _ y (i) are respectively the horizontal and vertical coordinates of the ith lane line characteristic point of the right lane line, and L _ R (.) is a constraint fitting equation of the right lane line; left _ num is nearThe number of the lane line characteristic points on the left lane line, L _ x (j), L _ y (j) are respectively the horizontal coordinate and the vertical coordinate of the jth characteristic point of the left lane line, and L _ L (.) is a constraint fitting equation of the left lane line.
In the embodiment of the invention, the
Wherein Residual is greater than 0;
vanising _ Y is the Vanishing point Y-direction coordinate, Min () is the minimum value, and n is a preset value.
The invention provides a compensation method for a parallel road vanishing point in the Y direction by combining the inherent model characteristic of a lane line and an optical camera imaging model. Acquiring a frame of image information through a camera, detecting and identifying lane lines in an image coordinate system by using the lane lines, selecting a plurality of lane line coordinate points on each lane line, and calculating the coordinate positions of the lane line coordinate points on a world coordinate system by combining the height of the camera, the internal parameters and the initial camera vanishing point position. According to the inherent parallel characteristic of the lane lines, when the camera vanishing point is estimated correctly, the world coordinates of the coordinate points of the lane lines are also characterized in parallel, and the Y direction of the camera vanishing point is iteratively supplemented by an iteration method according to the characteristic to obtain an accurate Y direction numerical value of the vanishing point. According to the method, the compensation result of the vanishing point is compared with the real position of the vanishing point in a multi-curve simulation scene, and the difference between the Y coordinate of the vanishing point after compensation and a true value is not more than 1.5 pixels.
It will be understood by those skilled in the art that the foregoing is only a preferred embodiment of the present invention, and is not intended to limit the invention, and that any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (10)
1. A lane line vanishing point compensation method based on parallel constraint is characterized by comprising the following steps:
(1) acquiring a road surface image acquired by a vehicle-mounted camera, identifying a left lane line and a right lane line of the road surface image in an image coordinate system, and respectively selecting a plurality of lane line characteristic points on each lane line;
(2) respectively obtaining the coordinate position of each lane line characteristic point selected on each lane line in a world coordinate system by using the initial position of the vanishing point;
(3) if the world coordinates of the characteristic points of the lane lines on the left lane line and the world coordinates of the characteristic points of the lane lines on the right lane line do not have the parallel characteristic, compensating the positions of the vanishing points, respectively obtaining the coordinate positions of the characteristic points of the lane lines selected on each lane line in a world coordinate system according to the compensated positions of the vanishing points until the world coordinates of the characteristic points of the lane lines on the left lane line and the world coordinates of the characteristic points of the lane lines on the right lane line have the parallel characteristic, and taking the positions of the vanishing points compensated finally as the final positions of the vanishing points.
2. The method of claim 1, wherein step (3) comprises:
and performing double-line constraint fitting on each lane line characteristic point in the world coordinate system on the left lane line and the right lane line respectively, if the fitting residual is greater than a preset threshold, compensating the position of the vanishing point, respectively obtaining the coordinate position of each lane line characteristic point selected on each lane line in the world coordinate system according to the compensated vanishing point position, and then performing double-line constraint fitting on each lane line characteristic point in the world coordinate system on each lane line until the fitting residual is not greater than the preset threshold, thereby obtaining the vanishing point position after the final compensation.
3. The method of claim 2, wherein after the two-line constraint fitting, if the vanishing point is too large in position, the curve obtained by the constraint fitting intersects with the world coordinate point of the corresponding lane line characteristic point and the near end surrounds the world coordinate point; and if the position of the vanishing point is smaller, the curve obtained by constraint fitting is intersected with the world coordinate point of the corresponding lane line characteristic point, and the fitted curve is surrounded by the world coordinate point at the near end, wherein the near end represents the position where the curve obtained by constraint fitting is intersected with the world coordinate point.
4. The method of claim 3, wherein the method is performed by
Obtaining fitting Residual errors Residual, wherein right _ num is the number of the lane line characteristic points on the near-end right lane line, R _ x (i), R _ y (i) are respectively the horizontal and vertical coordinates of the ith lane line characteristic point of the right lane line, and L _ R (.) is a constraint fitting equation of the right lane line; left _ num is the number of the lane line characteristic points on the near-end left lane line, L _ x (j), L _ y (j) are respectively the horizontal and vertical coordinates of the jth characteristic point of the left lane line, and L _ L (.) is the constraint fitting equation of the left lane line.
5. The method of claim 4, wherein the method is performed by
Wherein Residual is greater than 0;
vanising _ Y is the Vanishing point Y-direction coordinate, Min () is the minimum value, and n is a preset value.
6. The utility model provides a lane line vanishing point compensation device based on parallel constraint which characterized in that includes:
the lane line detection module is used for acquiring a road image acquired by a vehicle-mounted camera, identifying a left lane line and a right lane line of the road image in an image coordinate system, and respectively selecting a plurality of lane line characteristic points on each lane line;
the coordinate conversion module is used for respectively solving the coordinate positions of the characteristic points of each lane line selected on each lane line in the world coordinate system by utilizing the initial positions of the vanishing points;
and the vanishing point compensating module is used for compensating the position of the vanishing point when the world coordinates of the characteristic points of the lane lines on the left lane line and the world coordinates of the characteristic points of the lane lines on the right lane line are not parallel, respectively calculating the coordinate position of each selected characteristic point of the lane lines on each lane line in a world coordinate system according to the compensated vanishing point position until the world coordinates of the characteristic points of the lane lines on the left lane line and the world coordinates of the characteristic points of the lane lines on the right lane line are parallel, and taking the position of the vanishing point finally compensated as the final position of the vanishing point.
7. The apparatus according to claim 6, wherein the vanishing point compensating module is specifically configured to perform a two-line constraint fitting on feature points of each lane line in the world coordinate system on the left lane line and the right lane line, respectively, compensate the vanishing point position if a fitting residual is greater than a preset threshold, respectively obtain coordinate positions of the feature points of each lane line selected on each lane line in the world coordinate system according to the compensated vanishing point position, and then perform the two-line constraint fitting on the feature points of each lane line in the world coordinate system on each lane line until the fitting residual is not greater than the preset threshold, so as to obtain the vanishing point position after the final compensation.
8. The apparatus of claim 7, wherein after the two-line constraint fitting, if the vanishing point is located too far away, the curve obtained by the constraint fitting intersects with the world coordinate point of the corresponding lane line feature point and the near end surrounds the world coordinate point; and if the position of the vanishing point is smaller, the curve obtained by constraint fitting is crossed with the world coordinate point of the corresponding lane line characteristic point, and the near end surrounds the fitted curve by the world coordinate point, wherein the near end represents the position where the curve obtained by constraint fitting is crossed with the world coordinate point.
9. The device of claim 8, wherein the device is made of
Obtaining fitting Residual errors Residual, wherein right _ num is the number of the lane line characteristic points on the near-end right lane line, R _ x (i), R _ y (i) are respectively the horizontal and vertical coordinates of the ith lane line characteristic point of the right lane line, and L _ R (.) is a constraint fitting equation of the right lane line; left _ num is the number of the lane line characteristic points on the near-end left lane line, L _ x (j), L _ y (j) are respectively the horizontal and vertical coordinates of the jth characteristic point of the left lane line, and L _ L (.) is the constraint fitting equation of the left lane line.
10. The device of claim 9, wherein the device is made of
Wherein Residual is greater than 0;
vanising _ Y is the Vanishing point Y-direction coordinate, Min () is the minimum value, and n is a preset value.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011168364.2A CN112348752B (en) | 2020-10-28 | 2020-10-28 | Lane line vanishing point compensation method and device based on parallel constraint |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011168364.2A CN112348752B (en) | 2020-10-28 | 2020-10-28 | Lane line vanishing point compensation method and device based on parallel constraint |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112348752A CN112348752A (en) | 2021-02-09 |
| CN112348752B true CN112348752B (en) | 2022-08-16 |
Family
ID=74359247
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011168364.2A Active CN112348752B (en) | 2020-10-28 | 2020-10-28 | Lane line vanishing point compensation method and device based on parallel constraint |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112348752B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113375687B (en) * | 2021-05-12 | 2023-06-02 | 武汉极目智能技术有限公司 | Method, system and device for lane line vanishing point compensation based on parallel constraint |
| CN114120258B (en) * | 2022-01-26 | 2022-05-03 | 深圳佑驾创新科技有限公司 | Lane line identification method and device and storage medium |
| CN117315048B (en) * | 2023-11-22 | 2024-04-12 | 深圳元戎启行科技有限公司 | External parameter self-calibration method of vehicle-mounted camera, electronic equipment and storage medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103927754A (en) * | 2014-04-21 | 2014-07-16 | 大连理工大学 | Calibration method of on-board camera |
| JP2017010553A (en) * | 2015-06-24 | 2017-01-12 | 株式会社リコー | Detection method and detection device for road boundary body |
| CN106875448A (en) * | 2017-02-16 | 2017-06-20 | 武汉极目智能技术有限公司 | A kind of vehicle-mounted monocular camera external parameter self-calibrating method |
| CN107787496A (en) * | 2015-05-26 | 2018-03-09 | Plk科技株式会社 | Disappearance point-correcting and method |
| CN108898638A (en) * | 2018-06-27 | 2018-11-27 | 江苏大学 | A kind of on-line automatic scaling method of vehicle-mounted camera |
| CN109255316A (en) * | 2018-08-30 | 2019-01-22 | 深圳市路畅科技股份有限公司 | A kind of lane shift detection method and system |
| CN110264525A (en) * | 2019-06-13 | 2019-09-20 | 惠州市德赛西威智能交通技术研究院有限公司 | A kind of camera calibration method based on lane line and target vehicle |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8311283B2 (en) * | 2008-07-06 | 2012-11-13 | Automotive Research&Testing Center | Method for detecting lane departure and apparatus thereof |
-
2020
- 2020-10-28 CN CN202011168364.2A patent/CN112348752B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103927754A (en) * | 2014-04-21 | 2014-07-16 | 大连理工大学 | Calibration method of on-board camera |
| CN107787496A (en) * | 2015-05-26 | 2018-03-09 | Plk科技株式会社 | Disappearance point-correcting and method |
| JP2017010553A (en) * | 2015-06-24 | 2017-01-12 | 株式会社リコー | Detection method and detection device for road boundary body |
| CN106875448A (en) * | 2017-02-16 | 2017-06-20 | 武汉极目智能技术有限公司 | A kind of vehicle-mounted monocular camera external parameter self-calibrating method |
| CN108898638A (en) * | 2018-06-27 | 2018-11-27 | 江苏大学 | A kind of on-line automatic scaling method of vehicle-mounted camera |
| CN109255316A (en) * | 2018-08-30 | 2019-01-22 | 深圳市路畅科技股份有限公司 | A kind of lane shift detection method and system |
| CN110264525A (en) * | 2019-06-13 | 2019-09-20 | 惠州市德赛西威智能交通技术研究院有限公司 | A kind of camera calibration method based on lane line and target vehicle |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112348752A (en) | 2021-02-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112348752B (en) | Lane line vanishing point compensation method and device based on parallel constraint | |
| CN108805934B (en) | External parameter calibration method and device for vehicle-mounted camera | |
| CN107392103B (en) | Method and device for detecting road lane line and electronic equipment | |
| KR101143176B1 (en) | Method and Apparatus for Recognizing Parking Slot Marking by Using Bird's Eye View and Parking Assist System Using Same | |
| US11205284B2 (en) | Vehicle-mounted camera pose estimation method, apparatus, and system, and electronic device | |
| JP6473571B2 (en) | TTC measuring device and TTC measuring program | |
| CN110766760B (en) | Method, device, equipment and storage medium for camera calibration | |
| EP3676796A1 (en) | Systems and methods for correcting a high-definition map based on detection of obstructing objects | |
| CN111141311B (en) | Evaluation method and system of high-precision map positioning module | |
| CN110728720B (en) | Method, apparatus, device and storage medium for camera calibration | |
| CN110766761B (en) | Method, apparatus, device and storage medium for camera calibration | |
| CN110751693B (en) | Method, apparatus, device and storage medium for camera calibration | |
| CN111260539B (en) | Fish eye pattern target identification method and system thereof | |
| JP2009085651A (en) | Image processing system | |
| CN106651963A (en) | Mounting parameter calibration method for vehicular camera of driving assistant system | |
| CN113740871B (en) | Laser SLAM method, system equipment and storage medium under high dynamic environment | |
| CN105809669A (en) | Method and apparatus of calibrating an image detecting device | |
| CN111222417A (en) | Method and device for improving lane line extraction precision based on vehicle-mounted image | |
| JP2017181476A (en) | Vehicle location detection device, vehicle location detection method and vehicle location detection-purpose computer program | |
| CN114419143A (en) | Depth measuring method, depth measuring apparatus, and storage medium | |
| CN106651903B (en) | A kind of Mobile object detection method | |
| CN113029185A (en) | Road marking change detection method and system in crowdsourcing type high-precision map updating | |
| EP3288260B1 (en) | Image processing device, imaging device, equipment control system, equipment, image processing method, and carrier means | |
| US11477371B2 (en) | Partial image generating device, storage medium storing computer program for partial image generation and partial image generating method | |
| CN115236643A (en) | Sensor calibration method, system, device, electronic equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |