WO2024098953A1 - 车道线的拼接方法、装置、电子设备和存储介质 - Google Patents

车道线的拼接方法、装置、电子设备和存储介质 Download PDF

Info

Publication number
WO2024098953A1
WO2024098953A1 PCT/CN2023/118652 CN2023118652W WO2024098953A1 WO 2024098953 A1 WO2024098953 A1 WO 2024098953A1 CN 2023118652 W CN2023118652 W CN 2023118652W WO 2024098953 A1 WO2024098953 A1 WO 2024098953A1
Authority
WO
WIPO (PCT)
Prior art keywords
lane line
curve
constraint
splicing
target
Prior art date
Application number
PCT/CN2023/118652
Other languages
English (en)
French (fr)
Inventor
蒋晓路
Original Assignee
南京地平线集成电路有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 南京地平线集成电路有限公司 filed Critical 南京地平线集成电路有限公司
Publication of WO2024098953A1 publication Critical patent/WO2024098953A1/zh

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle

Definitions

  • the present disclosure relates to assisted driving technology, and in particular to a lane line splicing method, device, electronic device and storage medium.
  • lane lines are fitted based on the environmental images of each perspective.
  • the lane lines corresponding to each environmental image are then transformed into a unified bird's-eye view (Bird's Eye View, abbreviated as: BEV) coordinate system to obtain the lane lines in the bird's-eye view coordinate system.
  • BEV bird's Eye View
  • the lane lines in the bird's-eye view coordinate system corresponding to each perspective are fused to form a surrounding lane line bird's-eye view for downstream use.
  • Embodiments of the present disclosure provide a lane line splicing method, device, electronic device, and storage medium.
  • a lane line splicing method including: determining a first lane line curve and a second lane line curve to be spliced in a first coordinate system; determining a splicing constraint point based on the first lane line curve and the second lane line curve; adjusting a first curve parameter corresponding to the first lane line curve based on the splicing constraint point to obtain an adjusted first target curve parameter corresponding to the first curve parameter; adjusting a second curve parameter corresponding to the second lane line curve based on the splicing constraint point to obtain an adjusted second target curve parameter corresponding to the second curve parameter; and determining a spliced lane line based on the first target curve parameter and the second target curve parameter.
  • a lane line splicing device including: a first determination module, used to determine a first lane line curve and a second lane line curve to be spliced in a first coordinate system; a first processing module, used to determine a splicing constraint point based on the first lane line curve and the second lane line curve; a second processing module, used to adjust a first curve parameter corresponding to the first lane line curve based on the splicing constraint point, and obtain an adjusted first target curve parameter corresponding to the first curve parameter; a third processing module, used to adjust a second curve parameter corresponding to the second lane line curve based on the splicing constraint point, and obtain an adjusted second target curve parameter corresponding to the second curve parameter; and a fourth processing module, used to determine the spliced lane line based on the first target curve parameter and the second target curve parameter.
  • a computer-readable storage medium stores a computer program, and the computer program is used to execute the lane line splicing method described in any of the above embodiments of the present disclosure.
  • an electronic device comprising: a processor; a memory for storing executable instructions of the processor; the processor is configured to read the executable instructions from the memory and execute the instructions to implement the lane line splicing method described in any of the above embodiments of the present disclosure.
  • splicing constraint points are determined based on the two lane line curves to be spliced, and the curve parameters of the two lane line curves are adjusted based on the splicing constraint points, so that the adjusted two lane lines can be spliced smoothly and continuously, which helps to improve the continuity and smoothness of the lane lines after fusion.
  • FIG1 is an exemplary application scenario of the lane line splicing method provided by the present disclosure
  • FIG2 is a schematic flow chart of a lane line splicing method provided by an exemplary embodiment of the present disclosure
  • FIG3 is a schematic flow chart of a lane line splicing method provided by another exemplary embodiment of the present disclosure.
  • FIG4 is a flow chart of step 2031 provided by an exemplary embodiment of the present disclosure.
  • FIG5 is a flow chart of step 2041 provided by an exemplary embodiment of the present disclosure.
  • FIG6 is a schematic diagram of a state quantity update iteration process provided by an exemplary embodiment of the present disclosure.
  • FIG7 is a schematic diagram of a flow chart of determining a constraint amount provided by an exemplary embodiment of the present disclosure
  • FIG8 is a schematic diagram of a situation where a first lane line curve and a second lane line curve overlap in a longitudinal direction provided by an exemplary embodiment of the present disclosure
  • FIG9 is a schematic diagram of a longitudinal gap between a first lane line curve and a second lane line curve provided by an exemplary embodiment of the present disclosure
  • FIG10 is a flow chart of step 2021 provided by an exemplary embodiment of the present disclosure.
  • FIG11 is a schematic structural diagram of a lane line splicing device provided by an exemplary embodiment of the present disclosure.
  • FIG12 is a schematic structural diagram of a lane line splicing device provided by another exemplary embodiment of the present disclosure.
  • FIG13 is a schematic diagram of the structure of a first processing unit 5031 provided by an exemplary embodiment of the present disclosure.
  • FIG. 14 is a schematic diagram of the structure of an application embodiment of the electronic device disclosed herein.
  • the inventors found that in an assisted driving scenario, environmental images are collected by cameras with multiple perspectives, and lane lines are fitted based on the environmental images of each perspective, and then the lane lines corresponding to each environmental image are transformed into a unified bird's-eye view (BEV) coordinate system to obtain the lane lines in the bird's-eye view coordinate system, and then the lane lines in the bird's-eye view coordinate system corresponding to each perspective are fused to form a surrounding lane line bird's-eye view for downstream use.
  • the fused surrounding lane line bird's-eye view may have poor continuity and smoothness in the fused part of two lane line segments belonging to the same lane line.
  • FIG. 1 is an exemplary application scenario of the lane line splicing method provided by the present disclosure.
  • environmental images can be collected by cameras of multiple perspectives, and lane lines can be fitted based on the environmental images of each perspective, and then the lane lines corresponding to each environmental image can be transformed into a unified bird's-eye view (BEV) coordinate system to obtain the lane lines in the bird's-eye view coordinate system corresponding to each perspective, and then the lane lines in the bird's-eye view coordinate system corresponding to each perspective can be fused to form a surrounding lane line bird's-eye view.
  • BEV bird's-eye view
  • a first lane line curve and a second lane line curve to be spliced can be determined based on a certain matching rule.
  • the first lane line curve and the second lane line curve are different lane line segments of the same lane line corresponding to the two perspectives, such as the first lane line segment of the leftmost lane line from the perspective of the left front camera and the second lane line segment of the leftmost lane line from the perspective of the left rear camera.
  • the first lane line segment and the second lane segment are collected by the left front camera and the left rear camera respectively, they are Road line detection, fitting and coordinate conversion are performed to obtain the first lane line curve and the second lane line curve corresponding to the bird's-eye view coordinate system respectively.
  • the splicing constraint point can be determined based on the first lane line curve and the second lane line curve.
  • the first curve parameter corresponding to the first lane line curve is adjusted to obtain the adjusted first target curve parameter corresponding to the first curve parameter
  • the second curve parameter corresponding to the second lane line curve is adjusted to obtain the adjusted second target curve parameter corresponding to the second curve parameter, so that the first target curve corresponding to the adjusted first target curve parameter and the second target curve corresponding to the second target curve parameter can be smoothly and continuously spliced, so that based on the first target curve parameter and the second target curve parameter, the spliced continuous and smooth lane line can be determined, which is helpful to improve the continuity and smoothness of the lane line after the fusion of each perspective.
  • lane lines in the bird's-eye view coordinate system of each perspective are merged, multiple groups of lane lines to be spliced may be included, or lane line curves in the bird's-eye view coordinate system corresponding to multiple different line segments of the same lane line need to be spliced together in pairs.
  • Any two lane line curves to be spliced can be respectively used as the first lane line curve and the second lane line curve, and spliced using the above method, which will not be described in detail one by one.
  • FIG2 is a flow chart of a lane line splicing method provided by an exemplary embodiment of the present disclosure. This embodiment can be applied to electronic devices, such as a vehicle-mounted computing platform, as shown in FIG2, including the following steps:
  • Step 201 determining a first lane line curve and a second lane line curve to be spliced in a first coordinate system.
  • the first coordinate system can be a coordinate system corresponding to the bird's-eye view (BEV) or a world coordinate system. It can be set according to actual needs as long as the lane line curves collected by cameras with different view angles can be unified into the same coordinate system.
  • the first lane line curve and the second lane line curve are curve segments, respectively, with a starting point and an end point.
  • xs1 and ys1 represent the horizontal coordinate and vertical coordinate of the starting point of the first lane line curve respectively.
  • Coordinates, x e1 and y e1 represent the abscissa and ordinate of the end point of the first lane line curve respectively
  • x s2 and y s2 represent the abscissa and ordinate of the starting point of the second lane line curve respectively
  • x e2 and y e2 represent the abscissa and ordinate of the end point of the second lane line curve respectively.
  • a preset matching rule can be used to match multiple lane lines in the first coordinate system in pairs, where the preset matching rule can be set according to the characteristics of the lane line. For example, in the bird's-eye view coordinate system, the center of the vehicle is the coordinate origin, the direction directly in front of the vehicle is the longitudinal axis (y-axis), and the direction directly to the right of the vehicle is the horizontal axis (x-axis). The lane lines of different perspectives extend in the longitudinal direction (y-direction) and have a small distance in the horizontal direction. The corresponding matching rule is set accordingly to determine the two lane line curves that need to be spliced.
  • step 201 may be executed by the processor calling a corresponding instruction stored in a memory, or may be executed by a first determination module executed by the processor.
  • Step 202 Determine a splicing constraint point based on the first lane line curve and the second lane line curve.
  • the splicing constraint point is used to constrain the adjustment process of the first lane line curve and the second lane line curve when adjusting them, so that the adjusted two lane line curves can be continuously and smoothly spliced at the splicing constraint point.
  • the splicing constraint point can be determined based on the starting point and the ending point of the first lane line curve and the second lane line curve. The starting point and the ending point can also be called endpoints.
  • the splicing end of the two lane line curves can be determined.
  • the first ending point of the first lane line curve and the second starting point of the second lane line curve need to be spliced.
  • the splicing constraint point is determined.
  • the first lane line curve and the second lane line curve may include two situations in the longitudinal direction: overlap (i.e., the longitudinal coordinate of the first end point is within the longitudinal coordinate range of the second lane line curve, and the longitudinal coordinate of the second starting point is within the longitudinal coordinate range of the first lane line curve) or gap (i.e., the longitudinal coordinate of the first end point is outside the longitudinal coordinate range of the second lane line curve, and the longitudinal coordinate of the second starting point is outside the longitudinal coordinate range of the first lane line curve).
  • the corresponding splicing constraint point is determined according to different situations. For example, for the gap situation, the center of the gap area can be determined as the splicing constraint point based on the first end point and the second starting point, which can be set according to actual needs.
  • step 202 may be executed by the processor calling corresponding instructions stored in the memory, or may be executed by the first processing module executed by the processor.
  • Step 203 Based on the splicing constraint points, first curve parameters corresponding to the first lane line curve are adjusted to obtain adjusted first target curve parameters corresponding to the first curve parameters.
  • the first curve parameter corresponding to the first lane line curve is a multi-curve coefficient used to represent the first lane line curve, such as a cubic curve coefficient.
  • the principle of adjusting the first curve parameter is to enable the first target curve corresponding to the adjusted first target curve parameter to be continuously and smoothly spliced with the adjusted second target curve at the splicing constraint point.
  • the first curve parameters may be adjusted by any feasible optimization method, such as a constrained extended Kalman filter algorithm, nonlinear optimization, etc., which may be specifically set according to actual needs.
  • any feasible optimization method such as a constrained extended Kalman filter algorithm, nonlinear optimization, etc., which may be specifically set according to actual needs.
  • step 203 may be executed by the processor calling a corresponding instruction stored in a memory, or may be executed by a second processing module executed by the processor.
  • Step 204 adjusting the second curve parameters corresponding to the second lane line curve based on the splicing constraint points to obtain adjusted second target curve parameters corresponding to the second curve parameters.
  • the second curve parameter corresponding to the second lane line curve is a multi-curve coefficient used to represent the second lane line curve, such as a cubic curve coefficient.
  • the principle of adjusting the second curve parameter is to enable the second target curve corresponding to the adjusted second target curve parameter to be continuously and smoothly spliced with the adjusted first target curve at the splicing constraint point.
  • the second lane line curve is a cubic curve, and the second curve parameters are C 20 , C 21 , C 22 , and C 23 .
  • step 203 and step 204 are performed in no particular order.
  • step 204 may be executed by the processor calling corresponding instructions stored in the memory, or may be executed by a third processing module executed by the processor.
  • Step 205 determining a spliced lane line based on the first target curve parameter and the second target curve parameter.
  • the first target curve parameter and the second target curve parameter are curve parameters adjusted based on the splicing constraint point.
  • the first target curve and the second target curve corresponding to the two can be continuously and smoothly spliced at the splicing constraint point to obtain the spliced lane line.
  • the lane line curves that need to be spliced in the first coordinate system corresponding to each perspective can be spliced, so as to obtain the lane line in the first coordinate system. Slide the continuous surround lane map.
  • the first coordinate system is a coordinate system corresponding to a bird's-eye view.
  • the spliced lane lines can be fused with other lane lines to obtain a surrounding lane line map from a bird's-eye view.
  • the surrounding lane line map from a bird's-eye view is a lane line map of a local area around the vehicle.
  • the surrounding lane line map from a bird's-eye view can also be converted to a global coordinate system (such as a world coordinate system), so that a global lane line map can be obtained over time.
  • the specific settings can be based on actual needs.
  • step 205 may be executed by the processor calling a corresponding instruction stored in the memory, or may be executed by a fourth processing module executed by the processor.
  • the lane line splicing method provided in this embodiment determines the splicing constraint points based on the two lane line curves to be spliced, and adjusts the curve parameters of the two lane line curves based on the splicing constraint points, so that the adjusted two lane lines can be spliced smoothly and continuously, effectively improving the continuity and smoothness of the lane lines after fusion.
  • FIG. 3 is a schematic flow chart of a lane line splicing method provided by another exemplary embodiment of the present disclosure.
  • step 203 may specifically include the following steps:
  • Step 2031 Based on the splicing constraint points, a constrained extended Kalman filter algorithm is used to adjust the first curve parameters to obtain the first target curve parameters.
  • the first curve parameter can be used as the initial state quantity, and the corresponding constraint quantity is set based on the splicing constraint point and the conditions that need to be met for the continuous and smooth splicing of the first lane line curve and the second lane line curve, and the state transfer is realized through iterative filtering, and the first curve parameter is updated to obtain the updated first target curve parameter, so that the updated first target curve parameter satisfies the continuous and smooth splicing constraint.
  • the constraint quantity may include the lateral coordinate constraint and the first-order partial derivative constraint at the splicing constraint point
  • the lateral coordinate constraint is used to make the first target curve corresponding to the first target curve parameter contain the splicing constraint point
  • the first-order partial derivative constraint is used to make the first-order partial derivative of the first target curve at the splicing constraint point equal to that of the second target curve, so that the first target curve and the second target curve can be spliced continuously and smoothly.
  • step 2031 may be executed by the processor calling a corresponding instruction stored in a memory, or may be executed by the first processing unit executed by the processor.
  • step 204 may specifically include the following steps:
  • Step 2041 Based on the splicing constraint points, the second curve parameters are adjusted using a constrained extended Kalman filter algorithm to obtain second target curve parameters.
  • step 2041 refers to step 2031 and will not be repeated here.
  • step 2041 may be executed by the processor calling a corresponding instruction stored in a memory, or may be executed by a second processing unit executed by the processor.
  • the disclosed embodiment uses a constrained extended Kalman filter algorithm to adjust the parameters of the two curves to be spliced, so that the adjusted two target curves can meet the smoothness and continuity of the splicing, further improving the lane line fusion effect.
  • FIG4 is a flowchart of step 2031 provided by an exemplary embodiment of the present disclosure.
  • step 2031 uses a constrained extended Kalman filter algorithm based on the splicing constraint point to adjust the first curve parameter to obtain the first target curve parameter, including:
  • Step 20311 determining the initial state quantity based on the first curve parameter.
  • the initial state quantity is used as the initial state quantity of the constrained extended Kalman filter algorithm.
  • the determined initial state quantity S 10 can be expressed as:
  • step 20311 may be executed by the processor calling a corresponding instruction stored in a memory, or may be executed by the first determining subunit executed by the processor.
  • Step 20312 based on the splicing constraint point, the first curve parameter and the second curve parameter, determine the constraint amount, the constraint amount includes the lateral coordinate value of the splicing constraint point and the first-order partial derivative constraint value at the longitudinal coordinate value of the splicing constraint point.
  • the constraint is used to constrain the adjustment of the first curve parameters so that the adjusted first target curve parameters meet the continuous smooth splicing constraint.
  • the horizontal coordinate value of the splicing constraint point is used as the horizontal coordinate constraint to make the first target curve corresponding to the first target curve parameter contain the splicing constraint point, that is, the splicing constraint point is on the first target curve.
  • the first-order partial derivative constraint value is used as the first-order partial derivative constraint to make the first-order partial derivative of the first target curve at the splicing constraint point the same as that of the second target curve, so that the first target curve and the second target curve can be spliced continuously and smoothly.
  • the constraint quantity is expressed as:
  • the first-order partial derivative constraint value can be determined based on the first-order partial derivative of the first lane line curve at yc and the first-order partial derivative of the second lane line curve at yc . It can be set according to actual needs, such as taking the average of the two first-order partial derivatives, which is expressed as follows:
  • the first-order partial derivative constraint value can be determined.
  • Step 20311 and step 20312 are in no particular order.
  • step 20312 may be executed by the processor calling a corresponding instruction stored in a memory, or may be executed by a second determining subunit executed by the processor.
  • Step 20313 determine the initial transfer matrix from the initial state quantity to the constraint quantity.
  • the initial state transfer matrix represents the transfer relationship from the initial state quantity to the constraint quantity.
  • the initial transfer matrix is expressed as follows:
  • step 20313 may be executed by the processor calling a corresponding instruction stored in a memory, or may be executed by a third determining subunit executed by the processor.
  • Step 20314 determine the Kalman gain based on the initial transfer matrix.
  • the Kalman gain represents the weight of the observation deviation, which can be expressed as K t , and is used to update the state quantity.
  • the state quantity is adjusted iteratively so as to satisfy the constraint of the constraint quantity.
  • the specific determination method of the Kalman gain can be set according to actual needs.
  • step 20314 may be executed by the processor calling a corresponding instruction stored in a memory, or may be executed by a fourth determining subunit executed by the processor.
  • Step 20315 determine the target state quantity based on the Kalman gain, constraint quantity, initial transfer matrix and initial state quantity.
  • the state quantity can be updated based on the Kalman gain, constraint quantity, initial transfer matrix, initial state quantity and preset update formula to obtain the updated state quantity. If the updated state quantity can meet the preset conditions or reach the preset number of iterations, the updated state quantity can be used as the target state quantity. Otherwise, the state quantity is continued to be updated based on the above process until the updated state quantity meets the preset conditions or reaches the preset number of iterations to obtain the target state quantity.
  • step 20315 may be executed by the processor calling a corresponding instruction stored in a memory, or may be executed by a fifth determining subunit executed by the processor.
  • Step 20316 determining the first target curve parameters based on the target state quantity.
  • each state value included in the target state quantity is the updated first target curve parameter.
  • the target state quantity can be expressed as follows:
  • step 20316 may be executed by the processor calling a corresponding instruction stored in a memory, or may be executed by a sixth determining subunit executed by the processor.
  • FIG5 is a flowchart of step 2041 provided by an exemplary embodiment of the present disclosure.
  • step 2041 based on the splicing constraint points, uses a constrained extended Kalman filter algorithm to adjust the second curve parameters to obtain the second target curve parameters, including:
  • Step 20411 determining the initial state quantity based on the second curve parameter.
  • Step 20412 based on the splicing constraint point, the first curve parameter and the second curve parameter, determine the constraint amount, the constraint amount includes the lateral coordinate value of the splicing constraint point and the first-order partial derivative constraint value at the longitudinal coordinate value of the splicing constraint point.
  • Step 20413 determine the initial transfer matrix from the initial state quantity to the constraint quantity.
  • Step 20414 determine the Kalman gain based on the initial transfer matrix.
  • Step 20415 determine the target state quantity based on the Kalman gain, constraint quantity, initial transfer matrix and initial state quantity.
  • Step 20416 determining the second target curve parameters based on the target state quantity.
  • steps 20411 to 20416 in this example refer to the aforementioned steps 20311 to 20316, the difference being that the initial state quantity in this example is the second curve parameter of the second lane line curve, and the specific principles will not be repeated one by one.
  • steps 20411 to 20416 may be executed by the processor calling corresponding instructions stored in the memory, or may be executed by a second processing unit executed by the processor.
  • step 20315 determines the target state quantity based on the Kalman gain, the constraint, the initial transfer matrix and the initial state quantity, including: iteratively updating the initial state quantity based on the Kalman gain, the constraint, and the initial transfer matrix until the preset iteration end condition is met to obtain the target state quantity.
  • the preset iteration end condition may include a maximum number of iterations and/or a condition that the covariance matrix needs to satisfy, which may be specifically set according to actual needs.
  • FIG6 is a schematic diagram of a state quantity update iteration process provided by an exemplary embodiment of the present disclosure.
  • the initial state quantity is iteratively updated until the preset iteration end condition is met to obtain the target state quantity, including:
  • Step 301 Determine the current curve parameter corresponding to the first curve parameter based on the new state quantity obtained in the previous iteration.
  • the new state quantity obtained in the previous iteration includes the updated curve parameter corresponding to the first curve parameter obtained in the previous update, and the updated curve parameter is used as the current curve parameter of the current iteration. If the current iteration is the first iteration, the new state quantity obtained in the previous iteration is the initial state quantity.
  • Step 302 determining the current state quantity based on the current curve parameters.
  • the current state quantity is expressed as:
  • i indicates that the current iteration is the i-th iteration.
  • the new state quantity obtained in the previous iteration can be used as the current state quantity of the current iteration.
  • Step 303 determining the current transfer matrix from the current state quantity to the constraint quantity.
  • the constraint quantity may be fixed and unchanged, see C_s above. Since the state quantity represents the curve parameter, the change of the curve parameter has no effect on the transfer matrix, so the current transfer matrix is the same as the initial transfer matrix.
  • the current transfer matrix is expressed as follows:
  • Step 304 Determine the current Kalman gain based on the current transfer matrix, the new covariance matrix obtained in the previous iteration, and the preset constraint noise matrix.
  • the covariance matrix is a diagonal matrix
  • the initial covariance matrix can be a preset diagonal matrix, which is continuously updated during the iteration process until convergence.
  • the preset constraint noise matrix represents the degree of trust in the constraint.
  • the preset constraint noise matrix is a diagonal matrix. When its values are all 0, it represents a hard constraint, that is, the first target curve parameter is obtained through one iteration. When its diagonal value is not 0, it represents a soft constraint. The end of the iteration can be controlled by the maximum number of iterations, which can be set according to actual needs.
  • the current Kalman matrix can be determined based on the preset Kalman gain formula. If the current iteration is the first iteration, the new covariance matrix obtained in the previous iteration is the preset initial covariance matrix.
  • the initial covariance matrix is expressed as:
  • ⁇ 1,0 - ⁇ 4,0 are preset values.
  • the new covariance matrix obtained in the previous iteration is expressed as:
  • ⁇ 1,i-1 - ⁇ 4,i-1 are the values obtained in the previous iteration.
  • the preset constraint noise matrix C_n 0 is expressed as:
  • ⁇ 1 and ⁇ 2 are preset values, for example, ⁇ 1 and ⁇ 2 are both set to 0.001 or 0, which can be set according to actual needs.
  • Step 305 based on the current Kalman gain, the constraint, and the current transfer matrix, the current state quantity is updated to obtain a new state quantity of the current iteration.
  • S 1i represents the new state quantity of the current iteration, and the meanings of other symbols refer to the above content. If the current iteration is the first iteration, the current state quantity is the initial state quantity S 10 .
  • Step 306 Based on the current Kalman gain and the current transfer matrix, the new covariance matrix obtained in the previous iteration is updated to obtain a new covariance matrix for the current iteration.
  • E_ci represents the new covariance matrix obtained in the current iteration.
  • Step 307 in response to the current iteration reaching a preset number of iterations and/or the new covariance matrix of the current iteration meeting a preset condition, the iteration process is terminated, and the new state quantity of the current iteration is used as the target state quantity.
  • the preset number of iterations and the preset conditions can be set according to actual needs to control the end of the iteration process.
  • the target state quantity is obtained.
  • the specific examples in the embodiments of the present disclosure take cubic curves as an example.
  • the first lane line curve and the second lane line curve to be spliced can also be curves of other order equations, such as straight lines or higher order curves.
  • the corresponding number of state quantities and other required data can be determined according to the number of curve parameters, without specific limitation.
  • the curve parameters include 5 coefficients
  • the corresponding initial state quantity includes 5 state values, which can be set according to actual needs.
  • the above steps 301 to 307 may be executed by the processor calling corresponding instructions stored in the memory, or may be executed by the fifth determining subunit executed by the processor.
  • step 20314 determines the Kalman gain based on the initial transfer matrix, including: determining the Kalman gain based on the initial transfer matrix, a preset initial covariance matrix, and a preset constraint noise matrix, wherein the preset initial covariance matrix is a diagonal matrix, and the preset constraint noise matrix is a diagonal matrix, which is used to represent the confidence of the constraint.
  • the disclosed embodiment uses a constrained extended Kalman filter algorithm to adjust the curve parameters of the first lane line curve and the second lane line curve to be spliced, and through the continuous updating of the constraint quantity and the state quantity, the two lane line curves are gradually approached to the splicing constraint point until the two lane line curves can be continuously and smoothly connected at the splicing constraint point, so that the spliced lane line is more in line with the continuous and smooth state of the real lane line.
  • FIG7 is a schematic diagram of a constraint amount determination process provided by an exemplary embodiment of the present disclosure.
  • step 20412 determines the constraint amount based on the splicing constraint point, the first curve parameter, and the second curve parameter, including:
  • y c represents the longitudinal coordinate value of the splicing constraint point
  • the first curve parameters are C 10 , C 11 , C 12 , and C 13 .
  • the second first-order partial derivative is expressed as follows:
  • y c represents the longitudinal coordinate value of the splicing constraint point
  • the second curve parameters are C 20 , C 21 , C 22 , and C 23 .
  • the first-order partial derivative constraint value is expressed as follows:
  • the disclosed embodiment constrains the adjustment process of the lateral coordinate value of the splicing constraint point in the constraint amount, so that the adjusted first target curve and the second target curve are continuously spliced at the splicing constraint point.
  • the constraint adjustment of the first-order partial derivative constraint value in the constraint amount makes the first-order partial derivatives of the adjusted first target curve and the second target curve at the splicing constraint point equal to the first-order partial derivative constraint value, that is, the first-order partial derivatives of the first target curve and the second target curve at the splicing constraint point are equal, thereby achieving smooth connection between the first target curve and the second target curve at the splicing constraint point.
  • the obtained first target curve and the second target curve can be continuously and smoothly spliced, further improving the splicing effect, so that the spliced lane line is more consistent with the state of the real lane line.
  • determining the splicing constraint point based on the first lane line curve and the second lane line curve in step 202 includes:
  • Step 2021 Determine the longitudinal coordinate value of the splicing constraint point based on the first lane line curve and the second lane line curve.
  • the longitudinal coordinate value yc of the splicing constraint point can be determined based on the longitudinal coordinates of the endpoints of the first lane line curve and the second lane line curve, or based on the overlapping area or gap area of the first lane line curve and the second lane line curve, and can be set according to actual needs.
  • step 2021 may be executed by the processor calling a corresponding instruction stored in a memory, or may be executed by a first determination unit executed by the processor.
  • Step 2022 Determine a first lateral coordinate value of the first lane line curve at the longitudinal coordinate value based on the longitudinal coordinate value.
  • C 10 , C 11 , C 12 , and C 13 are first curve parameters of the first lane line curve.
  • step 2022 may be executed by the processor calling a corresponding instruction stored in a memory, or may be executed by a second determination unit executed by the processor.
  • Step 2023 Determine a second lateral coordinate value of the second lane line curve at the longitudinal coordinate value based on the longitudinal coordinate value.
  • C 20 , C 21 , C 22 , and C 23 are second curve parameters of the second lane line curve.
  • step 2022 and step 2023 are not performed in any particular order.
  • step 2023 may be executed by the processor calling a corresponding instruction stored in the memory, or may be executed by a third determination unit executed by the processor.
  • Step 2024 Determine the horizontal coordinate value of the splicing constraint point based on the first horizontal coordinate value and the second horizontal coordinate value.
  • the longitudinal coordinate value of the splicing constraint point is determined based on the first lane line curve and the second lane line curve, when the first lane line curve and the second lane line curve do not overlap in the longitudinal direction but have a certain gap, the splicing constraint point may not be on the first lane line curve and the second lane line curve, but in the gap between the two. Therefore, the determined first transverse coordinate value and the second transverse coordinate value are obtained by extending the corresponding lane line curve in the longitudinal direction to the longitudinal coordinate value of the splicing constraint point. The obtained first transverse coordinate value and the second transverse coordinate value may not be equal.
  • the longitudinal coordinate value is determined based on the overlapping area, or the longitudinal coordinate value is determined based on the two ends of the overlap.
  • the determined first transverse coordinate value and the second transverse coordinate value are respectively the transverse coordinate values of the truncation points of the two lane line curves at the longitudinal coordinate value, which may also cause the first transverse coordinate value and the second transverse coordinate value of the two lane line curves at the longitudinal coordinate value to be different. Therefore, it is necessary to determine the transverse coordinate value of the splicing constraint point of the two lane line curves based on the first transverse coordinate value and the second transverse coordinate value.
  • FIG9 is a schematic diagram of the longitudinal overlap of the first lane line curve and the second lane line curve provided by an exemplary embodiment of the present disclosure.
  • step 2024 may be executed by the processor calling a corresponding instruction stored in the memory, or may be executed by a fourth determining unit executed by the processor.
  • Step 2025 Determine the splicing constraint point based on the horizontal coordinate value and the vertical coordinate value of the splicing constraint point.
  • the obtained splicing constraint point is (x c ,y c ).
  • step 2025 may be executed by the processor calling a corresponding instruction stored in the memory, or may be executed by a fifth determining unit executed by the processor.
  • the disclosed embodiment first determines the longitudinal coordinate value of the splicing constraint point according to the extension characteristics of the lane line in the longitudinal direction, and then determines the lateral coordinate value of the splicing constraint point based on the average of the lateral coordinate values of the two lane line curves at the longitudinal coordinate value, so that the splicing constraint point can constrain the two lane line curves to approach the splicing constraint point during the adjustment of the curve parameters until the two lane line curves can be continuously and smoothly connected at the splicing constraint point, which helps to improve the adjustment effect and avoid the situation where one of the two lane line curves is adjusted too much, resulting in a decrease in the accuracy of the lane line.
  • FIG10 is a flowchart of step 21 provided by an exemplary embodiment of the present disclosure. Determining the longitudinal coordinate value of the splicing constraint point based on the first lane line curve and the second lane line curve includes:
  • Step 20211 determine the first endpoint ordinate and the second endpoint ordinate of the first lane line curve, and the third endpoint ordinate and the fourth endpoint ordinate of the second lane line curve.
  • the first endpoint ordinate and the second endpoint ordinate are respectively the ordinate ys1 of the first starting point and the ordinate ye1 of the first ending point of the first lane line curve
  • the third endpoint ordinate and the fourth endpoint ordinate are respectively the ordinate ys2 of the second starting point and the ordinate ye2 of the second ending point of the second lane line curve.
  • the ordinates of each endpoint may be obtained from the coordinate range of the first lane line curve and the second lane line curve, and will not be described in detail.
  • step 20211 may be executed by the processor calling a corresponding instruction stored in a memory, or may be executed by a first determination unit executed by the processor.
  • Step 20212 based on the first endpoint ordinate, the second endpoint ordinate, the third endpoint ordinate and the fourth endpoint ordinate, determine the first target endpoint ordinate and the second target endpoint ordinate adjacent to the first lane line curve and the second lane line curve.
  • the positional relationship between the two lane line curves can be determined by the coordinates of each endpoint. Specifically, it can be determined by the range of the ordinate or by comparing the size of the ordinates of each endpoint. Then, the ordinates of the endpoints of the two lane line curves that need to be spliced can be determined as the first target endpoint ordinate and the second target endpoint ordinate.
  • step 20212 may be executed by the processor calling corresponding instructions stored in the memory, or may be executed by the first determination unit executed by the processor.
  • Step 20213 based on the longitudinal coordinate of the first target endpoint and the longitudinal coordinate of the second target endpoint, determine the longitudinal coordinate value of the splicing constraint point.
  • the average value of the longitudinal coordinate of the first target endpoint and the second target endpoint can be used as the longitudinal coordinate value of the splicing constraint point.
  • step 20213 may be executed by the processor calling a corresponding instruction stored in a memory, or may be executed by a first determination unit executed by the processor.
  • determining the first lane line curve and the second lane line curve to be spliced in the first coordinate system in step 201 includes:
  • Step 2011 Acquire first image data corresponding to at least two viewing angles.
  • the first image data may be acquired by collecting at least two viewing angles of cameras arranged on the vehicle.
  • the surround view camera installed on the vehicle can collect the first image data corresponding to each perspective of the surrounding road environment in real time, and the first image data corresponding to at least two perspectives can be collected by each camera at the same time.
  • the first image data collected by each camera at the current moment of the vehicle's driving It can also be the first image data of at least two perspectives collected and stored in advance at the same time, without specific limitation.
  • step 2011 may be executed by the processor calling a corresponding instruction stored in a memory, or may be executed by a first acquisition unit executed by the processor.
  • Step 2012 Based on each of the first image data, determine the lane line area in the image coordinate system corresponding to each of the first image data.
  • the lane line area in the image coordinate system includes a pixel set belonging to the lane line in the image coordinate system.
  • the lane line area in the image coordinate system corresponding to each first image data can be detected by a pre-trained target detection model, and the details are not repeated here.
  • step 2012 may be executed by the processor calling the corresponding instruction stored in the memory, or by the processed The sixth determination unit of the device operation is executed.
  • Step 2013 convert the lane line areas corresponding to the first image data to the first coordinate system, and obtain the lane line coordinate points in the first coordinate system corresponding to the first image data.
  • the first coordinate system can be the coordinate system corresponding to the bird's-eye view (BEV). Since the lane line area of each perspective is based on the area of its own image coordinate system, in order to achieve the fusion of each perspective, it is necessary to convert it to a unified coordinate system.
  • the first coordinate system of the embodiment of the present disclosure adopts the coordinate system corresponding to the bird's-eye view, and the coordinate origin is the center of the vehicle.
  • the lane line curves of each perspective are converted to a unified first coordinate system, so that the lane lines of each perspective can be fused to obtain a surround lane line map around the vehicle.
  • the conversion from the image coordinate system to the first coordinate system can be achieved according to the mapping relationship between the two coordinate systems, and the specific principles will not be repeated here.
  • step 2013 may be executed by the processor calling corresponding instructions stored in the memory, or may be executed by a third processing unit executed by the processor.
  • Step 2014 Determine the lane line curve in the first coordinate system corresponding to each of the first image data based on the lane line coordinate points in the first coordinate system corresponding to each of the first image data.
  • the pixels in the lane line area in the image coordinate system are converted to lane line coordinate points in the first coordinate system, and then the lane line curves in the first coordinate system corresponding to each first image data can be obtained by fitting.
  • the specific fitting principle will not be repeated.
  • step 2014 may be executed by the processor calling a corresponding instruction stored in the memory, or may be executed by a seventh determination unit executed by the processor.
  • Step 2015 Determine a first lane line curve and a second lane line curve to be spliced in the first coordinate system based on the lane line curves in the first coordinate system corresponding to each of the first image data.
  • whether two lane line curves can be spliced can be determined by using preset matching rules to match multiple lane line curves in the first coordinate system.
  • the preset matching rules can be set according to the characteristics of the lane lines. For example, in the bird's-eye view coordinate system, lane line curves at different perspectives extend in the longitudinal direction (y direction) and have a relatively small distance in the lateral direction. Accordingly, corresponding matching rules are set to determine the two lane line curves that need to be spliced.
  • one of the two lane line curves can be sampled to obtain multiple sampling points, the distance between the multiple sampling points and the other lane line curve is less than a certain threshold, and the difference between the first-order coefficients of the two lane line curves (such as the difference between C11 of the first lane line curve and C21 of the second lane line curve mentioned above) meets certain conditions, and the specific matching principle is not repeated here.
  • step 2015 may be executed by the processor calling a corresponding instruction stored in the memory, or may be executed by an eighth determination unit executed by the processor.
  • the lane line splicing method determines the splicing constraint point and uses the constrained extended Kalman filter algorithm to constrain the continuity and smoothness of the two lane lines at the splicing constraint point, so that the lane line obtained by splicing is continuous and smooth at the splicing point; and the delay and calculation amount of the constrained extended Kalman filter algorithm are low, which can ensure the real-time and effective splicing of the lane line, and provide better surround lane line data for subsequent applications; in addition, since the cameras of each perspective of the embodiment of the present disclosure independently perceive the lane line, in the fusion process, different line segments of the same lane line can be spliced to obtain a complete lane line, and the determination of the splicing constraint point allows the lane lines of different perspectives to be extended or truncated, so that each camera has a certain fault tolerance rate. When one or several cameras fail, the lane line in the middle part can be obtained by extending and splicing
  • Any lane line splicing method provided in the embodiments of the present disclosure may be executed by any appropriate device with data processing capabilities, including but not limited to: a terminal device and a server, etc.
  • any lane line splicing method provided in the embodiments of the present disclosure may be executed by a processor, such as the processor executing any lane line splicing method mentioned in the embodiments of the present disclosure by calling corresponding instructions stored in a memory. This will not be described in detail below.
  • FIG11 is a schematic diagram of the structure of a lane line splicing device provided by an exemplary embodiment of the present disclosure.
  • the device of this embodiment can be used to implement the corresponding method embodiment of the present disclosure.
  • the device shown in FIG11 includes: a first determination module 501, a first processing module 502, a second processing module 503, a third processing module 504 and a fourth processing module 505.
  • a first determining module 501 is used to determine a first lane line curve and a second lane line curve to be spliced in a first coordinate system; a first processing module 502 is used to determine a splicing constraint point based on the first lane line curve and the second lane line curve determined by the first determining module 501; The second processing module 503 is used to adjust the first curve parameters corresponding to the first lane line curve based on the splicing constraint points determined by the first processing module 502, and obtain the adjusted first target curve parameters corresponding to the first curve parameters; the third processing module 504 is used to adjust the second curve parameters corresponding to the second lane line curve based on the splicing constraint points determined by the first processing module 502, and obtain the adjusted second target curve parameters corresponding to the second curve parameters; the fourth processing module 505 is used to determine the spliced lane line based on the first target curve parameters and the second target curve parameters.
  • FIG. 12 is a schematic structural diagram of a lane line splicing device provided by another exemplary embodiment of the present disclosure.
  • the second processing module 503 includes: a first processing unit 5031, configured to adjust the first curve parameters based on the splicing constraint points and adopt a constrained extended Kalman filter algorithm to obtain the first target curve parameters.
  • the third processing module 504 includes: a second processing unit 5041, configured to adjust the second curve parameters based on the splicing constraint points and adopt a constrained extended Kalman filter algorithm to obtain second target curve parameters.
  • Figure 13 is a structural diagram of the first processing unit 5031 provided by an exemplary embodiment of the present disclosure.
  • the first processing unit 5031 includes: a first determination subunit 50311, a second determination subunit 50312, a third determination subunit 50313, a fourth determination subunit 50314, a fifth determination subunit 50315 and a sixth determination subunit 50316.
  • the first determination subunit 50311 is used to determine the initial state quantity based on the first curve parameter; the second determination subunit 50312 is used to determine the constraint quantity based on the splicing constraint point, the first curve parameter and the second curve parameter, the constraint quantity including the lateral coordinate value of the splicing constraint point and the first-order partial derivative constraint value at the longitudinal coordinate value of the splicing constraint point; the third determination subunit 50313 is used to determine the initial transfer matrix from the initial state quantity to the constraint quantity; the fourth determination subunit 50314 is used to determine the Kalman gain based on the initial transfer matrix; the fifth determination subunit 50315 is used to determine the target state quantity based on the Kalman gain, the constraint quantity, the initial transfer matrix and the initial state quantity; the sixth determination subunit 50316 is used to determine the first target curve parameter based on the target state quantity.
  • the fifth determination subunit 50315 is specifically used to iteratively update the initial state quantity based on the Kalman gain, the constraint quantity, and the initial transfer matrix until a preset iteration end condition is met to obtain the target state quantity.
  • the fifth determination subunit 50315 is specifically used to: during the iteration process, take any iteration as the current iteration, and perform the following steps: determine the current curve parameters corresponding to the first curve parameters based on the new state quantity obtained in the previous iteration; determine the current state quantity based on the current curve parameters; determine the current transfer matrix from the current state quantity to the constraint quantity; determine the current Kalman gain based on the current transfer matrix, the new covariance matrix obtained in the previous iteration, and the preset constraint quantity noise matrix; update the current state quantity based on the current Kalman gain, the constraint quantity, and the current transfer matrix to obtain a new state quantity for the current iteration; update the new covariance matrix obtained in the previous iteration based on the current Kalman gain and the current transfer matrix to obtain a new covariance matrix for the current iteration; in response to the current iteration reaching a preset number of iterations, and/or the new covariance matrix of the current iteration meets a
  • the fourth determination subunit 50314 is specifically used to determine the Kalman gain based on the initial transfer matrix, the preset initial covariance matrix, and the preset constraint noise matrix, the preset initial covariance matrix is a diagonal matrix, the preset constraint noise matrix is a diagonal matrix, and is used to represent the confidence of the constraint.
  • the second determination subunit 50312 is specifically used to: determine the first first-order partial derivative of the first lane line curve at the longitudinal coordinate value of the splicing constraint point based on the first curve parameter; determine the second first-order partial derivative of the second lane line curve at the longitudinal coordinate value of the splicing constraint point based on the second curve parameter; determine the first-order partial derivative constraint value based on the first first-order partial derivative and the second first-order partial derivative; determine the constraint amount based on the lateral coordinate value of the splicing constraint point and the first-order partial derivative constraint value.
  • the first processing module 502 includes: a first determining unit 5021 , a second determining unit 5022 , a third determining unit 5023 , a fourth determining unit 5024 and a fifth determining unit 5025 .
  • the first determining unit 5021 is used to determine the longitudinal coordinate value of the splicing constraint point based on the first lane line curve and the second lane line curve; the second determining unit 5022 is used to determine the first lateral coordinate value of the first lane line curve at the longitudinal coordinate value based on the longitudinal coordinate value; the third determining unit 5023 is used to determine the second lateral coordinate value of the second lane line curve at the longitudinal coordinate value based on the longitudinal coordinate value; the fourth determining unit 5024 is used to determine the lateral coordinate value of the splicing constraint point based on the first lateral coordinate value and the second lateral coordinate value; the fifth determining unit 5025 is used to determine the splicing constraint point based on the lateral coordinate value and the longitudinal coordinate value of the splicing constraint point.
  • the first determination unit 5021 is specifically used to: determine the first endpoint ordinate and the second endpoint ordinate of the first lane line curve, and the third endpoint ordinate and the fourth endpoint ordinate of the second lane line curve; determine the first target endpoint ordinate and the second target endpoint ordinate adjacent to the first lane line curve and the second lane line curve based on the first endpoint ordinate, the second endpoint ordinate, the third endpoint ordinate and the fourth endpoint ordinate; determine the longitudinal coordinate value of the splicing constraint point based on the first target endpoint ordinate and the second target endpoint ordinate.
  • the first determination module 501 includes: a first acquisition unit 5011 , a sixth determination unit 5012 , a third processing unit 5013 , a seventh determination unit 5014 and an eighth determination unit 5015 .
  • the first acquisition unit 5011 is used to acquire first image data corresponding to at least two perspectives; the sixth determination unit 5012 is used to determine the lane line area in the image coordinate system corresponding to each first image data based on each first image data; the third processing unit 5013 is used to Used to convert the lane line areas corresponding to each first image data into the first coordinate system, and obtain the lane line coordinate points in the first coordinate system corresponding to each first image data; the seventh determination unit 5014 is used to determine the lane line curve in the first coordinate system corresponding to each first image data based on the lane line coordinate points in the first coordinate system corresponding to each first image data; the eighth determination unit 5015 is used to determine the first lane line curve and the second lane line curve to be spliced in the first coordinate system based on the lane line curve in the first coordinate system corresponding to each first image data.
  • FIG14 is a schematic diagram of a structure of an application embodiment of an electronic device disclosed in the present invention.
  • the electronic device 10 includes one or more processors 11 and a memory 12.
  • the processor 11 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and/or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.
  • CPU central processing unit
  • the memory 12 may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory.
  • Volatile memory may include, for example, random access memory (RAM) and/or cache memory (cache), etc.
  • Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc.
  • One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 11 may run the program instructions to implement the methods of the various embodiments of the present disclosure above and/or other desired functions.
  • Various contents such as input signals, signal components, noise components, etc. may also be stored in the computer-readable storage medium.
  • the electronic device 10 may further include: an input device 13 and an output device 14, and these components are interconnected via a bus system and/or other forms of connection mechanisms (not shown).
  • the input device 13 may also include, for example, a keyboard, a mouse, etc.
  • the output device 14 can output various information to the outside, and the output device 14 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.
  • FIG14 only shows some of the components related to the present disclosure in the electronic device 10, omitting components such as a bus, an input/output interface, etc.
  • the electronic device 10 may further include any other appropriate components according to specific application scenarios.
  • an embodiment of the present disclosure may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the method according to various embodiments of the present disclosure described in the above-mentioned "Exemplary Method" section of this specification.
  • the computer program product may be written in any combination of one or more programming languages to write program code for performing the operations of the disclosed embodiments, including object-oriented programming languages such as Java, C++, etc., and conventional procedural programming languages such as "C" or similar programming languages.
  • the program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.
  • an embodiment of the present disclosure may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, enables the processor to execute the steps of the method according to various embodiments of the present disclosure described in the above “Exemplary Method” section of this specification.
  • Computer readable storage media can adopt any combination of one or more readable media.
  • the readable medium can be a readable signal medium or a readable storage medium.
  • the readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above.
  • readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.
  • RAM random access memory
  • ROM read-only memory
  • EPROM or flash memory erasable programmable read-only memory
  • CD-ROM compact disk read-only memory
  • magnetic storage device or any suitable combination of the above.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Health & Medical Sciences (AREA)
  • Databases & Information Systems (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Traffic Control Systems (AREA)
  • Image Processing (AREA)

Abstract

本公开实施例公开了一种车道线的拼接方法、装置、电子设备和存储介质，其中，方法包括：确定第一坐标系下的待拼接的第一车道线曲线和第二车道线曲线；基于第一车道线曲线和第二车道线曲线，确定拼接约束点；基于拼接约束点，对第一车道线曲线对应的第一曲线参数进行调整，获得第一曲线参数对应的调整后的第一目标曲线参数；基于拼接约束点，对第二车道线曲线对应的第二曲线参数进行调整，获得第二曲线参数对应的调整后的第二目标曲线参数；基于第一目标曲线参数和第二目标曲线参数，确定拼接后的车道线。本公开实施例通过基于拼接约束点对两车道线曲线的曲线参数进行调整，使得调整后的两车道线可以平滑连续地拼接，有助于提高车道线的融合效果。

Description

车道线的拼接方法、装置、电子设备和存储介质
本公开要求在2022年11月09日提交国家知识产权局、申请号为CN202211401951.0、发明名称为“车道线的拼接方法、装置、电子设备和存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本公开中。
技术领域
本公开涉及辅助驾驶技术，尤其是一种车道线的拼接方法、装置、电子设备和存储介质。
背景技术
在辅助驾驶场景，通过多个视角的相机采集环境图像，并基于各视角的环境图像分别拟合车道线，再将各环境图像分别对应的车道线变换到统一的鸟瞰图(Bird's Eye View，简称：BEV)坐标系下，获得该鸟瞰图坐标系下的车道线，进而将各视角分别对应的鸟瞰图坐标系下的车道线融合，形成环视的车道线鸟瞰图供下游使用。
发明内容
本公开的实施例提供了一种车道线的拼接方法、装置、电子设备和存储介质。
根据本公开实施例的一个方面，提供了一种车道线的拼接方法，包括：确定第一坐标系下的待拼接的第一车道线曲线和第二车道线曲线；基于所述第一车道线曲线和所述第二车道线曲线，确定拼接约束点；基于所述拼接约束点，对所述第一车道线曲线对应的第一曲线参数进行调整，获得所述第一曲线参数对应的调整后的第一目标曲线参数；基于所述拼接约束点，对所述第二车道线曲线对应的第二曲线参数进行调整，获得所述第二曲线参数对应的调整后的第二目标曲线参数；基于所述第一目标曲线参数和所述第二目标曲线参数，确定拼接后的车道线。
根据本公开实施例的另一个方面，提供了一种车道线的拼接装置，包括：第一确定模块，用于确定第一坐标系下的待拼接的第一车道线曲线和第二车道线曲线；第一处理模块，用于基于所述第一车道线曲线和所述第二车道线曲线，确定拼接约束点；第二处理模块，用于基于所述拼接约束点，对所述第一车道线曲线对应的第一曲线参数进行调整，获得所述第一曲线参数对应的调整后的第一目标曲线参数；第三处理模块，用于基于所述拼接约束点，对所述第二车道线曲线对应的第二曲线参数进行调整，获得所述第二曲线参数对应的调整后的第二目标曲线参数；第四处理模块，用于基于所述第一目标曲线参数和所述第二目标曲线参数，确定拼接后的车道线。
根据本公开实施例的再一方面，提供一种计算机可读存储介质，所述存储介质存储有计算机程序，所述计算机程序用于执行本公开上述任一实施例所述的车道线的拼接方法。
根据本公开实施例的又一方面，提供一种电子设备，所述电子设备包括：处理器；用于存储所述处理器可执行指令的存储器；所述处理器，用于从所述存储器中读取所述可执行指令，并执行所述指令以实现本公开上述任一实施例所述的车道线的拼接方法。
基于本公开上述实施例提供的车道线的拼接方法、装置、电子设备和存储介质，通过基于待拼接的两车道线曲线，确定出拼接约束点，基于拼接约束点对两车道线曲线的曲线参数进行调整，使得调整后的两车道线可以平滑连续地拼接，有助于提高融合后车道线的连续性和平滑性。
下面通过附图和实施例，对本公开的技术方案做进一步的详细描述。
附图说明
图1是本公开提供的车道线的拼接方法的一个示例性的应用场景；
图2是本公开一示例性实施例提供的车道线的拼接方法的流程示意图；
图3是本公开另一示例性实施例提供的车道线的拼接方法的流程示意图；
图4是本公开一示例性实施例提供的步骤2031的流程示意图；
图5是本公开一示例性实施例提供的步骤2041的流程示意图；
图6是本公开一示例性实施例提供的本公开一示例性实施例提供的状态量更新迭代流程示意图；
图7是本公开一示例性实施例提供的约束量的确定流程示意图；
图8是本公开一示例性实施例提供的第一车道线曲线和第二车道线曲线纵向方向交叠情况示意图；
图9是本公开一示例性实施例提供的第一车道线曲线和第二车道线曲线纵向方向间隙情况示意图；
图10是本公开一示例性实施例提供的步骤2021的流程示意图；
图11是本公开一示例性实施例提供的车道线的拼接装置的结构示意图；
图12是本公开另一示例性实施例提供的车道线的拼接装置的结构示意图；
图13是本公开一示例性实施例提供的第一处理单元5031的结构示意图；
图14是本公开电子设备一个应用实施例的结构示意图。
具体实施方式
为了解释本公开，下面将参考附图详细地描述本公开的示例实施例，显然，所描述的实施例仅是本公开的一部分实施例，而不是全部实施例，应理解，本公开不受示例性实施例的限制。
应注意到：除非另外具体说明，否则在这些实施例中阐述的部件和步骤的相对布置、数字表达式和数值不限制本公开的范围。
本公开概述
在实现本公开的过程中，发明人发现，在辅助驾驶场景，通过多个视角的相机采集环境图像，并基于各视角的环境图像分别拟合车道线，再将各环境图像分别对应的车道线变换到统一的鸟瞰图(BEV)坐标系下，获得该鸟瞰图坐标系下的车道线，进而将各视角分别对应的鸟瞰图坐标系下的车道线融合，形成环视的车道线鸟瞰图供下游使用，融合获得的环视的车道线鸟瞰图可能在属于同一车道线的两段车道线的融合部分连续性、平滑性较差。
示例性概述
图1是本公开提供的车道线的拼接方法的一个示例性的应用场景。
在辅助驾驶场景，可以通过多个视角的相机采集环境图像，并基于各视角的环境图像分别拟合车道线，再将各环境图像分别对应的车道线变换到统一的鸟瞰图(BEV)坐标系下，获得各视角分别对应的鸟瞰图坐标系下的车道线，进而将各视角分别对应的鸟瞰图坐标系下的车道线融合，形成环视的车道线鸟瞰图，利用本公开的车道线的拼接方法，在将各视角分别对应的鸟瞰图坐标系下的车道线进行融合时，可以基于一定的匹配规则，确定出待拼接的第一车道线曲线和第二车道线曲线，该第一车道线曲线和第二车道线曲线是两个视角分别对应的同一车道线的不同车道线段，比如左前相机视角的最左侧车道线的第一车道线段和左后相机视角的最左侧车道线的第二车道线段，第一车道线段和第二车道线段分别被左前相机和左后相机采集后，经车道线检测、拟合及坐标转换，获得鸟瞰图坐标系下分别对应第一车道线曲线和第二车道线曲线，在确定出鸟瞰图坐标系下的待拼接的第一车道线曲线和第二车道线曲线之后，可以基于第一车道线曲线和第二车道线曲线，确定拼接约束点，基于拼接约束点，对第一车道线曲线对应的第一曲线参数进行调整，获得第一曲线参数对应的调整后的第一目标曲线参数，并基于拼接约束点，对第二车道线曲线对应的第二曲线参数进行调整，获得第二曲线参数对应的调整后的第二目标曲线参数，使得调整后的第一目标曲线参数所对应的第一目标曲线与第二目标曲线参数所对应的第二目标曲线能够平滑连续地拼接，从而基于第一目标曲线参数和第二目标曲线参数，可以确定拼接后的连续平滑的车道线，有助于提高各视角融合后的车道线的连续性和平滑性。可以理解地，在各视角的鸟瞰图坐标系下的车道线融合时，可能包括多组待拼接的车道线，或者同一车道线对应的多个不同线段的鸟瞰图坐标系下的车道线曲线需要两两相互拼接，对其中任意两条待拼接的车道线曲线，均可以分别作为第一车道线曲线和第二车道线曲线，并利用上述方法进行拼接，具体不再一一赘述。
示例性方法
图2是本公开一示例性实施例提供的车道线的拼接方法的流程示意图。本实施例可应用在电子设备上，具体比如车载计算平台上，如图2所示，包括如下步骤：
步骤201，确定第一坐标系下的待拼接的第一车道线曲线和第二车道线曲线。
其中，第一坐标系可以为鸟瞰视角(BEV)对应的坐标系，也可以为世界坐标系，具体可以根据实际需求设置，只要能把不同视角相机采集的车道线曲线统一到同一坐标系即可。第一车道线曲线和第二车道线曲线分别为曲线段，具有起始点和终止点，比如第一车道线曲线和第二车道线曲线分别可以表示如下：
f1(x,y)＝0,    x∈[xs1,xe1),y∈[ys1,ye1)
f2(x,y)＝0,    x∈[xs2,xe2),y∈[ys2,ye2)
其中，f1(x,y)＝0表示第一车道线曲线，xs1和ys1分别表示第一车道线曲线的起始点的横坐标和纵 坐标，xe1和ye1分别表示第一车道线曲线的终止点的横坐标和纵坐标；f2(x,y)＝0表示第二车道线曲线，xs2和ys2分别表示第二车道线曲线的起始点的横坐标和纵坐标，xe2和ye2分别表示第二车道线曲线的终止点的横坐标和纵坐标。
对于两个车道线曲线是否可以拼接的确定，可以采用预设匹配规则对第一坐标系下的多个车道线两两之间进行匹配获得，其中预设匹配规则可以根据车道线的特性来设置，比如在鸟瞰图坐标系下，以车辆中心为坐标原点，以车辆正前方向为纵轴(y轴)，以车辆正右方向为横轴(x轴)，不同视角的车道线在纵向方向(y方向)延伸，在横向方向距离很小，依此设置相应的匹配规则用于确定出需要拼接的两个车道线曲线。
在一个可选示例中，该步骤201可以由处理器调用存储器存储的相应指令执行，也可以由被处理器运行的第一确定模块执行。
步骤202，基于第一车道线曲线和第二车道线曲线，确定拼接约束点。
其中，由于不同视角车道线检测、拟合及坐标转换可能存在的误差，导致第一车道线曲线和第二车道线曲线在连接处不连续或者不平滑，拼接约束点用于在调整第一车道线曲线和第二车道线曲线时，约束其调整过程，使得调整后的两条车道线曲线能够在拼接约束点连续平滑拼接。拼接约束点可以是基于第一车道线曲线与第二车道线曲线的起始点和终止点确定，起始点和终止点也可以称为端点，比如基于第一车道线曲线的第一起始点和第一终止点、第二车道线曲线的第二起始点和第二终止点，可以确定两车道线曲线的拼接端，比如第一车道线曲线的第一终止点与第二车道线曲线的第二起始点需要进行拼接，基于第一终止点和第二起始点的位置关系，确定出拼接约束点。比如第一车道线曲线和第二车道线曲线在纵向方向可以包括交叠(即第一终止点的纵坐标在第二车道线曲线的纵坐标范围内，第二起始点的纵坐标在第一车道线曲线的纵坐标范围内)或间隙(即第一终止点的纵坐标在第二车道线曲线的纵坐标范围外，第二起始点的纵坐标在第一车道线曲线的纵坐标范围外)两种情况，根据不同情况确定出相应的拼接约束点。比如对于间隙情况，可以基于第一终止点和第二起始点确定间隙区域的中心作为拼接约束点，具体可以根据实际需求设置。
在一个可选示例中，该步骤202可以由处理器调用存储器存储的相应指令执行，也可以由被处理器运行的第一处理模块执行。
步骤203，基于拼接约束点，对第一车道线曲线对应的第一曲线参数进行调整，获得第一曲线参数对应的调整后的第一目标曲线参数。
其中，第一车道线曲线对应的第一曲线参数是用于表示第一车道线曲线的多次曲线系数，比如三次曲线系数，对第一曲线参数进行调整的原理是，使调整后的第一目标曲线参数对应的第一目标曲线在拼接约束点能够与调整后的第二目标曲线连续平滑拼接。
示例性的，第一车道线曲线为三次曲线，第一曲线参数为C10、C11、C12、C13，则第一车道线曲线可以表示为：
x＝C10+C11y+C12y2+C13y3
可选地，对第一曲线参数进行调整可以采用任意可实施的优化方式，比如基于带约束的扩展卡尔曼滤波算法、非线性优化等方式，具体可以根据实际需求设置。
在一个可选示例中，该步骤203可以由处理器调用存储器存储的相应指令执行，也可以由被处理器运行的第二处理模块执行。
步骤204，基于拼接约束点，对第二车道线曲线对应的第二曲线参数进行调整，获得第二曲线参数对应的调整后的第二目标曲线参数。
其中，第二车道线曲线对应的第二曲线参数是用于表示第二车道线曲线的多次曲线系数，比如三次曲线系数。同理，对第二曲线参数进行调整的原理是，使调整后的第二目标曲线参数对应的第二目标曲线在拼接约束点能够与调整后的第一目标曲线连续平滑拼接。
示例性的，第二车道线曲线为三次曲线，第二曲线参数为C20、C21、C22、C23，则第二车道线曲线可以表示为：
x＝C20+C21y+C22y2+C23y3
需要说明的是，步骤203和步骤204不分先后顺序。
在一个可选示例中，该步骤204可以由处理器调用存储器存储的相应指令执行，也可以由被处理器运行的第三处理模块执行。
步骤205，基于第一目标曲线参数和第二目标曲线参数，确定拼接后的车道线。
其中，第一目标曲线参数和第二目标曲线参数是基于拼接约束点调整后的曲线参数，两者分别对应的第一目标曲线和第二目标曲线在拼接约束点能够连续平滑拼接，获得拼接后的车道线。基于此，可以将各视角分别对应的第一坐标系下的需要拼接的车道线曲线进行拼接，从而可以获得第一坐标系下平 滑连续的环视车道线图。
在一个可选示例中，第一坐标系为鸟瞰视角对应的坐标系，获得拼接后的车道线后，可以基于拼接后的车道线与其他车道线融合，获得鸟瞰视角下的环视车道线图，鸟瞰视角下的环视车道线图是车辆周围局部区域的车道线图，在实际应用中，还可以将鸟瞰视角下的环视车道线图转换到全局坐标系(比如世界坐标系)下，从而可以随着时间推移获得全局的车道线图，具体可以根据实际需求设置。
在一个可选示例中，该步骤205可以由处理器调用存储器存储的相应指令执行，也可以由被处理器运行的第四处理模块执行。
本实施例提供的车道线的拼接方法，通过基于待拼接的两车道线曲线，确定出拼接约束点，基于拼接约束点对两车道线曲线的曲线参数进行调整，使得调整后的两车道线可以平滑连续地拼接，有效提高融合后车道线的连续性和平滑性。
图3是本公开另一示例性实施例提供的车道线的拼接方法的流程示意图。
在一个可选示例中，步骤203具体可以包括以下步骤：
步骤2031，基于拼接约束点，采用带约束的扩展卡尔曼滤波算法，对第一曲线参数进行调整，获得第一目标曲线参数。
其中，基于带约束的扩展卡尔曼滤波算法，可以将第一曲线参数作为初始状态量，基于拼接约束点、及第一车道线曲线和第二车道线曲线连续平滑拼接需要满足的条件设置相应的约束量，通过迭代滤波实现状态转移，更新第一曲线参数，获得更新后的第一目标曲线参数，使得更新后的第一目标曲线参数满足连续平滑拼接约束。其中，约束量可以包括拼接约束点处的横向坐标约束和一阶偏导数约束，横向坐标约束用于使第一目标曲线参数对应的第一目标曲线能够包含拼接约束点，一阶偏导数约束用于使第一目标曲线在拼接约束点处的一阶偏导数与第二目标曲线相等，从而使得第一目标曲线与第二目标曲线能够连续平滑拼接。
在一个可选示例中，该步骤2031可以由处理器调用存储器存储的相应指令执行，也可以由被处理器运行的第一处理单元执行。
在一个可选示例中，步骤204具体可以包括以下步骤：
步骤2041，基于拼接约束点，采用带约束的扩展卡尔曼滤波算法，对第二曲线参数进行调整，获得第二目标曲线参数。
其中，步骤2041的具体原理参见步骤2031，在此不再赘述。
在一个可选示例中，该步骤2041可以由处理器调用存储器存储的相应指令执行，也可以由被处理器运行的第二处理单元执行。
本公开实施例通过采用带约束的扩展卡尔曼滤波算法实现对待拼接的两曲线参数进行调整，使得调整后的两目标曲线能够满足拼接的平滑性和连续性，进一步提高车道线融合效果。
在一个可选示例中，图4是本公开一示例性实施例提供的步骤2031的流程示意图。在本示例中，步骤2031的基于拼接约束点，采用带约束的扩展卡尔曼滤波算法，对第一曲线参数进行调整，获得第一目标曲线参数，包括：
步骤20311，基于第一曲线参数确定初始状态量。
其中，初始状态量作为带约束的扩展卡尔曼滤波算法的初始状态量。
示例性的，对于第一曲线参数C10、C11、C12、C13，确定的初始状态量S10可以表示为：
在一个可选示例中，该步骤20311可以由处理器调用存储器存储的相应指令执行，也可以由被处理器运行的第一确定子单元执行。
步骤20312，基于拼接约束点、第一曲线参数和第二曲线参数，确定约束量，约束量包括拼接约束点的横向坐标值和在拼接约束点的纵向坐标值处的一阶偏导数约束值。
其中，约束量用于约束第一曲线参数的调整，使得调整后的第一目标曲线参数满足连续平滑拼接约束。拼接约束点的横向坐标值作为横向坐标约束，用于使第一目标曲线参数对应的第一目标曲线能够包含拼接约束点，也即拼接约束点在第一目标曲线上，一阶偏导数约束值作为一阶偏导数约束，用于使第一目标曲线在拼接约束点处的一阶偏导数与第二目标曲线相同，从而使得第一目标曲线与第二目标曲线能够连续平滑拼接。
示例性的，拼接约束点表示为(xc,yc)，第一曲线参数为C10、C11、C12、C13，第二曲线参数为C20、C21、C22、C23，则第一车道线曲线为：
x＝C10+C11y+C12y2+C13y3
第二车道线曲线为：
x＝C20+C21y+C22y2+C23y3
约束量表示为：
其中，表示纵向坐标值yc处的一阶偏导数约束值，该一阶偏导数约束值可以基于第一车道线曲线在yc处的一阶偏导数和第二车道线曲线在yc处的一阶偏导数确定，具体可以根据实际需求设置，比如取两个一阶偏导数的均值，表示如下：
其中，表示第一车道线曲线在y＝yc处的一阶偏导数，表示第二车道线曲线在y＝yc处的一阶偏导数，表示如下：

在第一车道线曲线参数、第二车道线曲线参数及yc确定的情况下，即可确定一阶偏导数约束值。
步骤20311与步骤20312不分先后顺序。
在一个可选示例中，该步骤20312可以由处理器调用存储器存储的相应指令执行，也可以由被处理器运行的第二确定子单元执行。
步骤20313，确定从初始状态量到约束量的初始转移矩阵。
其中，初始状态转移矩阵表示从初始状态量到约束量的转移关系。
示例性的，初始转移矩阵表示如下：
在一个可选示例中，该步骤20313可以由处理器调用存储器存储的相应指令执行，也可以由被处理器运行的第三确定子单元执行。
步骤20314，基于初始转移矩阵，确定卡尔曼增益。
其中，卡尔曼增益表征了观测偏差的加权，可以表示为Kt，用于更新状态量，通过迭代调整使得状态量满足约束量的约束，卡尔曼增益的具体确定方式可以根据实际需求设置。
在一个可选示例中，该步骤20314可以由处理器调用存储器存储的相应指令执行，也可以由被处理器运行的第四确定子单元执行。
步骤20315，基于卡尔曼增益、约束量、初始转移矩阵及初始状态量，确定目标状态量。
其中，在确定了卡尔曼增益后，可以基于卡尔曼增益、约束量、初始转移矩阵、初始状态量及预设更新公式实现状态量的更新，获得更新后的状态量，若更新后的状态量能够满足预设条件或者达到预设迭代次数，则可以将更新后的状态量作为目标状态量，否则，基于上述过程继续进行状态量的更新，直至更新后的状态量满足预设条件或者达到预设迭代次数，获得目标状态量。
在一个可选示例中，该步骤20315可以由处理器调用存储器存储的相应指令执行，也可以由被处理器运行的第五确定子单元执行。
步骤20316，基于目标状态量，确定第一目标曲线参数。
其中，目标状态量包括的各状态值即为更新后的第一目标曲线参数，比如目标状态量可以表示如下：
其中，N表示经过N次迭代，则第一目标曲线参数即为
在一个可选示例中，该步骤20316可以由处理器调用存储器存储的相应指令执行，也可以由被处理器运行的第六确定子单元执行。
在一个可选示例中，图5是本公开一示例性实施例提供的步骤2041的流程示意图。在本示例中，步骤2041的，基于拼接约束点，采用带约束的扩展卡尔曼滤波算法，对第二曲线参数进行调整，获得第二目标曲线参数，包括：
步骤20411，基于第二曲线参数确定初始状态量。
步骤20412，基于拼接约束点、第一曲线参数和第二曲线参数，确定约束量，约束量包括拼接约束点的横向坐标值和在拼接约束点的纵向坐标值处的一阶偏导数约束值。
步骤20413，确定从初始状态量到约束量的初始转移矩阵。
步骤20414，基于初始转移矩阵，确定卡尔曼增益。
步骤20415，基于卡尔曼增益、约束量、初始转移矩阵及初始状态量，确定目标状态量。
步骤20416，基于目标状态量，确定第二目标曲线参数。
该示例中步骤20411至步骤20416的具体操作参见前述步骤20311至步骤20316，不同之处在于，该示例中初始状态量是第二车道线曲线的第二曲线参数，具体原理不再一一赘述。
在一个可选示例中，步骤20411至步骤20416可以由处理器调用存储器存储的相应指令执行，也可以由被处理器运行的第二处理单元执行。
在一个可选示例中，步骤20315的基于卡尔曼增益、约束量、初始转移矩阵及初始状态量，确定目标状态量，包括：基于卡尔曼增益、约束量、初始转移矩阵，对初始状态量进行迭代更新，直至满足预设迭代结束条件，获得目标状态量。
其中，预设迭代结束条件可以包括最大迭代次数和/或协方差矩阵需要满足的条件，具体可以根据实际需求设置。
在一个可选示例中，图6是本公开一示例性实施例提供的状态量更新迭代流程示意图。在本示例中，基于卡尔曼增益、约束量、初始转移矩阵，对初始状态量进行迭代更新，直至满足预设迭代结束条件，获得目标状态量，包括：
在迭代过程中，将任一次迭代作为当前迭代，执行以下步骤：
步骤301，基于前一次迭代获得的新状态量，确定第一曲线参数对应的当前曲线参数。
其中，对于当前迭代，前一次迭代获得的新状态量包括前一次更新获得的第一曲线参数对应的更新后的曲线参数，将该更新后的曲线参数作为当前迭代的当前曲线参数。若当前迭代为第1次迭代，则前一次迭代获得的新状态量为初始状态量。
步骤302，基于当前曲线参数，确定当前状态量。
示例性的，当前状态量表示为：
其中，i表示当前迭代为第i次迭代。
在实际应用中，可以将前一次迭代获得的新状态量作为当前迭代的当前状态量。
步骤303，确定从当前状态量到约束量的当前转移矩阵。
其中，约束量可以是确定不变的，参见上述C_s。由于状态量是表示曲线参数，曲线参数的变化对转移矩阵不影响，因此当前转移矩阵与初始转移矩阵是相同的。
示例性的，当前转移矩阵表示如下：
其中，各符号含义如上。
步骤304，基于当前转移矩阵、前一次迭代获得的新协方差矩阵、及预设约束量噪声矩阵，确定当前卡尔曼增益。
其中，协方差矩阵为对角矩阵，初始协方差矩阵可以是预设的对角矩阵，在迭代过程中不断更新直至收敛。预设约束量噪声矩阵表示对约束量的信任程度。预设约束量噪声矩阵为对角矩阵，当其值全为0时表示硬约束，即通过一次迭代获得第一目标曲线参数，当其对角值不为0时，表示软约束，可以通过最大迭代次数控制迭代的结束，具体可以根据实际需求设置。当前卡尔曼矩阵可以基于预设卡尔曼增益公式确定。若当前迭代为第1次迭代，前一次迭代获得的新协方差矩阵为预设初始协方差矩阵。
示例性的，初始协方差矩阵表示为：
其中，λ1,0-λ4,0为预设值。
前一次迭代获得的新协方差矩阵表示为：
其中，λ1,i-1-λ4,i-1为前一次迭代获得的值。
预设约束量噪声矩阵C_n0表示为：
其中，β1和β2为预设值，比如β1和β2均设置为0.001或0，具体可以根据实际需求设置。
则当前卡尔曼增益Kti表示如下：
其中，上角标T表示转置。
步骤305，基于当前卡尔曼增益、约束量、当前转移矩阵，对当前状态量进行更新，获得当前迭代的新状态量。
示例性的，状态更新公式表示如下：
S1i＝S1i-1+Kti(C_s-C_ti*S1i-1)
其中，S1i表示当前迭代的新状态量，其他各符号表示的含义参见前述内容。若当前迭代为第1次迭代，当前状态量为初始状态量S10。
步骤306，基于当前卡尔曼增益和当前转移矩阵，对前一次迭代获得的新协方差矩阵进行更新，获得当前迭代的新协方差矩阵。
示例性的，协方差矩阵更新公式表示如下：
E_ci＝E_ci-1-Kt*C_ti*E_ci-1
其中，E_ci表示当前迭代获得的新协方差矩阵。
步骤307，响应于当前迭代达到预设迭代次数，和/或当前迭代的新协方差矩阵满足预设条件，结束迭代过程，将当前迭代的新状态量作为目标状态量。
其中，预设迭代次数和预设条件均可以根据实际需求设置，用于控制迭代过程的结束。当迭代过程结束后，获得目标状态量。
可以理解地，第二车道线曲线的状态更新迭代流程与上述步骤301-307类似，具体参见上述内容，在此不再一一赘述。
需要说明的是，本公开实施例中的具体示例以三次曲线为例，在实际应用中，待拼接的第一车道线曲线和第二车道线曲线还可以是其他次方程的曲线，比如直线或更高次的曲线，针对不同类型的车道线曲线，可以根据其曲线参数的数量确定相应数量的状态量及其他需要的数据，具体不做限定。比如4次曲线，曲线参数包括5个系数，则相应初始状态量包括5个状态值，具体可以根据实际需求设置。
在一个可选示例中，上述步骤301至步骤307可以由处理器调用存储器存储的相应指令执行，也可以由被处理器运行的第五确定子单元执行。
在一个可选示例中，步骤20314的基于初始转移矩阵，确定卡尔曼增益，包括：基于初始转移矩阵、预设初始协方差矩阵、及预设约束量噪声矩阵，确定卡尔曼增益，预设初始协方差矩阵为对角矩阵，预设约束量噪声矩阵为对角矩阵，用于表示约束量的置信度。
具体卡尔曼增益的确定原理参见前述步骤304的具体说明，本示例是表示第1次迭代(即i＝1)过程中卡尔曼增益的确定，在此不再赘述。
本公开实施例通过带约束的扩展卡尔曼滤波算法，实现待拼接的第一车道线曲线和第二车道线曲线的曲线参数调整，通过约束量及状态量的不断更新，实现两车道线曲线逐渐靠近拼接约束点，直至两车道线曲线能够在拼接约束点处连续平滑连接，使得拼接的车道线更加符合真实车道线的连续平滑状态。
在一个可选示例中，图7是本公开一示例性实施例提供的约束量的确定流程示意图。在本示例中，步骤20412的基于拼接约束点、第一曲线参数和第二曲线参数，确定约束量，包括：
a、基于第一曲线参数，确定第一车道线曲线在拼接约束点的纵向坐标值处的第一一阶偏导数。
示例性的，第一一阶偏导数表示如下：
其中，yc表示拼接约束点的纵向坐标值，第一曲线参数为C10、C11、C12、C13。
b、基于第二曲线参数，确定第二车道线曲线在拼接约束点的纵向坐标值处的第二一阶偏导数。
示例性的，第二一阶偏导数表示如下：
其中，yc表示拼接约束点的纵向坐标值，第二曲线参数为C20、C21、C22、C23。
c、基于第一一阶偏导数和第二一阶偏导数，确定一阶偏导数约束值。
示例性的，一阶偏导数约束值表示如下：
d、基于拼接约束点的横向坐标值和一阶偏导数约束值，确定约束量。
示例性的，约束量表示如下：
本公开实施例通过约束量中的拼接约束点的横向坐标值约束调整过程，使得调整后的第一目标曲线与第二目标曲线在拼接约束点处连续拼接，通过约束量中的一阶偏导数约束值的约束调整，使得调整后的第一目标曲线和第二目标曲线在拼接约束点处的一阶偏导数均等于一阶偏导数约束值，也即第一目标曲线和第二目标曲线在拼接约束点处的一阶偏导数相等，从而实现第一目标曲线和第二目标曲线在拼接约束点处平滑连接，因此，通过约束量的约束，使得获得的第一目标曲线和第二目标曲线能够连续平滑地进行拼接，进一步提高拼接效果，使得拼接后的车道线更加符合真实车道线的状态。
在一个可选示例中，步骤202的基于第一车道线曲线和第二车道线曲线，确定拼接约束点，包括：
步骤2021，基于第一车道线曲线和第二车道线曲线，确定拼接约束点的纵向坐标值。
具体的，拼接约束点的纵向坐标值yc可以基于第一车道线曲线与第二车道线曲线的端点的纵坐标确定，也可以基于第一车道线曲线与第二车道线曲线的交叠区域或间隙区域确定，具体可以根据实际需求设置。
在一个可选示例中，该步骤2021可以由处理器调用存储器存储的相应指令执行，也可以由被处理器运行的第一确定单元执行。
步骤2022，基于纵向坐标值，确定第一车道线曲线在纵向坐标值处的第一横向坐标值。
在确定了拼接约束点的纵向坐标值后，可以基于该纵向坐标值确定第一车道线曲线在该纵向坐标值处的第一横向坐标值x1，比如可以表示如下：
x1＝C10+C11yc+C12yc 2+C13yc 3
其中，C10、C11、C12、C13为第一车道线曲线的第一曲线参数。
在一个可选示例中，该步骤2022可以由处理器调用存储器存储的相应指令执行，也可以由被处理器运行的第二确定单元执行。
步骤2023，基于纵向坐标值，确定第二车道线曲线在纵向坐标值处的第二横向坐标值。
示例性的，第二横向坐标值表示如下：
x2＝C20+C21yc+C22yc 2+C23yc 3
其中，C20、C21、C22、C23为第二车道线曲线的第二曲线参数。
需要说明的是步骤2022和步骤2023不分先后顺序。
在一个可选示例中，该步骤2023可以由处理器调用存储器存储的相应指令执行，也可以由被处理器运行的第三确定单元执行。
步骤2024，基于第一横向坐标值和第二横向坐标值，确定拼接约束点的横向坐标值。
需要说明的是，由于拼接约束点的纵向坐标值是基于第一车道线曲线和第二车道线曲线确定的，当第一车道线曲线与第二车道线曲线在纵向上不存在交叠情况而是具有一定间隙时，拼接约束点可能不在第一车道线曲线上和第二车道线曲线上，而是处于两者之间的间隙部分，因此，确定的第一横向坐标值和第二横向坐标值是通过将相应的车道线曲线在纵向方向延伸到拼接约束点的纵向坐标值处获得，获得的第一横向坐标值和第二横向坐标值可能不相等，或者当两车道线曲线在纵向上存在交叠情况时，基于交叠区域确定出纵向坐标值，或者基于交叠的两端点确定出纵向坐标值，确定的第一横向坐标值和第二横向坐标值分别为两车道线曲线在该纵向坐标值处的截断点的横向坐标值，也可能导致两车道线曲线在该纵向坐标值处的第一横向坐标值和第二横向坐标值不相同。因此，需要基于第一横向坐标值和第二横向坐标值，确定出两车道线曲线的拼接约束点的横向坐标值。
示例性的，图8是本公开一示例性实施例提供的第一车道线曲线和第二车道线曲线纵向方向交叠情况示意图，其中，(x1,yc)表示第一车道线曲线在y＝yc处的截断点，(x2,yc)表示第二车道线曲线在y＝yc处的截断点。图9是本公开一示例性实施例提供的第一车道线曲线和第二车道线曲线纵向方向间 隙情况示意图，其中，(x1,yc)表示第一车道线曲线在y＝yc处的延伸点，(x2,yc)表示第二车道线曲线在y＝yc处的延伸点。拼接约束点的横向坐标值xc可以表示如下：
xc＝(x1+x2)/2
在一个可选示例中，该步骤2024可以由处理器调用存储器存储的相应指令执行，也可以由被处理器运行的第四确定单元执行。
步骤2025，基于拼接约束点的横向坐标值和纵向坐标值，确定拼接约束点。
综上，获得的拼接约束点为(xc,yc)。
在一个可选示例中，该步骤2025可以由处理器调用存储器存储的相应指令执行，也可以由被处理器运行的第五确定单元执行。
本公开实施例通过根据车道线在纵向方向上的延伸特性，先确定出拼接约束点的纵向坐标值，进而基于两车道线曲线在该纵向坐标值处的横向坐标值的均值确定拼接约束点的横向坐标值，使得拼接约束点能够在曲线参数的调整过程中约束两车道线曲线向拼接约束点靠近，直至两车道线曲线能够在拼接约束点连续平滑连接，有助于提高调整效果，可以避免出现两车道线中的一个车道线曲线调整过大而导致该车道线准确性降低的情况。
在一个可选示例中，图10是本公开一示例性实施例提供的步骤2021的流程示意图。基于第一车道线曲线和第二车道线曲线，确定拼接约束点的纵向坐标值，包括：
步骤20211，确定第一车道线曲线的第一端点纵坐标和第二端点纵坐标，以及第二车道线曲线的第三端点纵坐标和第四端点纵坐标。
其中，第一端点纵坐标和第二端点纵坐标分别为第一车道线曲线的第一起始点的纵坐标ys1和第一终止点的纵坐标ye1，第三端点纵坐标和第四端点纵坐标分别为第二车道线曲线的第二起始点的纵坐标ys2和第二终止点的纵坐标ye2。各端点纵坐标可以是从第一车道线曲线和第二车道线曲线的坐标范围中获得，具体不再赘述。
在一个可选示例中，该步骤20211可以由处理器调用存储器存储的相应指令执行，也可以由被处理器运行的第一确定单元执行。
步骤20212，基于第一端点纵坐标、第二端点纵坐标、第三端点纵坐标和第四端点纵坐标，确定第一车道线曲线和第二车道线曲线相邻的第一目标端点纵坐标和第二目标端点纵坐标。
具体的，可以先通过各端点坐标确定出两车道线曲线之间的位置关系，比如交叠或间隙。具体可以通过纵坐标范围来确定或者通过比较各端点纵坐标的大小确定。进而可以确定出两车道线曲线需要拼接的端点纵坐标即为第一目标端点纵坐标和第二目标端点纵坐标。
示例性的，若ys1<ye1<ys2<ye2，或者ys1<ys2<ye1<ye2，则ye1与ys2分别为第一目标端点纵坐标和第二目标端点纵坐标。
在一个可选示例中，该步骤20212可以由处理器调用存储器存储的相应指令执行，也可以由被处理器运行的第一确定单元执行。
步骤20213，基于第一目标端点纵坐标和第二目标端点纵坐标，确定拼接约束点的纵向坐标值。
示例性的，确定了第一目标端点纵坐标和第二目标端点纵坐标，可以将第一目标端点纵坐标和第二目标端点纵坐标的均值作为拼接约束点的纵向坐标值。
在一个可选示例中，该步骤20213可以由处理器调用存储器存储的相应指令执行，也可以由被处理器运行的第一确定单元执行。
在一个可选示例中，步骤201的确定第一坐标系下的待拼接的第一车道线曲线和第二车道线曲线，包括：
步骤2011，获取至少两个视角分别对应的第一图像数据。
其中，第一图像数据可以是通过车辆上设置的至少两个视角的相机采集获得。
示例性的，在车辆行驶过程中，车辆上设置的环视相机可以实时采集周围道路环境各视角分别对应的第一图像数据，该至少两个视角分别对应的第一图像数据可以为各相机同一时间采集获得。比如车辆行驶的当前时刻各相机分别采集获得的第一图像数据。也可以是预先采集获得并存储的同一时间的至少两个视角的第一图像数据，具体不做限定。
在一个可选示例中，该步骤2011可以由处理器调用存储器存储的相应指令执行，也可以由被处理器运行的第一获取单元执行。
步骤2012，基于各第一图像数据，确定各第一图像数据分别对应的图像坐标系下的车道线区域。
其中，图像坐标系下的车道线区域包括图像坐标系下属于车道线的像素集，可以通过预先训练获得的目标检测模型，检测各第一图像数据分别对应的图像坐标系下的车道线区域，具体不再赘述。
在一个可选示例中，该步骤2012可以由处理器调用存储器存储的相应指令执行，也可以由被处理 器运行的第六确定单元执行。
步骤2013，将各第一图像数据分别对应的车道线区域转换到第一坐标系下，获得各第一图像数据分别对应的第一坐标系下的车道线坐标点。
其中，第一坐标系可以为鸟瞰视角(BEV)对应的坐标系。由于各视角的车道线区域是基于各自的图像坐标系的区域，为了实现各视角的融合，需要将其转换到统一的坐标系下，本公开实施例为了利用车道线纵向延伸性确定出拼接约束点，第一坐标系采用鸟瞰视角对应的坐标系，坐标原点为车辆中心，将各视角的车道线曲线转换到统一的第一坐标系下，以能够将各视角的车道线进行融合，获得车辆周围的环视车道线图。图像坐标系到第一坐标系的转换可以根据两坐标系间的映射关系实现，具体原理不再赘述。
在一个可选示例中，该步骤2013可以由处理器调用存储器存储的相应指令执行，也可以由被处理器运行的第三处理单元执行。
步骤2014，基于各第一图像数据分别对应的第一坐标系下的车道线坐标点，确定各第一图像数据分别对应的第一坐标系下的车道线曲线。
其中，图像坐标系下的车道线区域中的像素被转换到第一坐标系下的车道线坐标点，进而可以通过拟合获得各第一图像数据分别对应的第一坐标系下的车道线曲线。具体拟合原理不再赘述。
在一个可选示例中，该步骤2014可以由处理器调用存储器存储的相应指令执行，也可以由被处理器运行的第七确定单元执行。
步骤2015，基于各第一图像数据分别对应的第一坐标系下的车道线曲线，确定第一坐标系下的待拼接的第一车道线曲线和第二车道线曲线。
其中，两个车道线曲线是否可以拼接，可以采用预设匹配规则对第一坐标系下的多个车道线曲线两两之间进行匹配确定，其中预设匹配规则可以根据车道线的特性来设置，比如在鸟瞰图坐标系下，不同视角的车道线曲线在纵向方向(y方向)延伸，在横向方向距离相对很小，依此设置相应的匹配规则用于确定出需要拼接的两个车道线曲线。
比如，可以对两车道线曲线中的一个车道线曲线进行采样，获得多个采样点，该多个采样点与另一个车道线曲线的距离小于一定阈值，以及两车道线曲线的一次项系数之间差距(比如上述第一车道线曲线的C11与第二车道线曲线的C21之间的差距)满足一定条件，具体匹配原理不再赘述。
在一个可选示例中，该步骤2015可以由处理器调用存储器存储的相应指令执行，也可以由被处理器运行的第八确定单元执行。
本公开实施例提供的车道线的拼接方法，通过确定拼接约束点，采用带约束的扩展卡尔曼滤波算法对两车道线在拼接约束点处的连续性和平滑性进行约束，从而使得拼接获得的车道线在拼接点处连续且平滑；且采用带约束的扩展卡尔曼滤波算法延时和计算量均较低，能够保证车道线实时有效的拼接，为后续应用提供更好的环视车道线数据；此外，由于本公开实施例各视角的相机独立地感知车道线，在融合过程中对于同一车道线的不同线段，可以通过拼接获得完整的车道线，而拼接约束点的确定，使得不同视角的车道线可以延伸或截断，从而使得各相机具有一定的容错率，当某个或某几个相机出现故障时，通过其他相机的同一车道线的线段延伸拼接即可获得中间部分的车道线，从而可以不影响整体车道线的输出。
本公开上述各实施例或可选示例可以单独实施也可以在不冲突的情况下以任意组合方式结合实施，具体可以根据实际需求设置，本公开不作限定。
本公开实施例提供的任一种车道线的拼接方法可以由任意适当的具有数据处理能力的设备执行，包括但不限于：终端设备和服务器等。或者，本公开实施例提供的任一种车道线的拼接方法可以由处理器执行，如处理器通过调用存储器存储的相应指令来执行本公开实施例提及的任一种车道线的拼接方法。下文不再赘述。
本领域普通技术人员可以理解：实现上述方法实施例的全部或部分步骤可以通过程序指令相关的硬件来完成，前述的程序可以存储于一计算机可读取存储介质中，该程序在执行时，执行包括上述方法实施例的步骤；而前述的存储介质包括：ROM、RAM、磁碟或者光盘等各种可以存储程序代码的介质。
示例性装置
图11是本公开一示例性实施例提供的车道线的拼接装置的结构示意图。该实施例的装置可用于实现本公开相应的方法实施例，如图11所示的装置包括：第一确定模块501、第一处理模块502、第二处理模块503、第三处理模块504和第四处理模块505。
第一确定模块501，用于确定第一坐标系下的待拼接的第一车道线曲线和第二车道线曲线；第一处理模块502，用于基于第一确定模块501确定的第一车道线曲线和第二车道线曲线，确定拼接约束点； 第二处理模块503，用于基于第一处理模块502确定的拼接约束点，对第一车道线曲线对应的第一曲线参数进行调整，获得第一曲线参数对应的调整后的第一目标曲线参数；第三处理模块504，用于基于第一处理模块502确定的拼接约束点，对第二车道线曲线对应的第二曲线参数进行调整，获得第二曲线参数对应的调整后的第二目标曲线参数；第四处理模块505，用于基于第一目标曲线参数和第二目标曲线参数，确定拼接后的车道线。
图12是本公开另一示例性实施例提供的车道线的拼接装置的结构示意图。
在一个可选示例中，第二处理模块503包括：第一处理单元5031，用于基于拼接约束点，采用带约束的扩展卡尔曼滤波算法，对第一曲线参数进行调整，获得第一目标曲线参数。
在一个可选示例中，第三处理模块504包括：第二处理单元5041，用于基于拼接约束点，采用带约束的扩展卡尔曼滤波算法，对第二曲线参数进行调整，获得第二目标曲线参数。
在一个可选示例中，图13是本公开一示例性实施例提供的第一处理单元5031的结构示意图，在本示例中，第一处理单元5031包括：第一确定子单元50311、第二确定子单元50312、第三确定子单元50313、第四确定子单元50314、第五确定子单元50315和第六确定子单元50316。
第一确定子单元50311，用于基于第一曲线参数确定初始状态量；第二确定子单元50312，用于基于拼接约束点、第一曲线参数和第二曲线参数，确定约束量，约束量包括拼接约束点的横向坐标值和在拼接约束点的纵向坐标值处的一阶偏导数约束值；第三确定子单元50313，用于确定从初始状态量到约束量的初始转移矩阵；第四确定子单元50314，用于基于初始转移矩阵，确定卡尔曼增益；第五确定子单元50315，用于基于卡尔曼增益、约束量、初始转移矩阵及初始状态量，确定目标状态量；第六确定子单元50316，用于基于目标状态量，确定第一目标曲线参数。
在一个可选示例中，第五确定子单元50315具体用于：基于卡尔曼增益、约束量、初始转移矩阵，对初始状态量进行迭代更新，直至满足预设迭代结束条件，获得目标状态量。
在一个可选示例中，第五确定子单元50315具体用于：在迭代过程中，将任一次迭代作为当前迭代，执行以下步骤：基于前一次迭代获得的新状态量，确定第一曲线参数对应的当前曲线参数；基于当前曲线参数，确定当前状态量；确定从当前状态量到约束量的当前转移矩阵；基于当前转移矩阵、前一次迭代获得的新协方差矩阵、及预设约束量噪声矩阵，确定当前卡尔曼增益；基于当前卡尔曼增益、约束量、当前转移矩阵，对当前状态量进行更新，获得当前迭代的新状态量；基于当前卡尔曼增益和当前转移矩阵，对前一次迭代获得的新协方差矩阵进行更新，获得当前迭代的新协方差矩阵；响应于当前迭代达到预设迭代次数，和/或当前迭代的新协方差矩阵满足预设条件，结束迭代过程，将当前迭代的新状态量作为目标状态量。
在一个可选示例中，第四确定子单元50314具体用于：基于初始转移矩阵、预设初始协方差矩阵、及预设约束量噪声矩阵，确定卡尔曼增益，预设初始协方差矩阵为对角矩阵，预设约束量噪声矩阵为对角矩阵，用于表示约束量的置信度。
在一个可选示例中，第二确定子单元50312具体用于：基于第一曲线参数，确定第一车道线曲线在拼接约束点的纵向坐标值处的第一一阶偏导数；基于第二曲线参数，确定第二车道线曲线在拼接约束点的纵向坐标值处的第二一阶偏导数；基于第一一阶偏导数和第二一阶偏导数，确定一阶偏导数约束值；基于拼接约束点的横向坐标值和一阶偏导数约束值，确定约束量。
在一个可选示例中，第一处理模块502包括：第一确定单元5021、第二确定单元5022、第三确定单元5023、第四确定单元5024和第五确定单元5025。
第一确定单元5021，用于基于第一车道线曲线和第二车道线曲线，确定拼接约束点的纵向坐标值；第二确定单元5022，用于基于纵向坐标值，确定第一车道线曲线在纵向坐标值处的第一横向坐标值；第三确定单元5023，用于基于纵向坐标值，确定第二车道线曲线在纵向坐标值处的第二横向坐标值；第四确定单元5024，用于基于第一横向坐标值和第二横向坐标值，确定拼接约束点的横向坐标值；第五确定单元5025，用于基于拼接约束点的横向坐标值和纵向坐标值，确定拼接约束点。
在一个可选示例中，第一确定单元5021具体用于：确定第一车道线曲线的第一端点纵坐标和第二端点纵坐标，以及第二车道线曲线的第三端点纵坐标和第四端点纵坐标；基于第一端点纵坐标、第二端点纵坐标、第三端点纵坐标和第四端点纵坐标，确定第一车道线曲线和第二车道线曲线相邻的第一目标端点纵坐标和第二目标端点纵坐标；基于第一目标端点纵坐标和第二目标端点纵坐标，确定拼接约束点的纵向坐标值。
在一个可选示例中，第一确定模块501包括：第一获取单元5011、第六确定单元5012、第三处理单元5013、第七确定单元5014和第八确定单元5015。
第一获取单元5011，用于获取至少两个视角分别对应的第一图像数据；第六确定单元5012，用于基于各第一图像数据，确定各第一图像数据分别对应的图像坐标系下的车道线区域；第三处理单元5013， 用于将各第一图像数据分别对应的车道线区域转换到第一坐标系下，获得各第一图像数据分别对应的第一坐标系下的车道线坐标点；第七确定单元5014，用于基于各第一图像数据分别对应的第一坐标系下的车道线坐标点，确定各第一图像数据分别对应的第一坐标系下的车道线曲线；第八确定单元5015，用于基于各第一图像数据分别对应的第一坐标系下的车道线曲线，确定第一坐标系下的待拼接的第一车道线曲线和第二车道线曲线。
本装置示例性实施例对应的有益技术效果可以参见上述示例性方法部分的相应有益技术效果，在此不再赘述。
示例性电子设备
图14是本公开电子设备一个应用实施例的结构示意图。本实施例中，该电子设备10包括一个或多个处理器11和存储器12。
处理器11可以是中央处理单元(CPU)或者具有数据处理能力和/或指令执行能力的其他形式的处理单元，并且可以控制电子设备10中的其他组件以执行期望的功能。
存储器12可以包括一个或多个计算机程序产品，计算机程序产品可以包括各种形式的计算机可读存储介质，例如易失性存储器和/或非易失性存储器。易失性存储器例如可以包括随机存取存储器(RAM)和/或高速缓冲存储器(cache)等。非易失性存储器例如可以包括只读存储器(ROM)、硬盘、闪存等。在计算机可读存储介质上可以存储一个或多个计算机程序指令，处理器11可以运行程序指令，以实现上文的本公开的各个实施例的方法以及/或者其他期望的功能。在计算机可读存储介质中还可以存储诸如输入信号、信号分量、噪声分量等各种内容。
在一个示例中，电子设备10还可以包括：输入装置13和输出装置14，这些组件通过总线系统和/或其他形式的连接机构(未示出)互连。
此外，该输入装置13还可以包括例如键盘、鼠标等等。
该输出装置14可以向外部输出各种信息，该输出装置14可以包括例如显示器、扬声器、打印机、以及通信网络及其所连接的远程输出设备等等。
当然，为了简化，图14中仅示出了该电子设备10中与本公开有关的组件中的一些，省略了诸如总线、输入/输出接口等等的组件。除此之外，根据具体应用情况，电子设备10还可以包括任何其他适当的组件。
示例性计算机程序产品和计算机可读存储介质
除了上述方法和设备以外，本公开的实施例还可以是计算机程序产品，其包括计算机程序指令，计算机程序指令在被处理器运行时使得处理器执行本说明书上述“示例性方法”部分中描述的根据本公开各种实施例的方法中的步骤。
计算机程序产品可以以一种或多种程序设计语言的任意组合来编写用于执行本公开实施例操作的程序代码，程序设计语言包括面向对象的程序设计语言，诸如Java、C++等，还包括常规的过程式程序设计语言，诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算设备上执行、部分地在用户设备上执行、作为一个独立的软件包执行、部分在用户计算设备上部分在远程计算设备上执行、或者完全在远程计算设备或服务器上执行。
此外，本公开的实施例还可以是计算机可读存储介质，其上存储有计算机程序指令，计算机程序指令在被处理器运行时使得处理器执行本说明书上述“示例性方法”部分中描述的根据本公开各种实施例的方法中的步骤。
计算机可读存储介质可以采用一个或多个可读介质的任意组合。可读介质可以是可读信号介质或者可读存储介质。可读存储介质例如可以包括但不限于电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。可读存储介质的更具体的例子(非穷举的列表)包括：具有一个或多个导线的电连接、便携式盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、光纤、便携式紧凑盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。
以上结合具体实施例描述了本公开的基本原理，但是，在本公开中提及的优点、优势、效果等仅是示例而非限制，不能认为其是本公开的各个实施例必须具备的。另外，上述公开的具体细节仅是为了示例的作用和便于理解的作用，而非限制，上述细节并不限制本公开为必须采用上述具体的细节来实现。
本领域的技术人员可以对本公开进行各种改动和变型而不脱离本申请的精神和范围。这样，倘若本申请的这些修改和变型属于本公开权利要求及其等同技术的范围之内，则本公开也意图包含这些改动和变型在内。

Claims (14)

  1. 一种车道线的拼接方法，包括：
    确定第一坐标系下的待拼接的第一车道线曲线和第二车道线曲线；
    基于所述第一车道线曲线和所述第二车道线曲线，确定拼接约束点；
    基于所述拼接约束点，对所述第一车道线曲线对应的第一曲线参数进行调整，获得所述第一曲线参数对应的调整后的第一目标曲线参数；
    基于所述拼接约束点，对所述第二车道线曲线对应的第二曲线参数进行调整，获得所述第二曲线参数对应的调整后的第二目标曲线参数；
    基于所述第一目标曲线参数和所述第二目标曲线参数，确定拼接后的车道线。
  2. 根据权利要求1所述的方法，其中，所述基于所述拼接约束点，对所述第一车道线曲线对应的第一曲线参数进行调整，获得所述第一曲线参数对应的调整后的第一目标曲线参数，包括：
    基于所述拼接约束点，采用带约束的扩展卡尔曼滤波算法，对所述第一曲线参数进行调整，获得所述第一目标曲线参数。
  3. 根据权利要求2所述的方法，其中，所述基于所述拼接约束点，采用带约束的扩展卡尔曼滤波算法，对所述第一曲线参数进行调整，获得所述第一目标曲线参数，包括：
    基于所述第一曲线参数确定初始状态量；
    基于所述拼接约束点、所述第一曲线参数和所述第二曲线参数，确定约束量，所述约束量包括所述拼接约束点的横向坐标值和在所述拼接约束点的纵向坐标值处的一阶偏导数约束值；
    确定从所述初始状态量到所述约束量的初始转移矩阵；
    基于所述初始转移矩阵，确定卡尔曼增益；
    基于所述卡尔曼增益、所述约束量、所述初始转移矩阵及所述初始状态量，确定目标状态量；
    基于所述目标状态量，确定所述第一目标曲线参数。
  4. 根据权利要求3所述的方法，其中，所述基于所述卡尔曼增益、所述约束量、所述初始转移矩阵及所述初始状态量，确定目标状态量，包括：
    基于所述卡尔曼增益、所述约束量、所述初始转移矩阵，对所述初始状态量进行迭代更新，直至满足预设迭代结束条件，获得所述目标状态量。
  5. 根据权利要求4所述的方法，其中，所述基于所述卡尔曼增益、所述约束量、所述初始转移矩阵，对所述初始状态量进行迭代更新，直至满足预设迭代结束条件，获得所述目标状态量，包括：
    在迭代过程中，将任一次迭代作为当前迭代，执行以下步骤：
    基于前一次迭代获得的新状态量，确定所述第一曲线参数对应的当前曲线参数；
    基于所述当前曲线参数，确定当前状态量；
    确定从所述当前状态量到所述约束量的当前转移矩阵；
    基于所述当前转移矩阵、前一次迭代获得的新协方差矩阵、及预设约束量噪声矩阵，确定当前卡尔曼增益；
    基于所述当前卡尔曼增益、所述约束量、所述当前转移矩阵，对所述当前状态量进行更新，获得当前迭代的新状态量；
    基于所述当前卡尔曼增益和所述当前转移矩阵，对所述前一次迭代获得的新协方差矩阵进行更新，获得当前迭代的新协方差矩阵；
    响应于当前迭代达到预设迭代次数，和/或所述当前迭代的新协方差矩阵满足预设条件，结束迭代过程，将所述当前迭代的新状态量作为所述目标状态量。
  6. 根据权利要求3所述的方法，其中，所述基于所述初始转移矩阵，确定卡尔曼增益，包括：
    基于所述初始转移矩阵、预设初始协方差矩阵、及预设约束量噪声矩阵，确定所述卡尔曼增益，所述预设初始协方差矩阵为对角矩阵，所述预设约束量噪声矩阵为对角矩阵，用于表示所述约束量的置信 度。
  7. 根据权利要求3所述的方法，其中，所述基于所述拼接约束点、所述第一曲线参数和所述第二曲线参数，确定约束量，包括：
    基于所述第一曲线参数，确定所述第一车道线曲线在所述拼接约束点的纵向坐标值处的第一一阶偏导数；
    基于所述第二曲线参数，确定所述第二车道线曲线在所述拼接约束点的纵向坐标值处的第二一阶偏导数；
    基于所述第一一阶偏导数和所述第二一阶偏导数，确定所述一阶偏导数约束值；
    基于所述拼接约束点的横向坐标值和所述一阶偏导数约束值，确定所述约束量。
  8. 根据权利要求1所述的方法，其中，所述基于所述拼接约束点，对所述第二车道线曲线对应的第二曲线参数进行调整，获得所述第二曲线参数对应的调整后的第二目标曲线参数，包括：
    基于所述拼接约束点，采用带约束的扩展卡尔曼滤波算法，对所述第二曲线参数进行调整，获得所述第二目标曲线参数。
  9. 根据权利要求1所述的方法，其中，所述基于所述第一车道线曲线和所述第二车道线曲线，确定拼接约束点，包括：
    基于所述第一车道线曲线和所述第二车道线曲线，确定所述拼接约束点的纵向坐标值；
    基于所述纵向坐标值，确定所述第一车道线曲线在所述纵向坐标值处的第一横向坐标值；
    基于所述纵向坐标值，确定所述第二车道线曲线在所述纵向坐标值处的第二横向坐标值；
    基于所述第一横向坐标值和所述第二横向坐标值，确定所述拼接约束点的横向坐标值；
    基于所述拼接约束点的所述横向坐标值和所述纵向坐标值，确定所述拼接约束点。
  10. 根据权利要求9所述的方法，其中，所述基于所述第一车道线曲线和所述第二车道线曲线，确定所述拼接约束点的纵向坐标值，包括：
    确定所述第一车道线曲线的第一端点纵坐标和第二端点纵坐标，以及所述第二车道线曲线的第三端点纵坐标和第四端点纵坐标；
    基于所述第一端点纵坐标、所述第二端点纵坐标、所述第三端点纵坐标和所述第四端点纵坐标，确定所述第一车道线曲线和所述第二车道线曲线相邻的第一目标端点纵坐标和第二目标端点纵坐标；
    基于所述第一目标端点纵坐标和所述第二目标端点纵坐标，确定所述拼接约束点的纵向坐标值。
  11. 根据权利要求1-10任一所述的方法，其中，所述确定第一坐标系下的待拼接的第一车道线曲线和第二车道线曲线，包括：
    获取至少两个视角分别对应的第一图像数据；
    基于各所述第一图像数据，确定各所述第一图像数据分别对应的图像坐标系下的车道线区域；
    将各所述第一图像数据分别对应的所述车道线区域转换到所述第一坐标系下，获得各所述第一图像数据分别对应的所述第一坐标系下的车道线坐标点；
    基于各所述第一图像数据分别对应的所述第一坐标系下的所述车道线坐标点，确定各所述第一图像数据分别对应的所述第一坐标系下的车道线曲线；
    基于各所述第一图像数据分别对应的所述第一坐标系下的所述车道线曲线，确定所述第一坐标系下的待拼接的所述第一车道线曲线和所述第二车道线曲线。
  12. 一种车道线的拼接装置，包括：
    第一确定模块，用于确定第一坐标系下的待拼接的第一车道线曲线和第二车道线曲线；
    第一处理模块，用于基于所述第一车道线曲线和所述第二车道线曲线，确定拼接约束点；
    第二处理模块，用于基于所述拼接约束点，对所述第一车道线曲线对应的第一曲线参数进行调整，获得所述第一曲线参数对应的调整后的第一目标曲线参数；
    第三处理模块，用于基于所述拼接约束点，对所述第二车道线曲线对应的第二曲线参数进行调整，获得所述第二曲线参数对应的调整后的第二目标曲线参数；
    第四处理模块，用于基于所述第一目标曲线参数和所述第二目标曲线参数，确定拼接后的车道线。
  13. 一种计算机可读存储介质，所述存储介质存储有计算机程序，所述计算机程序用于执行上述权利要求1-11任一所述的车道线的拼接方法。
  14. 一种电子设备，所述电子设备包括：
    处理器；
    用于存储所述处理器可执行指令的存储器；
    所述处理器，用于从所述存储器中读取所述可执行指令，并执行所述指令以实现上述权利要求1-11任一所述的车道线的拼接方法。
PCT/CN2023/118652 2022-11-09 2023-09-13 车道线的拼接方法、装置、电子设备和存储介质 WO2024098953A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202211401951.0A CN115641266A (zh) 2022-11-09 2022-11-09 车道线的拼接方法、装置、电子设备和存储介质
CN202211401951.0 2022-11-09

Publications (1)

Publication Number Publication Date
WO2024098953A1 true WO2024098953A1 (zh) 2024-05-16

Family

ID=84948176

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/118652 WO2024098953A1 (zh) 2022-11-09 2023-09-13 车道线的拼接方法、装置、电子设备和存储介质

Country Status (2)

Country Link
CN (1) CN115641266A (zh)
WO (1) WO2024098953A1 (zh)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115641266A (zh) * 2022-11-09 2023-01-24 南京地平线集成电路有限公司 车道线的拼接方法、装置、电子设备和存储介质
CN117726533B (zh) * 2024-02-07 2024-05-31 腾讯科技（深圳）有限公司 一种接边处理方法和相关装置

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2021056341A1 (zh) * 2019-09-26 2021-04-01 深圳市大疆创新科技有限公司 车道线融合方法、车道线融合装置、车辆和存储介质
CN114283398A (zh) * 2021-12-21 2022-04-05 北京百度网讯科技有限公司 车道线的处理方法、装置和电子设备
CN115131757A (zh) * 2022-06-29 2022-09-30 上海安亭地平线智能交通技术有限公司 车道线的确定方法、装置、电子设备和存储介质
CN115290105A (zh) * 2022-07-20 2022-11-04 高德软件有限公司 虚拟车道线的生成方法、装置及电子设备
CN115641266A (zh) * 2022-11-09 2023-01-24 南京地平线集成电路有限公司 车道线的拼接方法、装置、电子设备和存储介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2021056341A1 (zh) * 2019-09-26 2021-04-01 深圳市大疆创新科技有限公司 车道线融合方法、车道线融合装置、车辆和存储介质
CN114283398A (zh) * 2021-12-21 2022-04-05 北京百度网讯科技有限公司 车道线的处理方法、装置和电子设备
CN115131757A (zh) * 2022-06-29 2022-09-30 上海安亭地平线智能交通技术有限公司 车道线的确定方法、装置、电子设备和存储介质
CN115290105A (zh) * 2022-07-20 2022-11-04 高德软件有限公司 虚拟车道线的生成方法、装置及电子设备
CN115641266A (zh) * 2022-11-09 2023-01-24 南京地平线集成电路有限公司 车道线的拼接方法、装置、电子设备和存储介质

Also Published As

Publication number Publication date
CN115641266A (zh) 2023-01-24

Similar Documents

Publication Publication Date Title
WO2024098953A1 (zh) 车道线的拼接方法、装置、电子设备和存储介质
WO2019165949A1 (zh) 图像处理方法、设备、存储介质和计算机程序产品
KR102502651B1 (ko) 지도를 구축하기 위한 방법 및 장치
WO2021189889A1 (zh) 场景图像中的文本检测方法、装置、计算机设备及存储介质
CN113486797A (zh) 无人车位置检测方法、装置、设备、存储介质及车辆
CN113869293B (zh) 车道线识别方法、装置、电子设备和计算机可读介质
WO2022198994A1 (zh) 一种机械臂运动规划方法、装置、可读存储介质及机械臂
WO2021114990A1 (zh) 人脸畸变校正方法、装置、电子设备及存储介质
WO2022160787A1 (zh) 一种机器人手眼标定方法, 装置, 可读存储介质及机器人
JP7411114B2 (ja) 空間幾何情報推定モデルの生成方法及び装置
WO2021104394A1 (zh) 图像处理方法及装置、电子设备、存储介质
CN114964296B (zh) 车辆行驶路径规划方法、装置、设备和计算机可读介质
WO2023142864A1 (zh) 初始位姿的确定方法、装置、电子设备和存储介质
WO2023231435A1 (zh) 视觉感知方法、装置、存储介质和电子设备
CN114399589A (zh) 三维车道线生成方法、装置、电子设备和计算机可读介质
WO2022036981A1 (zh) 机器人及其地图构建方法和装置
WO2023184869A1 (zh) 室内停车场的语义地图构建及定位方法和装置
WO2023040813A1 (zh) 人脸图像处理方法、装置、设备及介质
WO2024011557A1 (zh) 地图构建方法、装置及存储介质
WO2022126478A1 (zh) 影像获取方法、装置、可移动平台、控制终端、系统
WO2023168747A1 (zh) 基于域控制器平台的自动泊车的停车位标注方法及装置
CN112232451B (zh) 多传感器数据融合方法、装置、电子设备和介质
WO2022027191A1 (zh) 平面矫正方法及装置、计算机可读介质和电子设备
CN114708336B (zh) 多相机在线标定方法、装置、电子设备和计算机可读介质
WO2020103501A1 (zh) 拍摄视角的调整方法、装置、电子设备及存储介质