CN111383287B - External parameter calibration method and device for vehicle-mounted sensor - Google Patents

External parameter calibration method and device for vehicle-mounted sensor Download PDF

Info

Publication number
CN111383287B
CN111383287B CN202010095077.7A CN202010095077A CN111383287B CN 111383287 B CN111383287 B CN 111383287B CN 202010095077 A CN202010095077 A CN 202010095077A CN 111383287 B CN111383287 B CN 111383287B
Authority
CN
China
Prior art keywords
vehicle
target
target obstacle
sensor
coordinate system
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202010095077.7A
Other languages
Chinese (zh)
Other versions
CN111383287A (en
Inventor
田玉珍
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ecarx Hubei Tech Co Ltd
Original Assignee
Hubei Ecarx Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hubei Ecarx Technology Co Ltd filed Critical Hubei Ecarx Technology Co Ltd
Priority to CN202010095077.7A priority Critical patent/CN111383287B/en
Publication of CN111383287A publication Critical patent/CN111383287A/en
Application granted granted Critical
Publication of CN111383287B publication Critical patent/CN111383287B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/02—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S13/00
    • G01S7/40—Means for monitoring or calibrating
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
    • G01S7/497—Means for monitoring or calibrating

Abstract

The invention provides an external reference calibration method and device of vehicle-mounted sensors, wherein the method comprises the steps of determining starting positions of target obstacles detected by a plurality of vehicle-mounted sensors; generating a target trajectory corresponding to the target obstacle detected by the vehicle-mounted sensors based on the position of the target obstacle in each frame of detection data from the plurality of vehicle-mounted sensors from the time corresponding to the starting point position of the target obstacle; determining the point trace of the target obstacle detected by the corresponding vehicle-mounted sensor by adopting an interpolation algorithm aiming at the target trace; and determining an external reference relation between any two vehicle-mounted sensors to realize external reference calibration of the vehicle-mounted sensors based on the point traces of the target obstacle detected by any two vehicle-mounted sensors in the plurality of vehicle-mounted sensors. The embodiment of the invention can greatly improve the accuracy and stability of external reference calibration of the vehicle-mounted sensor, and the external reference calibration process does not require that the vehicle is positioned in a fixed place or in a special environment, so that the operability is strong.

Description

External parameter calibration method and device for vehicle-mounted sensor
Technical Field
The invention relates to the technical field of advanced assistant driving and automatic driving, in particular to an external parameter calibration method and device of a vehicle-mounted sensor.
Background
With the development of advanced assistant driving technology and automatic driving technology, the types and the number of vehicle-mounted sensors are continuously increased, meanwhile, the working mode of single function of a single sensor is gradually changed into the working mode of fusion and cooperation of a plurality of sensors to realize more complex functions, and the premise of fusion and cooperation of a plurality of sensors is that all the sensors work under the same coordinate system, namely, the information of the same target output by all the sensors is ensured to be consistent as much as possible. Therefore, the rotational-translational relationship between any two vehicle-mounted sensors needs to be determined through external reference calibration. Examples include millimeter-wave radar and millimeter-wave radar, millimeter-wave radar and laser radar, and millimeter-wave radar and camera rotational-translational relationships.
In general, the relative position relationship between different vehicle-mounted sensors and the center of the rear axle of the vehicle is calibrated before the factory, and the rotational-translational relationship is known, so that the rotational-translational relationship between different vehicle-mounted sensors is also known. However, with the use of the vehicle, the position of the sensor slightly changes, meanwhile, the performance is attenuated under different conditions, and if the translation-rotation transformation relation of the sensor is not calibrated and updated again, the effect of multi-sensor fusion is reduced. Meanwhile, due to the unstable output target, the millimeter wave radar is difficult to calibrate by using a conventional feature extraction method, for example, the commonly used multi-phase external parameter calibration principle cannot be directly applied to external parameter calibration of the millimeter wave radar and other sensors. For example, the calibration method of the millimeter wave radar and the camera in the prior art adopts a method of calibrating longitudinal and transverse parameters respectively, and the calibrated external parameters are obtained by calculating only one set of calibration data, so that the robustness and the accuracy of the calibration method have problems.
Disclosure of Invention
In view of the above, the present invention has been made to provide a method and apparatus for external reference calibration of an in-vehicle sensor that overcomes or at least partially solves the above-mentioned problems.
According to an aspect of the embodiments of the present invention, there is provided an external reference calibration method for a vehicle-mounted sensor, including:
determining starting point positions of target obstacles detected by a plurality of vehicle-mounted sensors;
generating a target track corresponding to the target obstacle detected by the vehicle-mounted sensors based on the position of the target obstacle in each frame of detection data from the plurality of vehicle-mounted sensors from the time corresponding to the starting point position of the target obstacle;
determining the point trace of the target obstacle detected by the corresponding vehicle-mounted sensor by adopting an interpolation algorithm aiming at the target trace;
and determining an external reference relation between any two vehicle-mounted sensors to realize external reference calibration of the vehicle-mounted sensors based on the point traces of the target obstacle detected by any two vehicle-mounted sensors in the plurality of vehicle-mounted sensors.
Optionally, determining the starting position of the target obstacle detected by the plurality of vehicle-mounted sensors includes:
determining whether M frames of detection data exist in the subsequent continuous N frames of detection data to detect the target obstacle from the moment when each vehicle-mounted sensor respectively detects the target obstacle, wherein M is not more than N;
and if M frames of detection data exist in the continuous N frames of detection data to detect the target obstacle, taking the position of the target obstacle of the first frame of detection data in the continuous N frames of detection data as the starting position of the target obstacle detected by the corresponding vehicle-mounted sensor.
Optionally, generating a target trajectory corresponding to the target obstacle detected by the vehicle-mounted sensor based on the position of the target obstacle in each frame of detection data from the plurality of vehicle-mounted sensors from the time corresponding to the starting point position of the target obstacle, includes:
defining a sensor coordinate system corresponding to each vehicle-mounted sensor by taking the central position of each vehicle-mounted sensor at the initial position of the vehicle as the origin of a sensor coordinate system, defining a vehicle coordinate system by taking the central position of a rear axle of the vehicle as the origin of the vehicle coordinate system, and defining the vehicle coordinate system at the moment corresponding to the starting position of the target obstacle as a world coordinate system;
determining coordinate values of the positions of the target obstacles in the detection data of each frame of the plurality of vehicle-mounted sensors in a corresponding sensor coordinate system based on the positions of the target obstacles in the detection data of each frame from the plurality of vehicle-mounted sensors from the time corresponding to the starting positions of the target obstacles;
converting the coordinate value of the position of the target obstacle under the corresponding sensor coordinate system into the coordinate value of the position of the target obstacle under the vehicle coordinate system, and converting the coordinate value of the position of the target obstacle under the vehicle coordinate system into the coordinate value of the position of the target obstacle under the world coordinate system;
and generating a target track of the target obstacle detected by the corresponding vehicle-mounted sensor in the world coordinate system based on the coordinate value of the position of the target obstacle in the world coordinate system.
Optionally, determining, by using an interpolation algorithm for the target trajectory, a point trajectory of the target obstacle detected by the corresponding vehicle-mounted sensor, includes:
and determining the trace points of the target obstacles corresponding to the vehicle-mounted sensors by adopting a cubic spline interpolation method aiming at the target track, wherein the trace points of the target obstacles detected by the plurality of vehicle-mounted sensors comprise the same point number.
Optionally, determining an external reference relationship between any two vehicle-mounted sensors to realize external reference calibration of the vehicle-mounted sensors based on the point traces of the target obstacle detected by any two vehicle-mounted sensors in the plurality of vehicle-mounted sensors, including:
aiming at the point traces of the target obstacle detected by the plurality of vehicle-mounted sensors, respectively adding one-dimensional data in the coordinate values corresponding to the points in the point traces;
and determining an external reference relation between any two vehicle-mounted sensors to realize external reference calibration of the vehicle-mounted sensors based on the point traces of the target obstacle detected by any two vehicle-mounted sensors after the one-dimensional data is added.
Optionally, determining an external reference relationship between any two vehicle-mounted sensors to realize external reference calibration of the vehicle-mounted sensors based on the point traces of the target obstacle detected by any two vehicle-mounted sensors after adding the one-dimensional data, including:
for any two vehicle-mounted sensors, the trace of the target obstacle detected by one of the vehicle-mounted sensors is taken as a main trace;
calculating a change matrix of the trace point of the target obstacle detected by the other vehicle-mounted sensor and the main trace point by using a closest point iterative algorithm according to the point in the main trace point and the point in the trace point of the target obstacle detected by the other vehicle-mounted sensor;
and converting the change matrix as an external reference relation of the coordinate values of the target obstacle detected by one vehicle-mounted sensor in the two vehicle-mounted sensors in the sensor coordinate system to the coordinate values of the other vehicle-mounted sensor in the sensor coordinate system so as to realize external reference calibration of the vehicle-mounted sensors.
According to another aspect of the embodiments of the present invention, there is also provided an external reference calibration apparatus for a vehicle-mounted sensor, including:
the system comprises a first determination module, a second determination module and a control module, wherein the first determination module is suitable for determining starting point positions of target obstacles detected by a plurality of vehicle-mounted sensors;
a generation module adapted to generate a target trajectory corresponding to the target obstacle detected by the vehicle-mounted sensors based on the position of the target obstacle in each frame of detection data from the plurality of vehicle-mounted sensors from a time corresponding to the starting point position of the target obstacle;
the second determination module is suitable for determining the point track of the target obstacle detected by the corresponding vehicle-mounted sensor by adopting an interpolation algorithm aiming at the target track;
the calibration module is suitable for determining the external reference relation between any two vehicle-mounted sensors to realize external reference calibration of the vehicle-mounted sensors based on the point traces of the target obstacle detected by any two vehicle-mounted sensors in the plurality of vehicle-mounted sensors.
According to still another aspect of the embodiments of the present invention, there is also provided an electronic device, including: a processor; a memory storing computer program code; the computer program code, when executed by the processor, causes the electronic device to perform the method for external reference calibration of an in-vehicle sensor of any of the embodiments above.
According to yet another aspect of an embodiment of the present invention, there is also provided a computer storage medium storing computer program code which, when run on a computing device, causes the computing device to perform the method for extrinsic calibration of an in-vehicle sensor of any of the above embodiments.
The external reference calibration process of the vehicle-mounted sensor provided by the embodiment of the invention has no limitation on the site and environment of the vehicle where the vehicle-mounted sensor is located, can perform external reference calibration in real time, and has strong operability. In addition, compared with the problem of poor robustness and accuracy of the currently adopted method for calibrating the longitudinal and transverse parameters respectively, the embodiment of the invention fully considers the characteristics of low resolution, jitter and frame loss of output results of some vehicle-mounted sensors, generates the target track of the target obstacle by adopting multi-frame detection data detected by the vehicle-mounted sensors, determines the trace point of the target obstacle detected by any two vehicle-mounted sensors according to the target track of the target obstacle, and determines the external reference relation between any two vehicle-mounted sensors according to the trace point, thereby greatly improving the accuracy and stability of external reference calibration.
The foregoing description is only an overview of the technical solutions of the present invention, and the embodiments of the present invention are described below in order to make the technical means of the present invention more clearly understood and to make the above and other objects, features, and advantages of the present invention more clearly understandable.
The above and other objects, advantages and features of the present invention will become more apparent to those skilled in the art from the following detailed description of specific embodiments thereof, taken in conjunction with the accompanying drawings.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:
FIG. 1 is a flow chart illustrating a method for external reference calibration of an onboard sensor in accordance with one embodiment of the present invention;
FIG. 2 is a schematic structural diagram illustrating an external reference calibration apparatus for an on-board sensor according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram illustrating an external reference calibration apparatus of a vehicle-mounted sensor according to another embodiment of the invention.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
In order to solve the technical problem, the embodiment of the invention provides an external parameter calibration method for a vehicle-mounted sensor. FIG. 1 is a flow chart illustrating a method for external reference calibration of an on-board sensor according to an embodiment of the invention. Referring to fig. 1, the method includes at least steps S102 to S108.
In step S102, the starting point positions of the target obstacle detected by the plurality of in-vehicle sensors are determined.
The vehicle-mounted sensor provided by the embodiment of the invention can adopt a vehicle-mounted millimeter wave radar, a vehicle-mounted laser radar, a vehicle-mounted camera or other vehicle-mounted equipment containing a camera and the like.
In step S104, a target trajectory corresponding to the target obstacle detected by the in-vehicle sensors is generated based on the position of the target obstacle in each frame of detection data from the plurality of in-vehicle sensors from the time when the start position of the target obstacle corresponds to the start position.
And S106, determining the point trace of the target obstacle detected by the corresponding vehicle-mounted sensor by adopting an interpolation algorithm according to the target trace.
The interpolation algorithm of the embodiment of the present invention may adopt a cubic spline interpolation method, and of course, other algorithms may also be adopted, which is not specifically limited in the embodiment of the present invention.
And S108, determining an external reference relation between any two vehicle-mounted sensors based on the point traces of the target obstacle detected by any two vehicle-mounted sensors in the plurality of vehicle-mounted sensors so as to realize external reference calibration of the vehicle-mounted sensors.
The external reference calibration process of the vehicle-mounted sensor provided by the embodiment of the invention has no limitation on the site and environment of the vehicle where the vehicle-mounted sensor is located, can perform external reference calibration in real time, and has strong operability. In addition, compared with the problem of poor robustness and accuracy of the currently adopted method for calibrating the longitudinal and transverse parameters respectively, the embodiment of the invention fully considers the characteristics of low resolution, jitter and frame loss of output results of some vehicle-mounted sensors, generates the target track of the target obstacle by adopting multi-frame detection data detected by the vehicle-mounted sensors, determines the trace point of the target obstacle detected by any two vehicle-mounted sensors according to the target track of the target obstacle, and determines the external reference relation between any two vehicle-mounted sensors according to the trace point, thereby greatly improving the accuracy and stability of external reference calibration.
Referring to step S102 above, in an embodiment of the present invention, when determining the starting positions of the target obstacles detected by the multiple vehicle-mounted sensors, it may be specifically determined whether M frames of detection data exist in the subsequent consecutive N frames of detection data to detect the target obstacle from the time when each vehicle-mounted sensor detects the target obstacle, where M is not greater than N. And if M frames of detection data exist in the continuous N frames of detection data to detect the target obstacle, taking the position of the target obstacle of the first frame of detection data in the continuous N frames of detection data as the starting position of the target obstacle detected by the corresponding vehicle-mounted sensor.
For example, M and N are 95 frames and 100 frames, respectively, and if the time when the vehicle-mounted millimeter wave radar detects the target obstacle is T0 and 95 frames of detection data exist in 100 consecutive frames of detection data from T0 to detect the target obstacle, the position of the target obstacle of the first frame of detection data in the 100 frames of detection data is taken as the starting point position of the target obstacle detected by the vehicle-mounted millimeter wave radar. Similarly, the starting position of the detected target obstacle is determined by the same method for other vehicle-mounted sensors, and details are not repeated here.
Referring to step S104 above, in an embodiment of the present invention, the detection data output by the vehicle-mounted sensor includes position information of the target obstacle. The process of generating the target trajectory corresponding to the target obstacle detected by the vehicle-mounted sensor may specifically include steps 1041 to 1044.
Step 1041, defining a sensor coordinate system corresponding to each of the plurality of vehicle-mounted sensors by taking the center position of each vehicle-mounted sensor at the initial position of the vehicle as the origin of the sensor coordinate system, defining a vehicle coordinate system by taking the center position of the rear axle of the vehicle as the origin of the vehicle coordinate system, and defining the vehicle coordinate system at the moment corresponding to the starting position of the target obstacle as a world coordinate system.
For example, the coordinate value of the object obj in the sensor coordinate system detected by the in-vehicle sensor a at the time i is represented by (x)ai，yai1), the coordinate value of the target obstacle obj detected by the in-vehicle sensor a at the time i in the vehicle coordinate system is represented as (x)avi，yavi1), v denotes a vehicle coordinate system. The coordinate system in the embodiment of the invention omits the value of the z axis and replaces the value with 1, and the calibration process related in the embodiment of the invention has enough calibration data precision obtained by an xy coordinate plane parallel to the ground to meet the application requirement of the system.The method for calibrating by adopting the three-dimensional coordinate system can be obtained by expanding the scheme of the invention.
In this embodiment, it is assumed that the plurality of in-vehicle sensors include an in-vehicle millimeter wave radar, an in-vehicle camera, and an in-vehicle laser radar. The central position of the vehicle-mounted millimeter wave radar is used as the origin of a coordinate system in a world coordinate system corresponding to the vehicle-mounted millimeter wave radar, the x direction is defined as the normal direction of the vehicle-mounted millimeter wave radar, the normal direction of the vehicle-mounted millimeter wave radar refers to the direction perpendicular to an antenna emitting surface on the millimeter wave radar, and the world coordinate system is a right-hand coordinate system (in the right-hand coordinate system, the thumb of the right hand points to the positive direction of the x axis, and the index finger direction is the positive direction of the y axis). And in a world coordinate system corresponding to the vehicle-mounted camera, the central position of the vehicle-mounted camera is taken as the origin of the coordinate system, the x direction is defined as the normal direction of the vehicle-mounted camera, the normal direction of the vehicle-mounted camera is the direction vertical to the plane of the photosensitive component on the vehicle-mounted camera, and the world coordinate system is a right-hand coordinate system. And in a world coordinate system corresponding to the vehicle-mounted laser radar, the central position of the vehicle-mounted laser radar is used as the origin of the coordinate system, the x direction is defined as the direction in front of the vehicle, and the y direction is defined as the direction on the left of the vehicle.
And step 1042, determining coordinate values of the positions of the target obstacles in the corresponding sensor coordinate system in each frame of detection data of the plurality of vehicle-mounted sensors respectively based on the positions of the target obstacles in each frame of detection data from the plurality of vehicle-mounted sensors from the time corresponding to the starting positions of the target obstacles.
In this embodiment, the plurality of vehicle-mounted sensors include a vehicle-mounted millimeter wave radar, a vehicle-mounted camera, and a vehicle-mounted laser radar. The coordinate value of the position of the target obstacle in each frame of detection data from the vehicle-mounted millimeter wave radar in the corresponding world coordinate system is Pri (x)ri,yri) Where P denotes a target obstacle position, x and y denote the abscissa and ordinate of the target obstacle, respectively, r denotes the in-vehicle millimeter wave radar, and i denotes the i-th frame detection data. The coordinate value corresponding to the starting point position of the target obstacle may be represented by Pr0 ═ (x)r0，yr0)。
Each from a vehicle-mounted cameraPci (x) for coordinate values of the position of the target obstacle in the frame detection data in the corresponding world coordinate systemci,yci) Where P denotes a target obstacle position, x and y denote an abscissa and an ordinate of the target obstacle, respectively, c denotes an in-vehicle camera, and i denotes an i-th frame detection data. The coordinate value corresponding to the start point position of the target obstacle may be represented by Pc0 ═ xc0，yc0)。
The coordinate value of the position of the target obstacle in each frame of detection data from the vehicle-mounted laser radar is represented by Pli (x) under the corresponding world coordinate systemli,yli) Where P denotes a target obstacle position, x and y denote an abscissa and an ordinate of the target obstacle, respectively, l denotes an on-vehicle laser radar, and i denotes the i-th frame detection data. The coordinate value corresponding to the starting point position of the target obstacle may be represented by (x) 0 ═ xl0，yl0)。
Step 1043, converting the coordinate value of the position of the target obstacle under the corresponding sensor coordinate system into a coordinate value of the position of the target obstacle under the vehicle coordinate system, and converting the coordinate value of the position of the target obstacle under the vehicle coordinate system into a coordinate value of the position of the target obstacle under the world coordinate system.
In this embodiment, the rotational-translational relationship between the sensor coordinate system and the vehicle coordinate system is determined at the time of vehicle shipment, and the rotational-translational relationship may be represented by a corresponding transformation matrix. For example, the transformation matrix of the sensor coordinate system of the vehicle-mounted sensor a and the vehicle coordinate system may be represented by SVRTa, where SVRT represents the transformation matrix. If the position of the origin of the sensor coordinate system of the vehicle-mounted sensor a in the vehicle coordinate system is (x)ao,yao) The included angle between the x axis of the sensor coordinate system of the vehicle-mounted sensor a and the x axis of the vehicle coordinate system is thetaaoHere, SVRTa may mean that the sensor coordinate system of the vehicle-mounted sensor a is that the vehicle coordinate system is translated by x in the positive x-axis direction of the vehicle coordinate systemaoIs translated in the positive direction of the y-axisaoAnd then rotated counterclockwise by theta about the translated coordinate system originaoAnd then obtaining the compound. Conversion of sensor coordinate system coordinate value of vehicle-mounted sensor a into vehicle coordinate system coordinate valueThe matrix SVRTa is represented as:
Figure BDA0002383868630000071
cos denotes a cosine function and sin denotes a sine function.
Referring to the target obstacle obj detected by the preceding vehicle-mounted sensor a, the coordinate value (x) of the target obstacle obj in the sensor coordinate system of the vehicle-mounted sensor aai，yai1) and coordinate value (x) of the target obstacle obj in the vehicle coordinate systemavi，yaviAnd, the transformation relationship of 1) can be expressed as:
Figure BDA0002383868630000081
can also be expressed as
Figure BDA0002383868630000082
Wherein, SVRTa-1The inverse matrix of SVRTa is represented and v represents the vehicle coordinate system.
In the embodiment of the invention, the world coordinate system is a vehicle coordinate system at the moment corresponding to the starting point position of the detected target obstacle. The coordinate value of the target obstacle obj detected by the in-vehicle sensor a at the time i in the world coordinate system may be expressed as (x)awi，yawi1), w represents the world coordinate system.
The rotational-translational relationship between the vehicle coordinate system and the world coordinate system of the embodiment of the invention can be determined by calculating the position change of the vehicle at each moment, and a commonly used method is a odometer calculation method, wherein the position of the vehicle relative to the starting point and the included angle between the vehicle body and the world coordinate system at the corresponding moment can be given by using the input of the vehicle sensor (such as a vehicle-mounted wheel speed meter, a vehicle-mounted inertial navigation device and the like) at each moment, specifically including the position of the vehicle, and the coordinate value of the vehicle position is used (x is the coordinate value of the vehicle position)ei，yei) Indicates the angle yaw between the x-axis of the vehicle coordinate system and the x-axis of the world coordinate system at the corresponding timeeiWhere e denotes the own vehicle and i denotes the time i, the vehicle position coordinate value (x) of this embodimentei，yei) The specific position of the vehicle coordinate system at the time of i is that the world coordinate system is in the positive direction of the x axis of the world coordinate systemMoving xeiTranslating y in the positive direction of the y-axis of the world coordinate systemeiThen rotate yaw counterclockwise around the translated origin of the coordinate systemei. The transformation matrix for transformation from the vehicle coordinate system to the world coordinate system can be represented by VWRTi:
Figure BDA0002383868630000083
where e denotes the own vehicle, i denotes the time i, w denotes the world coordinate system, cos denotes the cosine function, and sin denotes the sine function.
The coordinate values (x) of the target obstacle in the world coordinate system can be calculated by using the transformation matrix VWRTi (i represents the time i) and the coordinate values of the target obstacle obj detected by the vehicle-mounted sensor a in the coordinate system of the vehicle-mounted sensor aawi，yawi1), wherein,
Figure BDA0002383868630000084
VWRTi denotes a transformation matrix for transforming the coordinate values of the vehicle coordinate system into coordinate values of the world coordinate system, SVRTa denotes a transformation matrix for transforming the coordinate values of the sensor coordinate system of the on-vehicle sensor a into coordinate values of the vehicle coordinate system, v denotes the vehicle coordinate system, and w denotes the world coordinate system.
And step 1044, generating a target track of the target obstacle in the world coordinate system, which corresponds to the detection of the vehicle-mounted sensor, based on the coordinate value of the position of the target obstacle in the world coordinate system.
In the embodiment of the present invention, before generating the target trajectory of the target obstacle detected by the vehicle-mounted sensor, it is necessary to ensure that the target obstacle is in a moving state, and the embodiment of the present invention may determine whether the speed of the target obstacle relative to the vehicle is 0 or not by acquiring the speed of the vehicle itself, and if the speed of the target obstacle relative to the vehicle is not 0, determine that the obstacle is in a moving state.
In this embodiment, the target trajectory of the target obstacle is usually the target trajectory for the target obstacle in the detection data for a certain period of time, and therefore, it may be assumed that the starting point position of the target obstacle corresponds to the time T0, and each frame of data detected by each vehicle-mounted sensor during the period from the time T0 to the time T0+ Δ T includes the target obstacle, that is, there is no missing target obstacle, and then the target trajectory of the target obstacle detected by the vehicle-mounted millimeter wave radar may be represented as Tr ═ Pr0, Pr1, …, Prn), the target trajectory of the target obstacle detected by the vehicle-mounted camera may be represented as Tc ═ Pc0, Pc1, …, Pcn, and the target trajectory of the target obstacle detected by the vehicle-mounted laser radar may be represented as Tl ═ Pl (Pl0, Pl1, …, Pln). Wherein r represents a vehicle-mounted millimeter wave radar, c represents a vehicle-mounted camera, l represents a vehicle-mounted laser radar, and n represents the nth frame detection data.
In addition, if the target obstacle is not detected in the detection data of one frame or several frames from the vehicle-mounted sensor from the time T0 to the time T0+ Δ T, the target position information in the detection data of one frame or several frames in which the target obstacle is not detected can be calculated by a state estimation method, such as a kalman filter algorithm.
Referring to step S106 above, in an embodiment of the present invention, the interpolation algorithm may use a cubic spline interpolation method, and of course, other interpolation algorithms may also be used, which is not limited in this respect. When the trace of the target obstacle detected by the vehicle-mounted sensor is determined by adopting an interpolation algorithm aiming at the target track, the trace of the target obstacle detected by the vehicle-mounted sensor can be determined by adopting a cubic spline interpolation method aiming at the target track. For example, the vehicle-mounted sensor detects 20 frames of detection data, the position of the target obstacle in each frame of detection data corresponds to one coordinate value, the generated target track of the target obstacle contains 20 coordinate values, the number of points in the expected trace is set to be 40 (namely, 40 coordinate values are expected to be obtained), the 20 coordinate values and the expected point numerical value 40 are used as known parameters of a cubic spline interpolation method, the trace of the target obstacle is calculated according to the cubic spline interpolation method, and the trace of the point contains 40 coordinate values. In the embodiment of the invention, the trace of the target obstacle detected by the plurality of vehicle-mounted sensors contains the same number sn of points. In general, the number of coordinate values of the desired trace of dots is greater than the number of coordinate values of the target track, that is, the trace of the target obstacle calculated according to the cubic spline interpolation method contains a greater number of coordinate values than the number of coordinate values of the target track, and therefore, the trace of the target obstacle detected by the vehicle-mounted sensor determined by the cubic spline interpolation method is a smoother track than the target track of the target obstacle in the foregoing.
The trace of the target obstacle detected by the vehicle-mounted millimeter wave radar calculated by the interpolation algorithm may be represented as Trs ═ Prs0 (x)rs0,yrs0)，Prs1(xrs1,yrs1)，…，Prsn(xrsn,yrsn)]Where P denotes a target obstacle position, r denotes a vehicle-mounted millimeter wave radar, x and y denote an abscissa and an ordinate of the target obstacle, respectively, and sn denotes the number of points in the trace of the target obstacle.
The trace of the target obstacle detected by the vehicle-mounted camera calculated by the interpolation algorithm may be represented by Tcs ═ Pcs0 (x)cs0,ycs0)，Pcs1(xcs1,ycs1)，…，Pcsn(xcsn,ycsn)]Where P denotes a target obstacle position, c denotes a vehicle-mounted camera, x and y denote an abscissa and an ordinate of the target obstacle, respectively, and sn denotes the number of points in the trace of the target obstacle.
The trace of the target obstacle detected by the vehicle-mounted laser radar calculated by the interpolation algorithm may be represented as Tls ═ Pls0 (x)ls0,yls0)，Pls1(xls1,yls1)，…，Plsn(xlsn,ylsn)]Wherein, P represents the position of the target obstacle, l represents the vehicle-mounted laser radar, x and y represent the abscissa and ordinate of the target obstacle respectively, and sn represents the number of points in the trace of the target obstacle.
Referring to step S108 above, in an embodiment of the present invention, when the external reference relationship between any two vehicle-mounted sensors is determined to calibrate the external reference of the vehicle-mounted sensors based on the trace points of the target obstacle detected by any two vehicle-mounted sensors in the plurality of vehicle-mounted sensors, one-dimensional data may be respectively added to the coordinate values corresponding to the points in each trace point for the trace points of the target obstacle detected by the plurality of vehicle-mounted sensors. And then determining the external reference relation between any two vehicle-mounted sensors to realize external reference calibration of the vehicle-mounted sensors based on the point traces of the target obstacle detected by any two vehicle-mounted sensors after the one-dimensional data is added.
Adding one-dimensional data into the point trace of the target obstacle corresponding to the vehicle-mounted millimeter wave radar to obtain Trs ═ Prs0 (x)rs0,yrs0,1)，Prs1(xrs1,yrs1,1)，…，Prsn(xrsn,yrsn,1)]Where P denotes a target obstacle position, r denotes a vehicle-mounted millimeter wave radar, x and y denote an abscissa and an ordinate of the target obstacle, respectively, and sn denotes the number of points in the trace of the target obstacle.
Adding one-dimensional data to the trace of the target obstacle corresponding to the vehicle-mounted camera is Tcs ═ Pcs0 (x)cs0,ycs0,1)，Pcs1(xcs1,ycs1,1)，…，Pcsn(xcsn,ycsn,1)]Where P denotes a target obstacle position, c denotes a vehicle-mounted camera, x and y denote an abscissa and an ordinate of the target obstacle, respectively, and sn denotes the number of points in the trace of the target obstacle.
Adding one-dimensional data into the point trace of the target obstacle corresponding to the vehicle-mounted laser radar to obtain Tls ═ Pls0 (x)ls0,yls0,1)，Pls1(xls1,yls1,1)，…，Plsn(xlsn,ylsn,1)]Wherein, P represents the position of the target obstacle, l represents the vehicle-mounted laser radar, x and y represent the abscissa and ordinate of the target obstacle respectively, and sn represents the number of points in the trace of the target obstacle.
In the embodiment of the invention, the specific process of determining the external reference relation between any two vehicle-mounted sensors to realize external reference calibration of the vehicle-mounted sensors based on the point traces of the target obstacle detected by any two vehicle-mounted sensors after adding one-dimensional data is as follows.
Firstly, for any two vehicle-mounted sensors, the trace of a target obstacle detected by one of the vehicle-mounted sensors is taken as a main trace.
And then, calculating a change matrix of the trace point of the target obstacle detected by the other vehicle-mounted sensor and the main trace point by using a closest point iterative algorithm according to the point in the main trace point and the point in the trace point of the target obstacle detected by the other vehicle-mounted sensor. According to the embodiment of the invention, one-dimensional data is added in the coordinate value corresponding to the point in the trace of the target obstacle, so that the mathematical calculation of the change matrix is facilitated. Assuming that the main on-board sensor corresponding to the main point trace is the on-board sensor a, and the other on-board sensor is the on-board sensor b, the variation matrix of the point trace of the target obstacle detected by the on-board sensor b and the main point trace of the target obstacle detected by the on-board sensor a can be represented as RTab.
And finally, converting the coordinate value of the coordinate system corresponding to the target obstacle detected by one vehicle-mounted sensor in any two vehicle-mounted sensors into the external reference relation of the coordinate value of the coordinate system corresponding to the other vehicle-mounted sensor so as to realize external reference calibration of the vehicle-mounted sensors.
In the embodiment of the present invention, for example, the target obstacle obj is detected by the in-vehicle sensor b at time i, and the coordinate value of the target obstacle obj in the coordinate system of the in-vehicle sensor b is (x)bi，ybi1), the coordinate value of the target obstacle obj in the vehicle coordinate system is (x)bvi，ybvi1), coordinate values (x) of the target obstacle obj in the world coordinate systembwi，ybwi1), the coordinate values under the three coordinate systems satisfy the relationship:
Figure BDA0002383868630000111
wherein v represents a vehicle coordinate system, w represents a world coordinate system, and SVRTb is a transformation matrix for transforming coordinate values of the vehicle-mounted sensor b coordinate system into coordinate values of the vehicle coordinate system.
For the vehicle which is just shipped from factory, the coordinate value of the target obstacle obj in the sensor coordinate system of the sensor a at the time point i is converted to the coordinate value of the vehicle coordinate system, and the coordinate value of the sensor coordinate system of the sensor b converted to the coordinate value of the vehicle coordinate system is equal to the coordinate value of the target obstacle obj in the time point i
Figure BDA0002383868630000112
To which the respective sensor coordinate systems of sensor a and sensor b are convertedThe coordinate values of the vehicle coordinate system are respectively converted into coordinate values of the world coordinate system and are also equal,
Figure BDA0002383868630000113
therefore, the factory external parameter relationship extRTab between any two vehicle-mounted sensors (where ext denotes factory external parameter, RT denotes roto-translational transformation relationship, a denotes vehicle-mounted sensor a, and b denotes vehicle-mounted sensor b) can be obtained by multiplying SVRTb by an inverse matrix of SVRTa, and is expressed as extRTab ═ SVRTa-1SVRTb, wherein SVRTa-1And the SVRTa is a transformation matrix for converting the coordinate value of the coordinate system of the vehicle-mounted sensor a into the coordinate value of the coordinate system of the vehicle.
The coordinate values of the target obstacle obj detected by the vehicle-mounted sensor b and the coordinate values of the sensor coordinate system of the vehicle-mounted sensor a can be set up into an equality relationship by using the external reference relationship extRTab, which is expressed as:
Figure BDA0002383868630000121
wherein a represents an in-vehicle sensor a, b represents an in-vehicle sensor b, SVRTa-1And the SVRTa represents a transformation matrix for converting the coordinate value of the coordinate system of the vehicle-mounted sensor a into the coordinate value of the coordinate system of the vehicle.
Due to the long-term use of the sensor, the above-mentioned equality relationship between the coordinate value of the target obstacle obj detected by the vehicle-mounted sensor b and the coordinate value of the sensor coordinate system of the vehicle-mounted sensor a cannot be established, and other applications depending on the detection results of the two sensors will be affected, specifically, the coordinate value of the target obstacle obj in the respective coordinate systems of the vehicle-mounted sensor a and the vehicle-mounted sensor b at the time i is transformed to the coordinate value of the vehicle coordinate system, and then the coordinate values in the vehicle coordinate system transformed from the coordinate values of the respective vehicle-mounted sensor coordinate systems are transformed to the coordinate values in the world coordinate system, that is, the coordinate values
Figure BDA0002383868630000122
Where a denotes an in-vehicle sensor a, b denotes an in-vehicle sensor b, v denotes a vehicle coordinate system, and w denotes a world coordinate system.
The RTab obtained by the embodiment of the invention can enable the trace point b to coincide with the trace point a through transformation, and specifically, an equality relation is established between coordinate values of a vehicle coordinate system of the target obstacle obj detected by the vehicle-mounted sensor a and the vehicle-mounted sensor b at the moment i
Figure BDA0002383868630000123
Wherein VWRTi represents a transformation matrix for transforming coordinate values of the vehicle coordinate system into coordinate values of the world coordinate system, VWRTi-1An inverse matrix of VWRTi is shown, RTab is a change matrix of the locus of the target obstacle detected by the vehicle-mounted sensor b and the principal locus of the target obstacle detected by the vehicle-mounted sensor a, v is a vehicle coordinate system, and w is a world coordinate system. And the coordinate values in the vehicle coordinate system converted from the coordinate values in the vehicle-mounted sensor coordinate system are converted into the coordinate values in the world coordinate system to establish an equality relation
Figure BDA0002383868630000124
Wherein RTab represents a change matrix of a locus of a target obstacle detected by the vehicle-mounted sensor b and a principal locus of the target obstacle detected by the vehicle-mounted sensor a, v represents a vehicle coordinate system, and w represents a world coordinate system. Further, according to NextRTab ═ SVRTa-1*VWRT-1RTab VWRT SVRTb can obtain a calibrated external reference relation NextRTab, wherein SVRTa is a transformation matrix for converting coordinate values of a vehicle-mounted sensor coordinate system into coordinate values of a vehicle coordinate system, and SVRTa-1An inverse matrix representing SVRTa, a transformation matrix representing the transformation of coordinate values of the vehicle coordinate system to coordinate values of the world coordinate system, VWRT-1Represents the inverse matrix of VWRT.
In the embodiment of the present invention, an equality relationship can be established between the coordinate values of the target obstacle obj detected by the vehicle-mounted sensor b and the coordinate values of the coordinate system of the vehicle-mounted sensor a by using the calibrated external reference relationship nextttab, and is expressed as:
Figure BDA0002383868630000131
wherein a represents an in-vehicle sensor a, b represents an in-vehicle sensor b, SVRTa-1An inverse matrix of a transformation matrix representing a sensor coordinate system of the a-sensor and a vehicle coordinate system, SVRTb an inverse matrix of a transformation matrix representing a sensor coordinate system of the b-sensor and a vehicle coordinate system, VWRT a transformation matrix representing a transformation matrix of a vehicle coordinate system coordinate value into a world coordinate system coordinate value, VWRT-1An inverse matrix of VWRT is shown, and RTab is a change matrix of the dot trace of the target obstacle detected by the vehicle-mounted sensor a and the dot trace of the target obstacle detected by the vehicle-mounted sensor b.
In this embodiment, for example, any two vehicle-mounted sensors are a vehicle-mounted millimeter wave radar and a vehicle-mounted camera, the point trace Trs of a target obstacle detected by the vehicle-mounted millimeter wave radar is used as a main point trace, and a change matrix RTcr of Tcs and Trs is calculated by using a closest point iteration algorithm according to a point in the point trace Trs of the vehicle-mounted millimeter wave radar and a point in the point trace Tcs of the vehicle-mounted camera, and the three satisfy a relationship of Trs-RTcr-Tcs, and the change matrix RTcr may be a 3 × 3 matrix. Therefore, the change matrix RTcr is an extrinsic relationship in which the coordinate values in the sensor coordinate system corresponding to the onboard camera are converted into the coordinate values in the sensor coordinate system corresponding to the onboard millimeter wave radar, and the change matrix RTcr may also be referred to as an extrinsic matrix. The inverse matrix of the change matrix RTcr is an external reference relationship in which the coordinate values in the sensor coordinate system corresponding to the vehicle-mounted millimeter wave radar are converted to the coordinate values in the sensor coordinate system corresponding to the vehicle-mounted camera.
For another example, any two vehicle-mounted sensors are a vehicle-mounted millimeter wave radar and a vehicle-mounted laser radar, a point trace Trs of a target obstacle detected by the vehicle-mounted millimeter wave radar is used as a main point trace, a change matrix RTlr of Tls and Trs is calculated by using a closest point iteration algorithm according to a point in the point trace Trs of the vehicle-mounted millimeter wave radar and a point in the point trace Tls of the vehicle-mounted laser radar, a relationship of Trs-RTlr-Tls is satisfied between the three, and optionally, the change matrix RTlr may be a 3 × 3 matrix. Therefore, the change matrix RTlr is an external reference relationship in which the coordinate values in the sensor coordinate system corresponding to the vehicle-mounted laser radar are converted into the coordinate values in the sensor coordinate system corresponding to the vehicle-mounted millimeter wave radar, and the change matrix RTlr may also be referred to as an external reference matrix. And the inverse matrix of the change matrix RTlr is an external reference relation of converting the coordinate values under the sensor coordinate system corresponding to the vehicle-mounted millimeter wave radar into the coordinate values under the sensor coordinate system corresponding to the vehicle-mounted laser radar.
For another example, any two vehicle-mounted sensors are a vehicle-mounted laser radar and a vehicle-mounted camera, a point trace Tls of a target obstacle detected by the vehicle-mounted laser radar is used as a main point trace, a change matrix RTcl of Tcs and Tls is calculated by using a closest point iteration algorithm according to a point in the point trace Tls of the vehicle-mounted laser radar and a point in the point trace Tcs of the vehicle-mounted camera, a relationship of Tls and RTcl Tcs is satisfied between the three, and optionally, the change matrix RTcl may be a 3 × 3 matrix. Therefore, the change matrix RTcl is an extrinsic relationship in which the coordinate values in the sensor coordinate system corresponding to the vehicle-mounted camera are converted into the coordinate values in the sensor coordinate system corresponding to the vehicle-mounted laser radar, and the change matrix may also be referred to as an extrinsic matrix. And the inverse matrix of the change matrix RTcl is an external reference relation of converting the coordinate values under the sensor coordinate system corresponding to the vehicle-mounted laser radar into the coordinate values under the sensor coordinate system corresponding to the vehicle-mounted camera.
According to the embodiment of the invention, the target obstacle can be detected in real time through the vehicle-mounted sensors, the external reference relation between any two vehicle-mounted sensors is determined based on the calculated trace points of the target obstacle detected by any two vehicle-mounted sensors, and the external reference relations determined at different time are possibly different, so that the external reference matrix determined later is compared with the external reference matrix before, and when the deviation of the external reference matrix and the external reference matrix is greater than a certain threshold value, the external reference matrix before is replaced by the new external reference matrix, so that the purpose of correcting the external reference matrix in real time can be achieved. It is generally considered that the vehicle-mounted sensors are in a rigid connection relationship, that is, the positions of the vehicle-mounted sensors on the vehicle do not change obviously, but slight displacement is allowed. The embodiment of the invention corrects the external parameter matrix in real time by completing external parameter calibration of the vehicle-mounted sensor in time, and solves the problem of detection data deviation of a target obstacle caused by small position change of the vehicle-mounted sensor due to a complex use environment of the vehicle-mounted sensor.
Based on the same invention concept, the embodiment of the invention also provides an external reference calibration device of the vehicle-mounted sensor. Fig. 3 is a schematic structural diagram illustrating an external reference calibration apparatus of a vehicle-mounted sensor according to an embodiment of the invention. Referring to fig. 3, the external reference calibration apparatus for the vehicle-mounted sensor includes a first determining module 310, a generating module 320, a second determining module 330, and a calibrating module 340.
The functions of the components or devices of the external reference calibration device of the vehicle-mounted sensor and the connection relationship between the components are described as follows:
a first determining module 310 adapted to determine starting positions of the target obstacle detected by the plurality of vehicle-mounted sensors;
a generating module 320, coupled to the first determining module 310, adapted to generate a target trajectory corresponding to the target obstacle detected by the vehicle-mounted sensors based on the position of the target obstacle in each frame of detection data from the plurality of vehicle-mounted sensors from a time corresponding to the starting point position of the target obstacle;
a second determining module 330, coupled to the generating module 320, adapted to determine, by using an interpolation algorithm, a point trace of the target obstacle detected by the corresponding on-vehicle sensor with respect to the target trace;
the calibration module 340 is coupled to the second determining module 330, and is adapted to determine an external reference relationship between any two vehicle-mounted sensors based on the trace points of the target obstacle detected by any two vehicle-mounted sensors in the plurality of vehicle-mounted sensors, so as to implement external reference calibration on the vehicle-mounted sensors.
Based on the same inventive concept, an embodiment of the present invention further provides an electronic device, including: a processor; a memory storing computer program code; the computer program code, when executed by the processor, causes the electronic device to perform the method for external reference calibration of an in-vehicle sensor of any of the embodiments above.
Based on the same inventive concept, an embodiment of the present invention further provides a computer storage medium, where computer program codes are stored, and when the computer program codes are run on a computing device, the computing device is caused to execute the external reference calibration method for the vehicle-mounted sensor in any of the above embodiments.
According to any one or a combination of the above preferred embodiments, the following advantages can be achieved by the embodiments of the present invention:
the external reference calibration process of the vehicle-mounted sensor provided by the embodiment of the invention has no limitation on the site and environment of the vehicle where the vehicle-mounted sensor is located, can perform external reference calibration in real time, and has strong operability. In addition, compared with the problem of poor robustness and accuracy of the currently adopted method for calibrating the longitudinal and transverse parameters respectively, the embodiment of the invention fully considers the characteristics of low resolution, jitter and frame loss of output results of some vehicle-mounted sensors, generates the target track of the target obstacle by adopting multi-frame detection data detected by the vehicle-mounted sensors, determines the trace point of the target obstacle detected by any two vehicle-mounted sensors according to the target track of the target obstacle, and determines the external reference relation between any two vehicle-mounted sensors according to the trace point, thereby greatly improving the accuracy and stability of external reference calibration.
In the description provided herein, numerous specific details are set forth. It is understood, however, that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.
Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. However, the disclosed method should not be interpreted as reflecting an intention that: that the invention as claimed requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this invention.
Those skilled in the art will appreciate that the modules in the device in an embodiment may be adaptively changed and disposed in one or more devices different from the embodiment. The modules or units or components of the embodiments may be combined into one module or unit or component, and furthermore they may be divided into a plurality of sub-modules or sub-units or sub-components. All of the features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or elements of any method or apparatus so disclosed, may be combined in any combination, except combinations where at least some of such features and/or processes or elements are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.
Furthermore, those skilled in the art will appreciate that while some embodiments described herein include some features included in other embodiments, rather than other features, combinations of features of different embodiments are meant to be within the scope of the invention and form different embodiments. For example, in the claims, any of the claimed embodiments may be used in any combination.
The various component embodiments of the invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It will be appreciated by those skilled in the art that a microprocessor or Digital Signal Processor (DSP) may be used in practice to implement some or all of the functions of some or all of the components of the external reference calibration apparatus for an on-board sensor in accordance with embodiments of the present invention. The present invention may also be embodied as apparatus or device programs (e.g., computer programs and computer program products) for performing a portion or all of the methods described herein. Such programs implementing the present invention may be stored on computer-readable media or may be in the form of one or more signals. Such a signal may be downloaded from an internet website or provided on a carrier signal or in any other form.
It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The usage of the words first, second and third, etcetera do not indicate any ordering. These words may be interpreted as names.
Thus, it should be appreciated by those skilled in the art that while a number of exemplary embodiments of the invention have been illustrated and described in detail herein, many other variations or modifications consistent with the principles of the invention may be directly determined or derived from the disclosure of the present invention without departing from the spirit and scope of the invention. Accordingly, the scope of the invention should be understood and interpreted to cover all such other variations or modifications.

Claims (4)

1. An external reference calibration method of a vehicle-mounted sensor comprises the following steps:
determining starting point positions of target obstacles detected by a plurality of vehicle-mounted sensors;
generating a target track corresponding to the target obstacle detected by the vehicle-mounted sensors based on the position of the target obstacle in each frame of detection data from the plurality of vehicle-mounted sensors from the time corresponding to the starting point position of the target obstacle;
determining the point trace of the target obstacle detected by the corresponding vehicle-mounted sensor by adopting an interpolation algorithm aiming at the target trace;
determining an external reference relation between any two vehicle-mounted sensors to realize external reference calibration of the vehicle-mounted sensors based on the point traces of the target obstacle detected by any two vehicle-mounted sensors in the plurality of vehicle-mounted sensors;
wherein the determining of the starting positions of the target obstacles detected by the plurality of vehicle-mounted sensors includes: determining whether M frames of detection data exist in the subsequent continuous N frames of detection data to detect the target obstacle from the moment when each vehicle-mounted sensor respectively detects the target obstacle, wherein M is not more than N; if M frames of detection data exist in the continuous N frames of detection data to detect the target obstacle, taking the position of the target obstacle of the first frame of detection data in the continuous N frames of detection data as the starting position of the target obstacle detected by the corresponding vehicle-mounted sensor;
generating a target trajectory corresponding to the target obstacle detected by the in-vehicle sensor based on the position of the target obstacle in each frame of detection data from the plurality of in-vehicle sensors from a time when the start position of the target obstacle corresponds to the start position, including: defining a sensor coordinate system corresponding to each vehicle-mounted sensor by taking the central position of each vehicle-mounted sensor at the initial position of the vehicle as the origin of a sensor coordinate system, defining a vehicle coordinate system by taking the central position of a rear axle of the vehicle as the origin of the vehicle coordinate system, and defining the vehicle coordinate system at the moment corresponding to the starting position of the target obstacle as a world coordinate system; determining coordinate values of the positions of the target obstacles in the detection data of each frame of the plurality of vehicle-mounted sensors in a corresponding sensor coordinate system based on the positions of the target obstacles in the detection data of each frame from the plurality of vehicle-mounted sensors from the time corresponding to the starting positions of the target obstacles; converting the coordinate value of the position of the target obstacle under the corresponding sensor coordinate system into the coordinate value of the position of the target obstacle under the vehicle coordinate system, and converting the coordinate value of the position of the target obstacle under the vehicle coordinate system into the coordinate value of the position of the target obstacle under the world coordinate system; generating a target track of the target obstacle detected by the corresponding vehicle-mounted sensor in the world coordinate system based on the coordinate value of the position of the target obstacle in the world coordinate system;
the method for determining the point trace of the target obstacle detected by the corresponding vehicle-mounted sensor by adopting an interpolation algorithm aiming at the target trace comprises the following steps: determining the trace points of the target obstacles detected by the corresponding vehicle-mounted sensors by adopting a cubic spline interpolation method aiming at the target trace, wherein the trace points of the target obstacles detected by the plurality of vehicle-mounted sensors comprise the same number of points;
the method for determining the external reference relationship between any two vehicle-mounted sensors based on the point traces of the target obstacle detected by any two vehicle-mounted sensors in the plurality of vehicle-mounted sensors to realize external reference calibration of the vehicle-mounted sensors comprises the following steps: aiming at the point traces of the target obstacle detected by the plurality of vehicle-mounted sensors, respectively adding one-dimensional data in the coordinate values corresponding to the points in the point traces; determining an external reference relation between any two vehicle-mounted sensors to realize external reference calibration of the vehicle-mounted sensors based on the point traces of the target obstacle detected by any two vehicle-mounted sensors after one-dimensional data is added;
the method for determining the external reference relationship between any two vehicle-mounted sensors to realize external reference calibration of the vehicle-mounted sensors based on the point traces of the target obstacles detected by any two vehicle-mounted sensors after adding one-dimensional data comprises the following steps: for any two vehicle-mounted sensors, the trace of the target obstacle detected by one of the vehicle-mounted sensors is taken as a main trace; calculating a change matrix of the trace point of the target obstacle detected by the other vehicle-mounted sensor and the main trace point by using a closest point iterative algorithm according to the point in the main trace point and the point in the trace point of the target obstacle detected by the other vehicle-mounted sensor; and converting the change matrix as an external reference relation of the coordinate values of the target obstacle detected by one vehicle-mounted sensor in the two vehicle-mounted sensors in the sensor coordinate system to the coordinate values of the other vehicle-mounted sensor in the sensor coordinate system so as to realize external reference calibration of the vehicle-mounted sensors.
2. An external reference calibration device of a vehicle-mounted sensor comprises:
the system comprises a first determination module, a second determination module and a control module, wherein the first determination module is suitable for determining starting point positions of target obstacles detected by a plurality of vehicle-mounted sensors;
wherein the determining of the starting point positions of the target obstacles detected by the plurality of in-vehicle sensors includes: determining whether M frames of detection data exist in the subsequent continuous N frames of detection data to detect the target obstacle from the moment when each vehicle-mounted sensor respectively detects the target obstacle, wherein M is not more than N; if M frames of detection data exist in the continuous N frames of detection data to detect the target obstacle, taking the position of the target obstacle of the first frame of detection data in the continuous N frames of detection data as the starting position of the target obstacle detected by the corresponding vehicle-mounted sensor;
a generation module adapted to generate a target trajectory corresponding to the target obstacle detected by the vehicle-mounted sensors based on the position of the target obstacle in each frame of detection data from the plurality of vehicle-mounted sensors from a time corresponding to the starting point position of the target obstacle;
wherein the generating of the target trajectory corresponding to the target obstacle detected by the in-vehicle sensor based on the position of the target obstacle in each frame of detection data from the plurality of in-vehicle sensors from the time corresponding to the start position of the target obstacle includes: defining a sensor coordinate system corresponding to each vehicle-mounted sensor by taking the central position of each vehicle-mounted sensor at the initial position of the vehicle as the origin of a sensor coordinate system, defining a vehicle coordinate system by taking the central position of a rear axle of the vehicle as the origin of the vehicle coordinate system, and defining the vehicle coordinate system at the moment corresponding to the starting position of the target obstacle as a world coordinate system; determining coordinate values of the positions of the target obstacles in the detection data of each frame of the plurality of vehicle-mounted sensors in a corresponding sensor coordinate system based on the positions of the target obstacles in the detection data of each frame from the plurality of vehicle-mounted sensors from the time corresponding to the starting positions of the target obstacles; converting the coordinate value of the position of the target obstacle under the corresponding sensor coordinate system into the coordinate value of the position of the target obstacle under the vehicle coordinate system, and converting the coordinate value of the position of the target obstacle under the vehicle coordinate system into the coordinate value of the position of the target obstacle under the world coordinate system; generating a target track of the target obstacle detected by the corresponding vehicle-mounted sensor in the world coordinate system based on the coordinate value of the position of the target obstacle in the world coordinate system;
the second determination module is suitable for determining the point track of the target obstacle detected by the corresponding vehicle-mounted sensor by adopting an interpolation algorithm aiming at the target track;
wherein, the determining the point trace of the target obstacle corresponding to the detection of the vehicle-mounted sensor by adopting an interpolation algorithm aiming at the target trace comprises the following steps: determining the trace points of the target obstacles detected by the corresponding vehicle-mounted sensors by adopting a cubic spline interpolation method aiming at the target trace, wherein the trace points of the target obstacles detected by the plurality of vehicle-mounted sensors comprise the same number of points;
the calibration module is suitable for determining the external parameter relation between any two vehicle-mounted sensors to realize external parameter calibration of the vehicle-mounted sensors based on the point traces of the target obstacles detected by any two vehicle-mounted sensors in the plurality of vehicle-mounted sensors;
the method for determining the external reference relationship between any two vehicle-mounted sensors based on the point traces of the target obstacle detected by any two vehicle-mounted sensors in the plurality of vehicle-mounted sensors to realize external reference calibration of the vehicle-mounted sensors comprises the following steps: aiming at the point traces of the target obstacle detected by the plurality of vehicle-mounted sensors, respectively adding one-dimensional data in the coordinate values corresponding to the points in the point traces; determining an external reference relation between any two vehicle-mounted sensors to realize external reference calibration of the vehicle-mounted sensors based on the point traces of the target obstacle detected by any two vehicle-mounted sensors after one-dimensional data is added;
the method for calibrating external parameters of the vehicle-mounted sensors based on the point traces of the target obstacles detected by any two vehicle-mounted sensors after adding the one-dimensional data comprises the following steps of: for any two vehicle-mounted sensors, the trace of the target obstacle detected by one of the vehicle-mounted sensors is taken as a main trace; calculating a change matrix of the trace point of the target obstacle detected by the other vehicle-mounted sensor and the main trace point by using a closest point iterative algorithm according to the point in the main trace point and the point in the trace point of the target obstacle detected by the other vehicle-mounted sensor; and converting the change matrix as an external reference relation of the coordinate values of the target obstacle detected by one vehicle-mounted sensor in the two vehicle-mounted sensors in the sensor coordinate system to the coordinate values of the other vehicle-mounted sensor in the sensor coordinate system so as to realize external reference calibration of the vehicle-mounted sensors.
3. An electronic device, comprising: a processor; a memory storing computer program code; the computer program code, when executed by the processor, causes the electronic device to perform the method for external reference calibration of an on-board sensor of claim 1.
4. A computer storage medium having computer program code stored thereon which, when run on a computing device, causes the computing device to perform the method of external reference calibration of an on-board sensor of claim 1.
CN202010095077.7A 2020-02-13 2020-02-13 External parameter calibration method and device for vehicle-mounted sensor Active CN111383287B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010095077.7A CN111383287B (en) 2020-02-13 2020-02-13 External parameter calibration method and device for vehicle-mounted sensor

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010095077.7A CN111383287B (en) 2020-02-13 2020-02-13 External parameter calibration method and device for vehicle-mounted sensor

Publications (2)

Publication Number Publication Date
CN111383287A CN111383287A (en) 2020-07-07
CN111383287B true CN111383287B (en) 2021-06-29

Family

ID=71219702

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010095077.7A Active CN111383287B (en) 2020-02-13 2020-02-13 External parameter calibration method and device for vehicle-mounted sensor

Country Status (1)

Country Link
CN (1) CN111383287B (en)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111982532B (en) * 2020-08-07 2021-07-20 湖北亿咖通科技有限公司 Automatic parking performance testing method
EP4207133A4 (en) * 2020-09-25 2023-11-01 Huawei Technologies Co., Ltd. Traffic element observation method and apparatus
CN112649013A (en) * 2020-12-29 2021-04-13 天津天瞳威势电子科技有限公司 Method and device for determining passable area and electronic equipment
CN112781893B (en) * 2021-01-04 2022-09-06 重庆长安汽车股份有限公司 Spatial synchronization method and device for vehicle-mounted sensor performance test data and storage medium
CN113361544B (en) * 2021-06-11 2024-04-19 广东省大湾区集成电路与系统应用研究院 Image acquisition equipment, and external parameter correction method, device and storage medium thereof

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101339240B (en) * 2008-08-26 2011-06-22 中国人民解放军海军工程大学 Wireless sensor network object tracking method based on double layer forecast mechanism
CN106405555B (en) * 2016-09-23 2019-01-01 百度在线网络技术（北京）有限公司 Obstacle detection method and device for Vehicular radar system
CN109507652B (en) * 2018-11-29 2021-04-16 驭势（上海）汽车科技有限公司 Vehicle-mounted radar calibration method and device, vehicle-mounted equipment and storage medium
CN110673115B (en) * 2019-09-25 2021-11-23 杭州飞步科技有限公司 Combined calibration method, device, equipment and medium for radar and integrated navigation system

Also Published As

Publication number Publication date
CN111383287A (en) 2020-07-07

Similar Documents

Publication Publication Date Title
CN111383287B (en) External parameter calibration method and device for vehicle-mounted sensor
CN112051590B (en) Detection method and related device for laser radar and inertial measurement unit
US11279045B2 (en) Robot pose estimation method and apparatus and robot using the same
CN111812658B (en) Position determination method, device, system and computer readable storage medium
CN106476728B (en) Motion compensation for vehicle-mounted vehicle sensors
CN110873883B (en) Positioning method, medium, terminal and device integrating laser radar and IMU
CN110554376A (en) Radar range finding method for vehicles
CN113432553B (en) Trailer pinch angle measuring method and device and vehicle
CN110579754A (en) Method for determining external parameters of a lidar and other sensors of a vehicle
CN112904396A (en) High-precision positioning method and system based on multi-sensor fusion
CN111486867B (en) Calibration device and method for installation parameters of vision and inertia mixed tracking assembly
CN108961337B (en) Vehicle-mounted camera course angle calibration method and device, electronic equipment and vehicle
CN116301058A (en) Unmanned flight feedback nonlinear yaw control method, system and equipment
CN116448146A (en) Inertial navigation system self-calibration method, device, equipment and storage medium
CN114019954B (en) Course installation angle calibration method, device, computer equipment and storage medium
CN114419118A (en) Three-dimensional point cloud registration method, mobile device and storage medium
CN114185020A (en) Laser radar motion distortion removing method and device based on Ackerman chassis
CN112506190A (en) Robot positioning method, robot positioning device and robot
CN113030504B (en) Vehicle speed measuring method and device, vehicle-mounted computer equipment and storage medium
CN111488419A (en) Method and device for creating indoor robot map, electronic equipment and storage medium
JP7409037B2 (en) Estimation device, estimation method, estimation program
CN115357049B (en) Visual-based unmanned aerial vehicle non-cooperative target limited time tracking method and system
CN117168472B (en) Repositioning method, repositioning system, storage medium and repositioning equipment for unmanned vehicle
CN115829879B (en) Attitude quaternion processing method, device and equipment for agile satellite
CN115876239A (en) Sensor calibration method, device, medium, equipment and vehicle

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
TR01 Transfer of patent right
TR01 Transfer of patent right

Effective date of registration: 20220321

Address after: 430051 No. b1336, chuanggu startup area, taizihu cultural Digital Creative Industry Park, No. 18, Shenlong Avenue, Wuhan Economic and Technological Development Zone, Wuhan, Hubei Province

Patentee after: Yikatong (Hubei) Technology Co.,Ltd.

Address before: No.c101, chuanggu start up area, taizihu cultural Digital Industrial Park, No.18 Shenlong Avenue, Wuhan Economic Development Zone, Hubei Province

Patentee before: HUBEI ECARX TECHNOLOGY Co.,Ltd.