EP4669981A1

EP4669981A1 - CALIBRATION OF A VEHICLE SENSOR WITH AN ASSIGNED MOTION SENSOR

Info

Publication number: EP4669981A1
Application number: EP24706556.8A
Authority: EP
Inventors: Sandipan Das; Rui OLIVEIRA
Original assignee: Scania CV AB
Current assignee: Scania CV AB
Priority date: 2023-02-21
Filing date: 2024-02-14
Publication date: 2025-12-31
Also published as: SE2350199A1; SE547642C2; WO2024177549A1

Abstract

A method for extrinsic calibration of a vehicle sensor (10) arranged on a vehicle (1), comprising calibrating (S3) the vehicle sensor (10) based on angular velocity (ω(ms1)) and linear acceleration (f(ms1) of a first motion sensor (ms1) collocated with the vehicle sensor (10), angular velocity (ω(ms2)) and linear acceleration (f(ms2)) of a second motion sensor (ms2) having a known pose in a reference frame of the vehicle (1), and an assumption that as angular velocity of a rigid body is the same at all points of said rigid body, angular velocity of the first motion sensor (ms1) and the second motion sensor (ms2) are the same when the orientations of the first motion sensor (ms1) and of the second motion sensor (ms2) are the same. The disclosure also relates to a system and a vehicle.

Description

Calibration of vehicle sensor using collocated motion sensor Technical Field The present disclosure relates to calibration of sensors, and in particular to calibration of sensors on vehicles. Background Autonomous vehicles and Advance Driver-Assistance System, ADAS, technologies rely on accurate perception systems. These perception systems use information from a multitude of sensors of different modalities, such as cameras, radars, lidars, and others, to provide a rich view of the surroundings. Before information from these sensors can be combined into a single reference frame, a transformation between the sensors must be found. The exact locations of the sensors are then estimated, also referred to as extrinsic calibration. Extrinsic calibration of sensors on vehicles may be performed offline when the vehicle is standing still, or online while the vehicle is driving. Offline calibration techniques are often time consuming and require additional hardware such as fiducial markers. Online calibration techniques may be performed by comparing estimated states of the sensor to those in a relevant sensor frame. However, over time the estimated states may become erroneous because they have intrinsic drifts, causing inaccuracies into the calibration process. Hence, there is a need for improved calibration techniques for sensors arranged on vehicles that can be performed online. Summary It is an objective of the present disclosure to alleviate at least some of the drawbacks of the prior art. It is a further objective to provide techniques that facilitate calibration of sensors arranged on vehicles, and especially different modalities of sensors. These objectives and others are at least partly achieved by the method, system and vehicle according to the independent claims, and by the embodiments according to the dependent claims. According to a first aspect, the disclosure relates to a method for extrinsic calibration of a vehicle sensor arranged on a vehicle. The method comprises calibrating the vehicle sensor based on the angular velocity and linear acceleration of a first motion sensor collocated with the vehicle sensor, and the angular velocity and linear acceleration of a second motion sensor having a known pose in a reference frame of the vehicle. The calibration assumes that as angular velocity of a rigid body is the same at all points of the rigid body, the angular velocity of the first motion sensor and the second motion sensor are the same when the orientations of the first motion sensor and of the second motion sensor are the same. The method provides fast and versatile calibration of sensors on vehicles, that can be performed online while the vehicle is driving, i.e., in real-time. The method can be used to calibrate different sensor modalities using one single method, whereby the need for different calibration routines, e.g., Camera-Camera, Camera-Lidar, Lidar-Lidar, Camera-IMU etc., is eliminated. The calibration is performed without the need for state estimation, thereby providing a faster and more reliable calibration than other prior known methods. In some embodiments, the method is performed online while the vehicle is driving. Hence, there is no need to stop operating the vehicle for calibration purposes. Online calibration enables, for example, continuous refinement of previous calibration during driving. In some embodiments, the method comprises providing calibrated sensor data from the vehicle sensor to a control arrangement of the vehicle for use in autonomous driving. The autonomous driving functionality will then have better data to rely on. In some embodiments, the first motion sensor, the second motion sensor and the vehicle sensor are rigidly arranged on the vehicle. Hence, the sensors will reflect motions of the rigid body of the vehicle directly. In some embodiments, the first motion sensor, the second motion sensor and the vehicle sensor are rigidly arranged on the same rigid body of the vehicle. Thereby, the angular velocities of the sensors are the same provided that their frames have the same orientation. In some embodiments, the calibrating comprises determining a pose of the vehicle sensor in the reference frame by matching the angular velocity of the first motion sensor with the angular velocity of the second motion sensor and matching the linear acceleration of the first motion sensor with the linear acceleration of the second motion sensor. Thereby, an optimization problem can be established. In some embodiments, the matching includes finding a pose of the vehicle sensor in the reference frame where a difference between the angular velocity of the first motion sensor and the angular velocity of the second motion sensor and a difference between the linear acceleration of the first motion sensor and the linear acceleration of the second motion sensor, are minimized. Thereby, a solution of the optimization problem can be found. In some embodiments, the calibrating comprises compensating linear acceleration from the first motion sensor for Coriolis effects. Thereby the linear acceleration can be correctly reflected. In some embodiments, the pose comprises a rotational component and a translational component. In some embodiments, the method comprises obtaining, from the first motion sensor having a known pose in a reference frame of the vehicle, angular velocity and linear acceleration of the first motion sensor and obtaining, from the second motion sensor collocated with the vehicle sensor, angular velocity and linear acceleration of the second motion sensor. In some embodiments, the angular velocity of the first motion sensor and second motion sensor includes angular velocity in three degrees of freedom, DoF, and the linear acceleration of the first motion sensor and second motion sensor includes linear acceleration in three degrees of freedom, DoF. In some embodiments, the first motion sensor and the second motion sensor are Inertial Measurement Units, IMUs. In some embodiments, the data from the first motion sensor and data from the second motion sensor are synchronized in time. In some embodiments, the calibrating comprises selecting angular velocity and linear acceleration of the first motion sensor, and angular velocity and linear acceleration of a second motion sensor that fulfill excitation criteria and using the selected angular velocities and linear accelerations for the calibration. In some embodiments, the vehicle sensor is any of a camera, a lidar, a radar, an ultrasonic sensor or any other vehicle sensor. In some embodiments, the reference frame is a base frame of the vehicle. In some embodiments, a relation between a frame of the vehicle sensor and a frame of the first motion sensor is known. The relation is a geometric or spatial relation. According to a second aspect, the disclosure relates to use of the method according to the first aspect, for online extrinsic calibration of a plurality of vehicle sensors arranged on the same vehicle. According to a third aspect, the disclosure relates to use of the method according to the first aspect, for online extrinsic calibration of at least two different types of modalities of vehicle sensors arranged on the same vehicle. According to a fourth aspect, the disclosure relates to a computer program comprising instructions which, when the computer program is executed by a computer, cause the computer to carry out the method according to the first aspect. According to a fifth aspect, the disclosure relates to a computer-readable medium comprising instructions which, when executed by a computer, cause the computer to carry out the method according to the first aspect. According to a sixth aspect, the disclosure relates to a system for extrinsic calibration of a vehicle sensor arranged on a vehicle. The system comprising a first motion sensor collocated with the vehicle sensor, the first motion sensor being configured to sense angular velocity and linear acceleration. The system further comprises a second motion sensor having a known pose in a reference frame of the vehicle, the second motion sensor being configured to sense angular velocity and linear acceleration. The system further comprises a control arrangement configured to calibrate the vehicle sensor based on angular velocity and linear acceleration of the first motion sensor, angular velocity and linear acceleration of the second motion sensor, and an assumption that as angular velocity of a rigid body is the same at all points of said rigid body, angular velocity of the first motion sensor and the second motion sensor are the same when the orientations of the first motion sensor and of the second motion sensor are the same. The same effects as of with the first aspect can be achieved. According to a seventh aspect, the disclosure relates to a vehicle comprising the system according to the sixth aspect. Brief description of the drawings The embodiments disclosed herein are illustrated by way of example, and by not by way of limitation, in the figures of the accompanying drawings. Like reference numerals refer to corresponding parts throughout the drawings, in which Fig.1 illustrates a top view of an example vehicle comprising a plurality of sensors. Fig.2 depicts a relation between a second motion sensor and a reference frame of the vehicle in the vehicle in Fig.1. Fig.3 is a flow chart of an example method according to the first aspect. Fig.4 are diagrams of signals from the first motion sensor and the second motion sensor without calibration. Fig.5 are diagrams of signals from the first motion sensor and the second motion sensor where signals of the first motion sensor are calibrated using the proposed technique. Detailed description Calibration of vehicle sensors is needed to perform advanced vehicle operations such as driver assistance and autonomous driving. Data from a multitude of different modalities of sensors such as cameras, lidars, radars and others are needed to create an accurate and reliable view of the surroundings of the vehicle. Different modalities of sensors often need different types of calibration techniques, making the calibration complicated and time consuming. Some calibration techniques require sensor state estimations making the calibration computationally demanding and sometimes slow. It is a goal of multi-sensor calibration to figure out how different sensors within the vehicle are geometrically related to each other (see Fig.1) with respect to a base frame (B). Knowing the exact location of the sensors in the vehicle is a basis to obtain good perception estimates in a common reference frame. The inventor of the present invention has realized that motion sensors, for example, Inertial Measurement Units (IMUs), can be used to calibrate a collocated vehicle sensor. In more detail, by having a motion sensor collocated with a vehicle sensor that needs to be calibrated and having another motion sensor with a known pose in a reference frame of the vehicle, the signals from the motion sensors can be used to calibrate the vehicle sensor. As the angular velocity of a rigid body is the same at all points of the rigid body, it can be assumed that the angular velocities of the motion sensors are the same provided that their orientations are the same, and the insight that this assumption can be used for enabling calibration. The calibration is based on raw signals from the motion sensors, with bias and noise compensations making the process simpler and faster than other prior art methods. Also, as the motion sensors do not need any overlapping field-of-view between them they can be used to calibrate vehicle sensors with non-overlapping field-of- view as well when collocated with them. Moreover, there is no need for state estimation (traditionally used for online calibration to match the motion) which is a computationally heavy process that may suffer from process noise. Instead, the raw signals from the sensors can be used. Bias and noise extraction are for example done using a traditional Kalman Filter, or similar filtering techniques. However, in some embodiments, to improve the accuracy of the bias modeling, motion sequences with standstill positions in between are used. This helps in the convergence of the bias terms quickly and limits the error growth. Hence, the calibration may be based on only comparing raw motion sensor signals without any need for state estimation. In the following disclosure, embodiments of the present invention will be explained in more detail, starting with an explanation of some features used. Vehicle sensor: a sensor used in a vehicle for perception purposes, for example a lidar, a radar, a camera or an ultrasonic sensor. The vehicle sensor is a kind of exteroceptive sensor, or external state sensor, that perceives and gathers information from the vehicle’s environment. A vehicle sensor may also be referred to as an autonomous sensor as it is used by an autonomous vehicle in autonomous driving. Lidar: Light detection and ranging. Lidar is a distant sensing technique that produces infrared or laser light pulses that reflect off target objects. The lidar detects these reflections and the time between emissions and reception of the light pulse allows for distance estimate. A lidar is an active sensor. Radar: Radio detection and ranging. A radar emits electromagnetic (EM) waves within a region of interest and receives reflections from targets for signal processing and range information. It can be used to determine relative speed and position of identified obstacles using the Doppler property of the EM waves. A radar is an active sensor. Camera: A camera, or image sensor, produces images of the surroundings by detecting lights emitted from the surroundings on a photosensitive surface (image plane) using a camera lens. A camera is a passive sensor. Ultrasonic sensor: an ultrasonic sensor measures the distance of a target object by emitting ultrasonic sound waves and converts the reflected sound into an electric signal. Autonomous vehicle: A vehicle that can travel without human input. Autonomous vehicles typically use vehicle sensors to sense their surroundings. Autonomous driving: Assisted driving at different levels from driver assistance such as cruise control where the vehicle only controls accelerating/decelerating, up to full driving automation where the vehicle performs all driving tasks without driver interaction. ADAS: Advance Driver-Assistance System, where the vehicle only can control steering and accelerating/decelerating without driver interaction. Motion sensor: a sensor configured to sense motion of the sensor, typically angular velocity and linear acceleration. Angular velocity is for example measured as rotational movement about one to three perpendicular axes, i.e., as roll, pitch and/or yaw. Translational movement is for example measured in one to three perpendicular axes, i.e., as surge, heave and sway. The motion sensor is an internal state sensor, also known as proprioceptive sensor, that records the dynamical state of a dynamic system. The motion sensor includes, for example, one or more gyroscopes for measuring angular velocity and one or more accelerometers for measuring force and/or acceleration. In some embodiments, the gyroscope is a 3-axis gyroscope measuring angular velocity in three degrees of freedom, DoF. In some embodiments, the accelerometer is a 3-axis accelerometer measuring force and/or acceleration in three DoF. A motion sensor with a 3-axis gyroscope and a 3-axis accelerometer may be referred to as a 6-axis motion sensor. One example of a motion sensor is an Inertial Measurement Unit (IMU). Hence, the motion sensor may be a 6-axis IMU. The 6-axis IMU has 6 DoF. In a particular example, the motion sensor is collocated and rigidly attached to the vehicle sensor, or rigidly mounted along with the vehicle sensor. In some embodiments, angular velocity components can be estimated based on optical flow vectors from vehicle sensor. Hence, the motion sensor is then embedded or even a part of the vehicle sensor. For newer generations of sensors like event cameras, it is possible to estimate the angular velocity seamlessly. Collocated: placed in a particular relation as close as possible to each other, for example placed side by side, together, or adjacent. The relation would typically be a rigid relation. The relation may e.g., be retrieved from a manufacturer as known extrinsic parameters. The relation may also be established with known calibration routines, such as Kalibr, which is commonly used for IMU to Camera calibration. For example, a motion sensor may be collocated with a vehicle sensor. In other words, collocated sensors are typically positioned such that sensor signals produced vary in the same, or similar, way as the vehicle moves. Online calibration: Calibration of vehicle sensors autonomously and typically in real-time. For example, the calibration is performed on the fly directly in the vehicle itself without manual intervention, typically when some driving maneuvers are performed in the vehicle to create rich sensor data. No external means are needed, like fiducial markers or similar. Extrinsic calibration: Extrinsic calibration aims to obtain the extrinsic parameters that define the geometric relationship, that is, the rotation matrix and translation vector, between two coordinate frames. Extrinsic parameters: the rotation matrix R and translation vector t between two coordinate frames. The extrinsic parameters typically include six DoF. Pose: A pose of an object comprises position and rotation of the object with respect to another object. Coordinate frames are attached to each object. The geometric relationship between these two coordinate frames may be represented by a transformation matrix including a rotation matrix R and a translation vector t. The rotation matrix R describes the orientation of one coordinate frame relative to the other coordinate frame. The translation vector describes the position of the origin of one coordinate frame with respect to the other coordinate frame. Hence, a pose comprises a rotational component and a translational component. The pose includes, for example, six DoF. Frame: A coordinate frame with an origin and three orthogonal axes, typically referred to as the x, y and z-axes. May also be referred to as a coordinate system. Reference frame: a coordinate frame that is known in relation to a known coordinate frame, such as the world coordinate frame, i.e., a world coordinate system. The reference frame may be a frame of the vehicle or more specifically the vehicle platform. The reference frame may in such case be referred to as a body frame or vehicle base frame, B. Hence, the reference frame is for example a base frame of the vehicle, which is typically present in center of the vehicle rear axis. Alternatively, the reference frame may be a frame of another sensor. The reference frame may then be referred to as a sensor frame. Fig.1 illustrates a top view of an example vehicle 1. Vehicle 1 is for example a car, bus, truck, or other kind of vehicle. Any such vehicle 1 comprises a body and chassis, engine/motor parts, drive transmission, and steering parts, suspension and brake parts, and electrical parts. The body may be integrated with the chassis, or the body and the chassis may be different parts that are fixed to each other, e.g., with bolts. The chassis is the main mounting for all parts of the vehicle and comprises a steel frame. The chassis further comprises a front axis 2 and a rear axis 3 depicted in Fig.1. The body is typically made of metal and/or fiber glass. The electrical parts comprise an electronic system including electronic control units (ECUs). It should be understood that vehicle 1 in Fig.1 is an example only, and that a vehicle in this disclosure may have, e.g., more axes and other means not illustrated herein. Vehicle 1 further comprises a plurality of sensors and a control arrangement 30. The sensors are used to monitor different functions and states of the vehicle, to provide information to the driver or to different systems of the vehicle 1. Each sensor converts sensed events or changes of a property into a signal or data that is sent to or collected by the control arrangement 30. Such signals or data are, for example, sent over a CAN (controller area network), or similar, of vehicle 1. The plurality of sensors comprises vehicle sensors 10 for monitoring the surroundings of the vehicle 1. The output from the vehicle sensors 10 may provide input to an autonomous control system of the control arrangement 30, for use in autonomous driving. These vehicle sensors 10 need to be calibrated to provide an accurate perception of the environment. The plurality of sensors also comprises motion sensors ms1, ms2. The control arrangement 30 comprises processor means 32 and memory means 34. The processor means 32 comprises one or more processors. The memory means 34 comprises one or more memories. The control arrangement 30 may include one or more ECUs. As illustrated in Fig.1, the vehicle base frame B is in the centre of the rear axis 3 of vehicle 1. Sensor readings from vehicle sensors 10 such as lidars, cameras, radars, and motion sensors ms1, ms2, are represented in their respective sensor frames respectively, where ^^^^ denotes the index of the corresponding sensor used. Each frame is illustrated with three orthogonal axes in Fig. 1. The example vehicle 1 in Fig. 1 comprises two lidars and two cameras with collocated motions sensors attached to a front part of the vehicle 1, and two lidars and one radar with collocated motions sensors attached to a rear part of the vehicle 1. A motion sensor is also attached to the centre of the rear axis 3 at the base frame B. Fig.1 also depicts a system 5 for extrinsic calibration of vehicle sensors 10 arranged on a vehicle 1, e.g., the vehicle in Fig.1. System 5 comprises a vehicle sensor 10, a first motion sensor ms1, a second motion sensor ms2 and the control arrangement 30. The first motion sensor ms1 is collocated with a vehicle sensor 10. Hence, the first motion sensor ms1 is placed side by side, together, or in a particular relation to vehicle sensor 10. In some embodiments, the first motion sensor ms1 is embedded in the vehicle sensor 10, which means that they can be handled as one unit. In particular, the first motion sensor ms1 may be a part of the vehicle sensor 10, or even the same unit. The first motion sensor ms1 is configured to sense angular velocity ω(ms1) and linear acceleration f(ms1). In some embodiments, the first motion sensor ms1 and the vehicle sensor 10 have the same orientation axis and/or has none or very small translation between them. The second motion sensor ms2 has a known pose in a reference frame of vehicle 1. Hence, the extrinsic parameters of the second motion sensor ms2 are known. The refence frame is for example the vehicle base frame B of vehicle 1, or a sensor frame of another vehicle sensor 10. Hence, the second motion sensor ms2 may be collocated with a vehicle sensor 10. The second motion sensor ms2 is configured to sense angular velocity ω(ms2) and linear acceleration f(ms2). The second motion sensor ms2 may in some embodiments be a motion sensor of a GNSS, Global Navigation Satellite System, of vehicle 1. In some embodiments, the first motion sensor ms1 and the second motion sensor ms2 are Inertial Measurement Units (IMUs). The control arrangement 30 is configured to calibrate the vehicle sensor 10 based on angular velocity ω(ms1) and linear acceleration f(ms1) of the first motion sensor ms1, angular velocity ω(ms2) and linear acceleration f(ms2) of the second motion sensor ms2. The calibration is further based on an assumption that as angular velocity of a rigid body is the same at all points of said rigid body, angular velocity of the first motion sensor ms1 and the second motion sensor ms2 are the same when the orientations of the first motion sensor ms1 and of the second motion sensor ms2 are the same, which will be further described in the following. The disclosure also relates to a vehicle 1 comprising system 5. In the following a method for extrinsic calibration of a vehicle sensor 10 arranged on a vehicle, for example the vehicle 1 in Fig.1, will be described. The method may be performed online while the vehicle is driving. Thereby the vehicle sensors 10 may be calibrated repeatedly (or continually in an ongoing manner) without any interruption of the operation of the vehicle 1. The method may be implemented as a computer program. The computer program then comprises instructions which, when the computer program is executed by a computer, cause the computer to carry out the method according to any one of the aspects, embodiments or examples as described herein. The computer is for example included in the control arrangement 30 as previously described. For example, as including processor means 32 and memory means 34 of the control arrangement 30. Hence, in some embodiments, the system 5 is configured to perform the method according to any one of the aspects, embodiments or examples as described herein. The disclosure also related to a computer-readable medium comprising instructions which, when executed by a computer, cause the computer to carry out the method according to any one of the aspects, embodiments or examples as described herein. The computer-readable memory is for example one or more of the memories in the control arrangement 30. The method will now be described in relation to the flow chart in Fig.3, and to the illustrations in Fig.1 and 2. Sensor data is collected from the first and second motion sensors. In other words, the method comprises obtaining S1, from the first motion sensor ms1, angular velocity ω(ms1) and linear acceleration f(ms1) of the first motion sensor ms1. The first motion sensor has a known pose in a reference frame of vehicle 1. The method further comprises obtaining S2, from the second motion sensor ms2 collocated with the vehicle sensor 10, angular velocity ω(ms2) and linear acceleration f(ms2) of the second motion sensor ms2. The collected sensor data is then used for calibrating the sensors. In other words, the method comprises calibrating S3 the vehicle sensor 10 based on angular velocity ω(ms1) and linear acceleration f(ms1) of a first motion sensor ms1 collocated with the vehicle sensor 10, and angular velocity ω(ms2) and linear acceleration f(ms2) of a second motion sensor ms2 having a known pose in a reference frame of the vehicle 1. The collected sensor data from the first motion sensor ms1 and the second motion sensor ms2 is typically synchronized in time, e.g., concurrent. If the sensors sense with different frequencies, measurement data from one or both of the sensors may be interpolated to establish data sensed at the same time instance, where a small difference in timing may be allowed. The calibration is also based on an assumption that as angular velocity of a rigid body is the same at all points of said rigid body, angular velocity of the first motion sensor ms1 and the second motion sensor ms2 are the same when the orientations of the first motion sensor ms1 and of the second motion sensor ms2 are the same. Hence, extrinsic parameters of vehicle sensor 10 can be estimated as the rotation matrix of the first motion sensor ms1 in the reference frame, where the angular velocities of the first motion sensor ms1 and the second motion sensor ms2 are the same. The sensors may be arranged on the vehicle 1 such that the assumption can be used in full. In some embodiments, this means that the first motion sensor MS1, the second motion sensor MS2 and the vehicle sensor 10 are rigidly arranged on vehicle 1. Hence, their sensor readings directly mirror the movement of vehicle 1. In other words, movement of vehicle 1 will induce a corresponding movement of the sensors. Additionally, in some embodiments, the first motion sensor MS1, the second motion sensor MS2 and the vehicle sensor 10 are rigidly arranged on the same rigid body of vehicle 1. The rigid body is, for example, the chassis or the body of vehicle 1. In other words, movement of vehicle 1 will induce a corresponding identical movement of all the sensors. Alternatively, the first motion sensor MS1, the second motion sensor MS2 and/or the vehicle sensor 10 are arranged with a known relationship, respectively, to the rigid body. “Rigid” here means stiff. “Rigidly arranged” or rigidly attached means that the parts to not move in relation to each other. Hence, they have a stiff relation and move strictly together. Rigidly arranged may include that they are mounted and/or bolted together directly or indirectly. The method provides a unified solution of calibrating all vehicle sensors 10 of different modalities together with only one method. For each vehicle sensor 10 to be calibrated, a first motion sensor ms1 is arranged in collocation with, hence collocated with, each vehicle sensor 10. The invention is based on that a calibration of the first motion sensor ms1 can be transferred to the vehicle sensor 10 as they are collocated. They will then typically be affected in the same way by vehicle movements while the vehicle is driving. Thus, sensor readings from the collocated first motion sensor ms1 can be used as if they were coming from the vehicle sensor 10 itself or translated to be coming from the vehicle sensor 10 using a known relation between the frames of the first motion sensor ms1 and the vehicle sensor 10. A relation between a frame of the vehicle sensor 10 and a frame of the first motion sensor ms1 is typically known. For example, if the first motion sensor ms1 is embedded in vehicle sensor 10, their frames are essentially the same. In other embodiments, the relation may be another, previously known or calibrated relation. This relationship is consistent over time, irrespective of how the vehicle moves. The relation may thus be used to translate sensor readings of the first motion sensor 1 to the vehicle sensor 10, or vice versa. The second motion sensor ms2 has a known pose in a reference frame of vehicle 1. Hence, the frame of the second motion sensor ms2 is always known. The second motion sensor ms2 is for example placed at the vehicle base frame B such that the frame of the second motion sensor ms2 and the vehicle base frame B are the same. The prior extrinsic parameters of the vehicle base frame B are for example obtained from vehicle CAD (Computer Aided Design) parameters. Alternatively, the second motion sensor ms2 is collocated with another vehicle sensor 10. The frame of the second motion sensor ms2 and the frame of the other vehicle sensor 10 are then essentially the same. The reference frame is then the sensor frame of the other vehicle sensor 10. A geometric relation between two vehicle sensors 10 can then be established. Hence, the rotation matrix R and the translation vector for the second motion sensor ms2 are known. Calibrating the vehicle sensor 10 typically includes determining the pose, extrinsic parameters and/or transformation matrix of the vehicle sensor 10 in relation to the reference frame. The extrinsic parameters/transformation matrix are used to translate the sensor measurements of the vehicle sensor 10 into, e.g., the vehicle base frame B. By matching, e.g., comparing, the angular velocity from the motion sensors ms1, ms2, a rotation matrix of the vehicle sensor 10 in the reference frame can be estimated. Further, by matching the linear acceleration from the motion sensors ms1, ms2, a translation vector of the vehicle sensor 10 in the reference frame can be estimated. Hence, in some embodiments, the calibrating S3 comprises determining a pose of the vehicle sensor 10 in the reference frame by matching angular velocity ω(ms1) of the first motion sensor ms1 with angular velocity ω(ms2) of the second motion sensor ms2 and matching linear acceleration f(ms1) of the first motion sensor ms1 with linear acceleration f(ms2) of the second motion sensor ms2. The matching, or in other words comparing, is for example performed by establishing and solving optimization problems, where the errors between the angular velocities and the linear accelerations are minimized. Hence, in some embodiments, the matching includes finding a pose of the vehicle sensor 10 in the reference frame where a difference between the angular velocity ω(ms1) of the first motion sensor ms1 and the angular velocity ω(ms2) of the second motion sensor ms2, and a difference between the linear acceleration f(ms1) of the first motion sensor ms1 and the linear acceleration f(ms2) of the second motion sensor ms2, are minimized. For example, let, [ ^^^^ ^^^^_{( ^^^^)} _^ , ^^^^ _{^ ( ^^^^)} _{^^^ ^^^ ^^^^} ] be the linear acceleration and angular velocity of the ^^^^ ^{^^^^ℎ} first motion sensor ms1 collocated with the vehicle sensor 10. Similarly, let ^^^^ ^{^^^^ℎ} ^_{^^^ ^^^^} _^^^^] be the linear acceleration and angular velocity of the ^^^^ second motion sensor ms2 which is arranged at reference frame B for which the prior extrinsic parameters can be obtained from vehicle CAD parameters. In this example, the first motion sensor ms1, and hence the vehicle sensor 10, is calibrated with respect to the vehicle base frame. By removing the ^^^^ ^{^^^^ℎ} superscript for brevity, the optimization problem may be established as, ^^^^ _^ ^∗ ^_{^^ ^^^^} = arg ⁽¹⁾ ^^^^ ^_^^^. which can be solved using various 3D-alignment techniques. ^^^^ _^ ^∗ ^_{^^ ^^^^} is here the translation matrix from the base frame B to the sensor frame I. “ ^^^^” denotes a time instance. For the acceleration components, the Coriolis forces are compensated for as illustrated in Fig.2. Fig.2 illustrates a stripped version of Fig.1 where the vehicle sensor frames have been removed for ease of illustration. A translation vector ^^^^ _^ ^∗ ^_{^^ ^^^^} between the base frame B and a motion sensor frame I is illustrated with an arrow. The translation components can be equated as, where ^^^^ _^^^^ × ( ^^^^ _^^^^ × ^^^^ _{^^^^ ^^^^}) is the centrifugal force (C) and ^^̇^^ _^^^^ × ^^^^ _{^^^^ ^^^^} is the Euler force (E) and [. ]_× is a skew-symmetric matrix. Since ^^^^ _^ ^∗ ^_{^^ ^^^^} is already computed, the optimization problem for the translation component becomes, and ^^^^ = [ ^^^^₀ ^^^^₁ … ^^^^ _^^^^] ^{^^^^} and ^^^^ = ^^^^ _^ ^∗ ^_{^^ ^^^^} ∀ ^^^^ ∈ [1, ^^^^]. Hence, in some embodiments the calibrating S3 comprises compensating linear acceleration from the first motion sensor ms1 for Coriolis effects. Since this is also a system of linear equations of the form, ^^^^ ^^^^ = ^^^^, it can be solved using bounded variable least squares (BVLS) if prior bounds from the CAD parameters are known. It can also be solved using a closed- form solution, however, BVLS can bound the solution within known bounds. The least-square problem can be formulated as, ^^^^ = arg where ^∑ denotes the co-variance of the residual. For these problems, the general strategy is to solve a sequence of approximations iteratively to the original problem _{by linearizing as ^^^^} ⁽ _{^^^^ + ∆ ^^^^} ⁾ _{≈ ^^^^} ⁽ _^^^^ ⁾ _{+ ^^^^} ⁽ _^^^^ ⁾ _{∆ ^^^^, where ^^^^ being the Jacobian of} ^^^^( ^^^^).Thus, ^^�^^ is updated in the current iteration as ^^�^^ ← ^^�^^ ⊞ ∆ ^^^^, where ⊞ is an additional operator in the manifold and the problem becomes, ∆ ^^^^ = arg The optimal solution is given by, ( ^^^^^{^ ∑−1} ^^^^ ), ∆ ^^^^ = − ^^^^^{^ ∑−1} ^^^^ ( ^^^^ ^^^^ − ^^^^) (6) where ^^^^^{^} ∑⁻¹ ^^^^ is the Fisher information matrix. If there are reliable initial estimates for the translation component from the CAD parameters, then ^^^^^∗ can be searched only in a local neighborhood within reasonable bounds. Thus, the bounded variable least square problem becomes, where L and U are the lower and upper bounds of x, respectively. Thus, the solution space in equation 6 is modified with an additional constraint as, ∆ ^^^^^∗ = arg ^^^^. ^^^^. ^^^^ ≤ ^^^^ + ∆ ^^^^ ≤ ^^^^ The method relies on data from the motion sensors that has sufficient information in all relevant DoF. This might include selecting data from driving on road segments that have a curvature and/or are hilly. In other words, in some embodiments the calibrating S3 comprises selecting angular velocity ω(ms1) and linear acceleration f(ms1) of the first motion sensor ms1, and angular velocity ω(ms2) and linear acceleration f(ms2) of a second motion sensor ms2 that fulfill excitation criteria and using the selected angular velocities and linear accelerations for the calibration. In some embodiments, the motion sensors are 6-axis motion sensors, e.g., IMUs, hence three axes för measuring angular velocity, and three axes for measuring linear acceleration. In other words, in some embodiments, the angular velocity of the first motion sensor ms1 and second motion sensor ms2 includes angular velocity in three DoF, and the linear acceleration of the first motion sensor ms1 and second motion sensor ms2 includes linear acceleration in three DoF. The method has mainly been described in relation to calibrating one vehicle sensor 10. However, as understood from the generality of the invention, the method can be used for online extrinsic calibration of a plurality of vehicle sensors 10 of the same vehicle 1. Also, as the method is agnostic to different modalities of vehicle sensors, the same method can be used for different modalities of vehicle sensors as well as for same modality vehicle sensors. Hence, the method can be used for online extrinsic calibration of at least two different types of modalities of vehicle sensors 10 of the same vehicle 1. In some embodiments, the method is used for online extrinsic calibration of three or four different modalities of sensors, however, the method can be extended to more modalities of sensors, there is no known limit. The calibrated sensor data may be used for various purposes. In some embodiments, the method includes providing S4 calibrated sensor data from the vehicle sensor 10 to a control arrangement 30 of the vehicle 1 for use in autonomous driving. The sensor data in the reference frame can then be fused with other sensor data in the same reference frame into a common picture of the surroundings of vehicle 1 and provide a more accurate base for decisions. Fig. 4 are diagrams of signals from a first motion sensor and a second motion sensor without calibration. Fig. 5 are diagrams of signals from the first motion sensor and the second motion sensor where signals of the first motion sensor are calibrated using the proposed technique. The left-hand side diagrams in Figs.4 and 5 illustrate translational movement as linear acceleration (m/s²) along three orthogonal x, y and z-axes versus time. The right-hand side diagrams in Fig. 4 and 5 illustrate rotational movement as angular velocity (rad/s) around three orthogonal x, y and z-axes versus time. The signals are retrieved from an example calibration of two lidars of Figs. 1 and 2, where the lidars have collocated IMUs embedded inside them. Hence, signals from a Front-Left-Top (FLT) and Front- Right-Top (FRT) IMU sensors of the respective FLT and FRT lidars in Fig.1 to 2 were used. The FLT IMU sensor signal is illustrated with a dashed-dotted line, and the FRT IMU sensor signal is illustrated with a solid line. The first motion sensor and the second motion sensor are thus IMUs embedded in the lidars. The signals are real-time data from the IMU sensors. To calibrate the lidars the IMU signals were matched, using the calibration method described herein. Note that the extrinsic parameters between the lidar and the embedded IMU were provided by the sensor supplier. The uncalibrated IMU signals look out of phase as can be seen in Fig.4. Instead, with the proposed method, a matching can be seen as in Fig.5, where the signals overlap to a greater extent than in Fig.4. Hence, the IMU signals for both the linear acceleration and angular velocity have matched and hence also the extrinsic parameters in between the lidars are recovered. The terminology used in the description of the embodiments as illustrated in the accompanying drawings is not intended to be limiting of the described method, control arrangement or computer program. Various changes, substitutions and/or alterations may be made, without departing from disclosure embodiments as defined by the appended claims. The term “or” as used herein, is to be interpreted as a mathematical OR, i.e., as an inclusive disjunction; not as a mathematical exclusive OR (XOR), unless expressly stated otherwise. In addition, the singular forms "a", "an" and "the" are to be interpreted as “at least one”, thus also possibly comprising a plurality of entities of the same kind, unless expressly stated otherwise. It will be further understood that the terms "includes", "comprises", "including" and/ or "comprising", specifies the presence of stated features, actions, integers, steps, operations, elements, and/ or components, but do not preclude the presence or addition of one or more other features, actions, integers, steps, operations, elements, components, and/ or groups thereof. A single unit such as e.g. a processor may fulfil the functions of several items recited in the claims. The present disclosure is not limited to the above-described preferred embodiments. Various alternatives, modifications and equivalents may be used. Therefore, the above embodiments should not be taken as limiting the scope of the disclosure, which is defined by the appending claims.

Claims

Claims 1. A method for extrinsic calibration of a vehicle sensor (10) arranged on a vehicle (1), comprising - calibrating (S3) the vehicle sensor (10) based on o angular velocity (ω(ms1)) and linear acceleration (f(ms1) of a first motion sensor (ms1) collocated with the vehicle sensor (10), o angular velocity (ω(ms2)) and linear acceleration (f(ms2)) of a second motion sensor (ms2) having a known pose in a reference frame of the vehicle (1), and o an assumption that as angular velocity of a rigid body is the same at all points of said rigid body, angular velocity of the first motion sensor (ms1) and the second motion sensor (ms2) are the same when the orientations of the first motion sensor (ms1) and of the second motion sensor (ms2) are the same.

2. The method according to claim 1, wherein the method is performed online while the vehicle is driving.

3. The method according to claim 1 or 2, comprising providing (S4) calibrated sensor data from the vehicle sensor (10) to a control arrangement (30) of the vehicle (1) for use in autonomous driving.

4. The method according to any one of the preceding claims, wherein the first motion sensor (MS1), the second motion sensor (MS2) and the vehicle sensor (10) are rigidly arranged on the vehicle.

5. The method according to any one of the preceding claims, wherein the first motion sensor (MS1), the second motion sensor (MS2) and the vehicle sensor (10) are rigidly arranged on a same rigid body of the vehicle.

6. The method according to any one of the preceding claims, wherein the calibrating (S3) comprises determining a pose of the vehicle sensor (10) in the reference frame by matching the angular velocity (ω(ms1)) of the first motion sensor (ms1) with the angular velocity (ω(ms2)) of the second motion sensor (ms2) and matching the linear acceleration (f(ms1) of the first motion sensor (ms1) with the linear acceleration (f(ms2)) of the second motion sensor (ms2).

7. The method according to claim 8, wherein the matching includes finding a pose of the vehicle sensor (10) in the reference frame where a difference between the angular velocity (ω(ms1)) of the first motion sensor (ms1) and the angular velocity (ω(ms2)) of the second motion sensor (ms2), and a difference between the linear acceleration (f(ms1) of the first motion sensor (ms1) and the linear acceleration (f(ms2)) of the second motion sensor (ms2), are minimized.

8. The method according to claims 6 or 7, wherein the calibrating (S3) comprises compensating linear acceleration from the first motion sensor (ms1) for Coriolis effects.

9. The method according to any one of the preceding claims, wherein the pose comprises a rotational component and a translational component.

10. The method according to any one of the preceding claims, comprising - obtaining (S1), from the first motion sensor (ms1) having a known pose in a reference frame of the vehicle (1), angular velocity (ω(ms1)) and linear acceleration (f(ms1)) of the first motion sensor (ms1); - obtaining (S2), from the second motion sensor (ms2) collocated with the vehicle sensor (10), angular velocity (ω(ms2)) and linear acceleration (f(ms2)) of the second motion sensor (ms2).

11. The method according to any one of the preceding claims, wherein the angular velocity of the first motion sensor (ms1) and second motion sensor (ms2) includes angular velocity in three degrees of freedom, DoF, and the linear acceleration of the first motion sensor (ms1) and second motion sensor (ms2) includes linear acceleration in three degrees of freedom, DoF.

12. The method according to any one of the preceding claims, wherein the first motion sensor (ms1) and the second motion sensor (ms2) are Inertial Measurement Units (IMUs).

13. The method according to any one of the preceding claims, wherein data from the first motion sensor (ms1) and data from the second motion sensor (ms2) are synchronized in time.

14. The method according to any one of the preceding claims, wherein the calibrating (S3) comprises selecting angular velocity (ω(ms1)) and linear acceleration (f(ms1) of the first motion sensor (ms1), and angular velocity (ω(ms2)) and linear acceleration (f(ms2)) of a second motion sensor (ms2) that fulfill excitation criteria and using the selected angular velocities and linear accelerations for the calibration.

15. The method according to any one of the preceding claims, wherein the vehicle sensor (10) is any of a camera, a lidar, a radar, an ultrasonic sensor or any other vehicle sensor.

16. The method according to any one of the preceding claims, wherein the reference frame is a base frame of the vehicle.

17. The method according to any one of the preceding claims, wherein a relation between a frame of the vehicle sensor (10) and a frame of the first motion sensor (ms1) is known.

18. Use of the method according to any one of the preceding claims, for online extrinsic calibration of a plurality of vehicle sensors (10) of the same vehicle (1).

19. Use of the method according to any one of the preceding claims, for online extrinsic calibration of at least two different types of modalities of vehicle sensors (10) of the same vehicle (1).

20. A computer program comprising instructions which, when the computer program is executed by a computer, cause the computer to carry out the method according to any one of the preceding claims.

21. A computer-readable medium comprising instructions which, when executed by a computer, cause the computer to carry out the method according to any one of the claims 1 to 17.

22. A system (1) for extrinsic calibration of a vehicle sensor (10) arranged on a vehicle (1), comprising - a first motion sensor (ms1) collocated with the vehicle sensor (10), the first motion sensor (ms1) being configured to sense angular velocity (ω(ms1)) and linear acceleration (f(ms1)), - a second motion sensor (ms2) having a known pose in a reference frame of the vehicle (1), the second motion sensor (ms2) being configured to sense angular velocity (ω(ms2)) and linear acceleration (f(ms2), and - a control arrangement (30) configured to calibrate the vehicle sensor (10) based on o angular velocity (ω(ms1)) and linear acceleration (f(ms1) of the first motion sensor (ms1), o angular velocity (ω(ms2)) and linear acceleration (f(ms2)) of the second motion sensor (ms2), and o an assumption that as angular velocity of a rigid body is the same at all points of said rigid body, angular velocity of the first motion sensor (ms1) and the second motion sensor (ms2) are the same when the orientations of the first motion sensor (ms1) and of the second motion sensor (ms2) are the same.

23. A vehicle (1) comprising the system according to claim 22. 5