Classifications

• • G—PHYSICS
  • G01—MEASURING; TESTING
  • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
  • G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
  • G01S7/02—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S13/00
  • G01S7/40—Means for monitoring or calibrating
  • G01S7/4004—Means for monitoring or calibrating of parts of a radar system
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T7/00—Image analysis
  • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• • G—PHYSICS
  • G01—MEASURING; TESTING
  • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
  • G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
  • G01S13/86—Combinations of radar systems with non-radar systems, e.g. sonar, direction finder
  • G01S13/865—Combination of radar systems with lidar systems
• • G—PHYSICS
  • G01—MEASURING; TESTING
  • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
  • G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
  • G01S13/86—Combinations of radar systems with non-radar systems, e.g. sonar, direction finder
  • G01S13/867—Combination of radar systems with cameras
• • G—PHYSICS
  • G01—MEASURING; TESTING
  • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
  • G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
  • G01S13/88—Radar or analogous systems specially adapted for specific applications
  • G01S13/93—Radar or analogous systems specially adapted for specific applications for anti-collision purposes
  • G01S13/931—Radar or analogous systems specially adapted for specific applications for anti-collision purposes of land vehicles
• • G—PHYSICS
  • G01—MEASURING; TESTING
  • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
  • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
  • G01S17/87—Combinations of systems using electromagnetic waves other than radio waves
• • G—PHYSICS
  • G01—MEASURING; TESTING
  • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
  • G01S19/00—Satellite radio beacon positioning systems; Determining position, velocity or attitude using signals transmitted by such systems
  • G01S19/01—Satellite radio beacon positioning systems transmitting time-stamped messages, e.g. GPS [Global Positioning System], GLONASS [Global Orbiting Navigation Satellite System] or GALILEO
  • G01S19/13—Receivers
  • G01S19/23—Testing, monitoring, correcting or calibrating of receiver elements
• • G—PHYSICS
  • G01—MEASURING; TESTING
  • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
  • G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
  • G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
  • G01S7/497—Means for monitoring or calibrating
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T7/00—Image analysis
  • G06T7/70—Determining position or orientation of objects or cameras
  • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T7/00—Image analysis
  • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
  • G06T7/85—Stereo camera calibration
• • B—PERFORMING OPERATIONS; TRANSPORTING
  • B60—VEHICLES IN GENERAL
  • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
  • B60W2420/00—Indexing codes relating to the type of sensors based on the principle of their operation
  • B60W2420/40—Photo, light or radio wave sensitive means, e.g. infrared sensors
  • B60W2420/403—Image sensing, e.g. optical camera
• • B—PERFORMING OPERATIONS; TRANSPORTING
  • B60—VEHICLES IN GENERAL
  • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
  • B60W2420/00—Indexing codes relating to the type of sensors based on the principle of their operation
  • B60W2420/40—Photo, light or radio wave sensitive means, e.g. infrared sensors
  • B60W2420/408—Radar; Laser, e.g. lidar
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/10—Image acquisition modality
  • G06T2207/10004—Still image; Photographic image
  • G06T2207/10012—Stereo images
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/10—Image acquisition modality
  • G06T2207/10028—Range image; Depth image; 3D point clouds
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/30—Subject of image; Context of image processing
  • G06T2207/30248—Vehicle exterior or interior
  • G06T2207/30252—Vehicle exterior; Vicinity of vehicle
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/30—Subject of image; Context of image processing
  • G06T2207/30248—Vehicle exterior or interior
  • G06T2207/30252—Vehicle exterior; Vicinity of vehicle
  • G06T2207/30261—Obstacle

Definitions

• the present invention relates to a method for calibrating a first and a second sensor of a vehicle and a calibration unit for calibrating a first and a second sensor of a vehicle.
• the present invention also relates to a computer-readable storage medium storing program code, the program code comprising instructions for carrying out such a method.
• Modern cars both autonomous and non-autonomous, comprise an increasing number of sensors of different modalities. These sensors need to be calibrated.
• an intrinsic calibration of a camera comprises determining a set of parameters for a camera projection model that relates 2D image points to 3D scene points, where optimal parameters correspond to minimal reprojection error. If there are sensors of more than one modality, an extrinsic calibration is required. Extrinsic calibration can involve determining parameters that match points from one sensor to the corresponding points of another sensor.
• the objective of the present invention is to provide a method for calibrating a first and a second sensor of a vehicle and a calibration unit for calibrating a first and a second sensor of a vehicle, which overcome one or more of the above-mentioned problems of the prior art.
• a first aspect of the invention provides a method for calibrating a first and a second sensor of a vehicle, the method comprising: obtaining first data from the first sensor and second data from the second sensor, filtering at least the second data based on positions of data points of the second data, and determining one or more parameters of a calibration between the first and the second sensor based on the first data and the filtered second data.
• the method of the first aspect has the advantage that the calibration is performed based on the filtered second data.
• the filtering can be performed such that clearly irrelevant points are filtered out and the calibration is performed on those points that have not been filtered out.
• a probability of an erroneous calibration can be reduced and an overall reliability of the calibration is improved.
• the first and/or second sensor can comprise a camera.
• Obtaining first and/or second data can comprise obtaining first and/or second images.
• the method can calibrate additional sensors at the same time.
• Filtering at least the second data means that optionally also, the first data can be filtered.
• Filtering at least the second data can comprise that datapoints of the second data that have positions that fall in certain one or more regions, e.g. one or more predetermined certain regions, are filtered out, i.e., removed.
• datapoints that have positions that are outside a certain target region can be removed.
• the target region may depend on properties of the steering, e.g., the steering direction of the vehicle.
• the calibration may comprise an alignment between the first and second sensor, e.g., the calibration may comprise three parameters for translation and tree parameters for rotation between the coordinate system of the first sensor and the second sensor.
• the method further comprises using an a priori calibration to convert the first data and/ or the second data into a common domain.
• a priori knowledge about the relative orientation between the first and second sensor can be used as a starting point for determining the final calibration between the first and the second sensor. For example, knowledge about a mounting position of the first sensor relative to a mounting position of the second sensor can be used to determine the a prior calibration.
• the method further comprises a step of detecting first objects in the first data and/or detecting second objects in the filtered second data and performing the determining of the one or more parameters based on the first and/or second objects.
• Performing the calibration based on detected objects has the advantage that even if the first and second sensor use different modalities (which may result in different kinds of datapoints in the first and second data), accurate calibration can be performed - based on the detected objects which may be fully corresponding in the first and second data.
• the determining the one or more parameters is based on center points of the first and/or second objects.
• the first sensor may acquire many datapoints for a given object, whereas the second sensor might require fewer datapoints for the same given object.
• the same center points can be determined.
• the first sensor and/ or the second sensor comprise one or more of a regular camera, a stereo camera, a radar, and a lidar.
• the first and/or second sensor may comprise any further sensor modalities providing data proper for executing the method on.
• the one or more parameters of the calibration include yaw and pitch parameters
• the method further comprises filtering the second data based on a filter region that has a region width around a center line from the sensor towards a distant point, wherein the distant point has a lateral and/or vertical coordinate component that corresponds to the lateral and/ or vertical coordinate component of a position of the sensor.
• the filter region can be rectangular region (seen from above) in front of the vehicle. Experiments have shown that this region is particularly useful as filter region.
• Filter region herein refers to that region where the datapoints are not filtered out, i.e., where the datapoints are kept and used in the further calibration process.
• the region width is determined based on a predetermined table that assigns a predetermined width to a given pair of first and second sensor, and/ or the region width is determined based on a position and/ or an orientation of the sensor.
• a position of the distant point in the coordinates of a reference sensor is determined as:
• Pref_sensor ⁇ ref sensor ® ⁇ cal sensor Pcal_sensor > wherein p cai sensor is a position of the distant point in coordinates of a calibration sensor, Pref_sensor is the position of the distant point in the reference sensor’s coordinates, T is a matrix transforming between the coordinate systems of the calibration sensor and the reference sensor, and K cal sensor and K ref _ sensor are the respective camera matrix transformations.
• K cai sensor an d ⁇ retsensor can be identity matrices.
• the used matrix depends on the used type of sensor.
• the one or more parameters of the calibration include at least one roll parameter
• the method further comprises filtering the second data based on a filtering region that has a fixed width around a center line from the sensor towards a distant point that in coordinates of the sensor is laterally shifted from the sensor position by a lateral shift.
• the filtering the second data comprises filtering based on a filtering region, wherein the filtering region is adjusted based on an angle of a steering direction of the vehicle.
• the determining a lateral location of the filtering region comprises evaluating a sinus of an angle of the steering direction, wherein preferably the evaluating comprises multiplying the sinus of the angle of the steering direction with a sensitivity constant, preferably further multiplying with a constant proportional to a focal length of a camera sensor.
• the method is carried out repeatedly, using a plurality of candidate a priori calibrations between the first and second sensor, and wherein preferably, the method comprises an additional step of choosing a preferred a priori calibration from the plurality of candidate a priori calibrations.
• the determining the one or more parameters of the calibration between the first sensor and the second sensor comprises: determining a pairing between points in the first data and points in the second data, and determining the one or more parameters such that an error between paired points is minimized.
• the calibrating the first and the second sensor further comprises calibrating a third sensor with the first and the second sensor, and the determining one or more parameter of the calibration comprises minimizing an error of point pairs between the first and second sensor, point pairs between the second and third sensor, and point pairs between the third and first sensor.
• the minimizing of the error of the point pairs comprises minimizing a weighted sum of the errors of the point pairs of the sensor pairs.
• the method further comprises validating the a priori calibration, wherein the validating comprises comparing the one or more parameters of the determined calibration with one or more corresponding parameters of an a priori calibration and validating the calibration if a difference between the determined calibration and the a priori calibration is smaller than a predetermined threshold.
• the validating comprises, for each of a plurality of sensors, which comprise the first and the second sensor: determining first center points of detected objects as projected onto coordinates of another sensor of the plurality of sensors using the parameters of the determined calibration, determining second center points of detected objects as projected onto coordinates of the other sensor of the plurality of sensors using parameters of the a priori calibration, and determining a distance between the first and the second center points, wherein preferably, the calibration between the sensor and the other sensor of the plurality of sensors is validated if the distance is smaller than a predetermined threshold.
• a second aspect of the invention provides a calibration system for calibrating a first and a second sensor of a vehicle, the calibration unit comprising: an obtaining unit for obtaining first data from a first sensor and second data from the second sensor, a filtering unit for filtering at least the second data based on positions of data points of the second data, and a determining unit for determining one or more parameters of a calibration between the first and the second sensor based on the first data and the filtered second data
• the calibration unit can be a calibration system.
• a further aspect of the invention refers to a computer-readable storage medium storing program code, the program code comprising instructions that when executed by a processor carry out the method of the second aspect or one of the implementations of the first aspect.
• FIG. i is a schematic illustration of vehicles that are detected as objects in front of a car in accordance with an embodiment of the present invention
• FIG. 2A is a schematic illustration of a car and the view regions of its two sensors (a radar and a camera) in accordance with an embodiment of the present invention
• Fig. 2B is a schematic illustration of a car, the view region of a sensor of the car and the base filter area, as well as the shifted filter area based on a steering angle of the car, in accordance with an embodiment of the present invention.
• FIGs. 3A and 3B are a flow chart of a method in accordance with an embodiment of the present invention.
• the sensor to be calibrated may provide additional data, e.g., the velocity, intensity, cross-section size and/or confidence of detections. These may be used for aiding the calibration process.
• these tracks can be further utilized for aiding the calibration process. (The tracks may also exhibit further properties, like physical size, confidence, stc.)
• Point cloud + point cloud with tracking information On the regular point cloud, object detection is executed. These objects can then be paired with the tracks.
• Point cloud + point cloud Object detection based, analog to the previous point. Also note, that since the density of these point clouds may differ greatly, the one-to-one pairing of them may not be an option.
• a lidar device may detect tens of thousands of points with each measurement, while a radar device may detect only ten to a hundred points with each of its measurements, making the one- to-one pairing of such data cumbersome.
• each detected object is represented with a single point.
• different (state-of-the-art) image processing and/or object detection methods may be used.
• the detected objects be the cars visible on the camera frame.
• the radar frames For processing the radar frames, we may use several methods, just like for the camera frames. For example, using the relative velocities provided by most radars, we can select which objects are moving and which are stationary. Furthermore, the point cloud can be clustered (several algorithms exist for this purpose), then the center points of the objects found this way can be easily set.
• FIG. i shows how a sensor too, e.g., a camera, sees a street 120 with vehicles 122, 124, and 126 in front of the vehicle (not shown), which carries the sensor too.
• a sensor too e.g., a camera
• the method may be implemented differently when using different sensors: Using a radar device capable of tracking objects, the clustering step may be eliminated, since we get the required positions from the device itself. Using e.g. a lidar instead of the camera, one can cluster its point cloud just like that of the radar.
• the vehicles 122, 124, 126, bounding boxes 132, 134? 136 and center points of the bounding boxes 132, 134, 136 can be identified.
• the crosses 142, 144, 146 in the cars are indicate the centers of the objects detected on the camera frames using a neural network.
• the circles 152, 154, 156 indicate the centers of the clusters detected on the radar frames projected onto the image.
• the centers 142, 144, 146 of the objects detected on the camera frame are not fully aligned with the centers of the clusters detected on the radar frames.
• both the data from a first sensor and the data from a second sensor can be filtered and in that case different filtering rules can be applied to the data from the different sensors.
• At least one of the applied filtering methods is based on a position of the data, i.e., the decision which datapoint to keep and which datapoint to filter out is based on a position of the datapoint in the data.
• the a priori calibration may contain significant errors.
• a goal of the presented method includes the elimination of these errors. If no or just minimal error is present, the distances of the point pairs is minimal, thus the result of the calibration will be highly similar to the a priori.
• the method can also be used for the validation of existing calibration parameters.
• the selection of the sensor pairs can be done by either the user or the software.
• the pair may be any combination of the camera, radar and lidar devices present, provided they share a common view region.
• the center of its view region shall be selected.
• the perimeter of its view region shall be selected.
• the selected view region depends on the first sensor’s view region and the a priori calibration (e.g. one sensor may look forward and the other may look sideways).
• Selecting the width of the filtered region can be done by using a pre-defined list, containing each sensor. (In our case, the widths were around 3-4 meters, which is equal to the width of a road lane.)
• the region is well defined.
• the utilized steps are the same as before, except for the position of the virtual point - its lateral distance is not zero, rather a value with which objects on the perimeter region of the view region can be optimally selected.
• FIG. 2A is a schematic illustration of a car 200 and the view regions of its two sensors (a radar 210 and a camera 220). Specifically, the radar 210 has a view region 212 and the camera 220 has a view region 222, which partially overlaps with the view region 212 of the radar.
• a left boundary 234a and a right boundary 234b of the base filter region 240 are defined as lines that are parallel to a center line (not shown in FIG. 2B) that is defined by longitudinal middle of the car. In horizontal direction, these boundaries 234a, 234b have an equal distance to the center line.
• the method’s execution may depend on the a priori calibration minimally, provided one of the paired sensors greatly differs from the direction the vehicle travels, while the other’s is well aligned with this traveling direction.
• the error of the a priori calibration is realized as an error in the projected filtered region.
• This dependency on the a priori calibration can be loosened by several methods: o If the first result is not ideal, executing the method iteratively will lead to a better solution. o Executing the filtering several times, with differently modified a priori calibrations can compensate for its error. (With increasing the number of said filtering executions, the execution time also increases, though since the filtering algorithm is lightweight (see before), the whole process will still remain lightweight.)
• the selected filtering region may also be modified based on the steering wheel angle. This may further increase the success of pairing, since on a curved road, the other cars may not be in the filtered region if it is not modified when steering.
• FIG. 2B is a schematic illustration of the car 200, the view region 234 of a sensor 230 of the car and a base filter area 240 as well as a shifted filter area 242 based on a steering angle of the car.
• the base filter area 240 which corresponds to the filter area when the car is driving straight, has boundaries 234a, 234b.
• the filter region is also shifted to the left, thus obtaining a new filter region 242, with boundaries 236a, 236b that are shifted to the left from the boundaries 234a, 234b of the base filter region.
• the shift of the boundaries is proportional to a steering angle or a sine of the steering angle or another function of the steering angle.
• the width of the shifted filter region may be the same as the width of the base filter region or in other embodiments it may also vary based on a function of the steering angle.
• the left-shifting of the filter region has the advantage that the filter region is shifted in the direction that the vehicle is steering to. Thus, it is more likely that a lead car 202 falls in the filter region. Even though the filtering significantly reduces the number of datapoints that fall in the overlap of the filter region and the view region, it is very likely that the lead car falls into this overlap.
• the car driving in front of the vehicle that carries the sensor is out of the center region as seen on the radar frames, if this region is not offset by the steering angle.
• using the steering angle for modifying the filtered area makes the data collection shorter, while not changing the results’ quality.
• the filter region typically only partially overlaps with the view region 232 of the sensor.
• the method of point pairing described before is fast and robust, plus it can be used in case of any of the mentioned sensor types. Furthermore, thanks to the method of point pairing, it is also possible to calibrate together more than two sensors at the same time, further increasing the stability of the method. This is also referred to as bundle adjustment.
• a possible formalization of it for our case is as follows. Let there be three sensors (A, B and C). Using regular optimization, only the error between the point pairs of e.g. the sensor pair A-B would be minimized. By using bundle adjustment, we can minimize the error between the point pairs of e.g. the sensor pairs A-B, A-C and B-C at the same time, which results in a better overall result.
• FIG. 1 Analogously for FIG. 1: On the figure, we can see the detected objects on the camera frame marked as small crosses, the detected objects on the radar frame marked as circles. Additionally, there can in practice be further points (not shown in FIG. 1), representing the detected objects on the lidar frame. The bundle adjustment minimizes the errors between the points pairs of all possible sensor combinations at the same time, thus decreasing the overall error efficiently.
• bundle adjustment the error metrics between any two sensor pairing may differ. Such decisions are up to the user to decide. Using bundle adjustment is not a must for implementing the method, regular optimization or any other method achieving the same goals could be proper for solving the problem. In a preferred embodiment bundle adjustment is used for its apparent advantages and its prevalence.
• the base of the validation is the calibration process described above. If the calibration of the sensors gives the same (or nearly the same) extrinsic parameters as the a priori calibration, the a priori calibration is deemed still valid.
• FIGs. 3A and 3B are a further detailed flow chart of a method for initializing 300, collecting data 310, calculating 320 results, and validating 340 the original parameters, also referred to as a priori parameters.
• the initialisation 300 comprises the following steps:
• a first step 302 original extrinsic parameters are initialised. This may involve determining an a priori calibration based on other sources, e.g. information about the location of the sensors. Then, in step 304 one or more sensors to be calibrated are selected and the sensor pairing is determined. In step 306, a basic filter area is calculated. This basic filter area maybe based on yaw-pitch and roll. Subsequently, in the 308 three proposed filter areas are determined and the following steps of collecting data and calculating results are performed separately for these three proposed filter areas. Determining the proposed filter areas may comprise determining one or more parameters of the proposed filter areas, but it is understood that further parameters maybe adjusted during data acquisition.
• the width and the center line of the filter area are determined during these steps.
• the center line is known on the sensor which should be calibrated, however, it is unknown on the reference sensor.
• the center line has to be transformed into the reference sensor’s coordinate system. Since this step (possibly) introduces errors, as the exact extrinsic parameters are not known, the following optimization can be performed: Multiple transformations with slightly different parameters are applied, e.g. based on different approaches for determining the transformation parameters or based on introduced a (e.g. random) variation to the parameters. Based on the multiple different transformations, multiple filter areas may be determined and the one that yields best results may be used at the end of the optimization.
• the data collection 310 may be performed when the vehicle (or preferably a fleet of vehicles) is travelling on roads for real-world data acquisition. It is, however, understood that the presented method may also be applied in a virtual scenario where a virtual vehicle is travelling through a virtual world (and acquiring virtual data from a simulated environment).
• the data collection 310 furthermore comprises the following steps:
• step 312a data for yaw-pitch calibration are collected.
• step 312b data for roll calibration are collected.
• filter areas are moved based on a current steering angle of the vehicle, on which the one or more sensors are mounted.
• the filter areas of one or more or all sensors may be moved based on the steering angle.
• additional information such as a speed of the vehicle may be used to move or otherwise adjust the filter area(s).
• steps 316a, 316b corresponding pairs are identified in the filtered data.
• the result calculation 320 comprises the following steps: First, for all three collected sets of data a cost function is evaluated and optimized in steps 322a, 322b and 322c. even after extensive optimization small errors remain and these remaining leftover errors in steps 324a, 324b, 324c. Subsequently, in step 326, the errors can be compared and the best result selected.
• the original extrinsic parameters 302 and the final result of calibrated extrinsic parameters 330 can then be provided to the validation 340.
• the validation 340 comprises the following steps :
• the paired points are reprojected in step 342, and an error is calculated in step 344.
• a validation metric is applied to evaluate whether the a priori parameters are correct. For example, the a priori parameters may be considered as correct if error calculated in step 344 is smaller than a predetermined threshold. Alternatively, the method may be performed for several a priori parameters and those that yield the smallest calculated error are determined as the correct parameters.

Landscapes

• Engineering & Computer Science (AREA)
• Radar, Positioning & Navigation (AREA)
• Remote Sensing (AREA)
• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• Computer Networks & Wireless Communication (AREA)
• Computer Vision & Pattern Recognition (AREA)
• Theoretical Computer Science (AREA)
• Electromagnetism (AREA)
• Length Measuring Devices By Optical Means (AREA)
• Image Analysis (AREA)

Priority Applications (4)

Applications Claiming Priority (2)

Publications (1)

Family

ID=82214435

Family Applications (1)

Country Status (6)

Families Citing this family (2)

Citations (1)

• 2022
  • 2022-06-22 EP EP22180544.3A patent/EP4296711A1/en active Pending
• 2023
  • 2023-05-11 US US18/875,560 patent/US20250363666A1/en active Pending
  • 2023-05-11 CN CN202380046754.8A patent/CN119365790A/zh active Pending
  • 2023-05-11 WO PCT/EP2023/062655 patent/WO2023247109A1/en not_active Ceased
  • 2023-05-11 KR KR1020257001028A patent/KR20250028355A/ko active Pending
  • 2023-05-11 JP JP2024573556A patent/JP2025519690A/ja active Pending

Patent Citations (1)

Non-Patent Citations (2)

Also Published As

Similar Documents

Legal Events