CN117492031A - On-line detection method and system for pose offset of vehicle-mounted ranging sensor - Google Patents

Abstract

The invention discloses an online detection method and system for the position and attitude deviation of a vehicle-mounted ranging sensor. The method is suitable for online detection of the relative position and attitude deviation of a parking space ranging sensor. The key points of the proposed method are: project the laser onto the road surface through the calibration laser, each vehicle-mounted ranging sensor measures the three-dimensional coordinates of the calibration laser point, and unify the three-dimensional coordinates of the calibration laser point measured by each ranging sensor into one coordinate system. Quantify the position and attitude offset of the vehicle-mounted ranging sensor based on the difference in unified geometric features formed by the calibration laser points between different vehicle-mounted ranging sensors, thereby determining whether the pose offset of the vehicle-mounted ranging sensor is too large. The method disclosed in the invention does not require expensive standard parts or any environmental characteristics, is simple to implement, and has strong anti-interference ability.

Description

An online detection method and system for position and attitude deviation of vehicle-mounted ranging sensors

Technical field

The invention belongs to the field of automobile assisted driving, and specifically relates to an online detection method and system for posture deviation of a vehicle-mounted ranging sensor.

Background technique

In recent years, with the rapid development of sensing and image processing technology, assisted driving technology has been significantly improved, and it has been increasingly used in the automotive field. Most of the high-end new energy vehicles currently manufactured are equipped with assisted driving functions, which reduce fatigue during long-distance driving and improve driving safety. It is foreseeable that in the future, assisted driving technology and even autonomous driving technology will be used more and more widely.

The realization of assisted driving depends on the effective work of vehicle-mounted ranging sensors. This not only requires that the vehicle-mounted ranging sensor can effectively sense the three-dimensional shape of the surrounding environment, but also requires that the posture of the vehicle-mounted ranging sensor cannot change significantly relative to the vehicle body. In fact, due to factors such as vibration caused by road bumps and vehicle body deformation, the vehicle-mounted ranging sensor will inevitably change its position relative to the vehicle body during use. If the posture changes too much, the three-dimensional perception of space by the vehicle-mounted ranging sensor will not be accurately fed back to the vehicle control system, which will lead to safety hazards in the assisted driving function.

In order to detect the possible position and orientation deviation of the vehicle-mounted ranging sensor, one way is to regularly calibrate the vehicle-mounted sensor through manually produced standard parts. A typical method is such as the invention patent "Automotive Calibration Equipment" applied by Shenzhen Daotong Technology Co., Ltd. ", the invention patent "Automotive Sensor Calibration Device" applied by Heduo Technology (Beijing) Co., Ltd. Although this type of method has high calibration accuracy, it requires driving the vehicle to a designated location for calibration, which will increase the cost, and it is difficult to detect possible pose deviations of the vehicle ranging sensor in a timely manner. Another way is to use the geometric characteristics of the scene around the road, such as lane lines, billboard size, etc., to achieve online calibration of the position and posture of the vehicle-mounted ranging sensor. A typical method is such as Huawei Technologies Co., Ltd.'s patent "External Parameter Calibration of Vehicle-mounted Sensors" methods and equipment". This method has obvious effects on urban roads, but it is difficult to apply on rural roads where there are no objects with obvious geometric features.

Contents of the invention

In order to solve the current problem that it is difficult to effectively detect the pose deviation of the vehicle-mounted ranging sensor online, the present invention provides an online detection method and system for the pose deviation of the vehicle-mounted ranging sensor.

An online detection method for posture offset of a vehicle-mounted ranging sensor according to the present invention includes the following steps:

Step 1: Fix the calibration laser at the selected position of the vehicle;

Step 2: When the calibration laser is turned on, the vehicle-mounted ranging sensor collects the calibration laser point signal;

Step 3: Process the calibration laser point signal collected by the vehicle-mounted ranging sensor to obtain the three-dimensional coordinates of the calibration laser point coverage area;

Step 4: Based on the three-dimensional coordinates of the calibration laser point coverage area, solve the relative pose offset characteristics of the vehicle ranging sensor.

Further, in step 1, the laser form emitted by the calibration laser has spatial discrimination. The orientation of the calibration laser in the vehicle is selected according to the spatial field of view of the detected displacement sensor, and is rigidly connected to the vehicle.

Further, in step 2, the vehicle-mounted ranging sensor includes a laser radar and a camera, and there is at least one landing point on the ground of the laser emitted by the calibration laser within the common field of view of any two ranging sensors.

Further, in step 3, the lidar realizes indirect measurement of the three-dimensional coordinates of the coverage area of the calibration laser point by detecting the interference caused by the laser emitted by the calibration laser; the indirect measurement process is: during the scanning measurement process, the lidar senses The sharp increase in light intensity caused by the calibration laser point or the abnormal flight time of the ranging laser can realize the identification and measurement of the three-dimensional coordinates of the area.

Further, in step 3, after the camera collects the calibration laser point signal, it extracts the geometric center of the calibration laser point signal, and then uses the internal and external parameters of the camera obtained by pre-calibration or the triangular relationship between the camera and the calibration laser to obtain the coverage area of the calibration laser point. three-dimensional coordinates.

Further, in step 4, the relative pose offset characteristics of the vehicle ranging sensor have the following typical categories:

With the help of camera and lidar calibration parameters, the measured three-dimensional coordinates of at least two sets of calibration laser point feature points in the camera or lidar are unified into the same coordinate system:

When only one calibration laser point feature point can be extracted, the distance between the three-dimensional coordinates of the calibration laser point feature points of different vehicle-mounted ranging sensors in the same coordinate system is used as the relative pose offset feature between the corresponding vehicle-mounted ranging sensors;

When two calibration laser point feature points can be extracted, the offset feature of one calibration laser point feature point, or the length difference between the two points measured by different vehicle-mounted ranging sensors, is used as the corresponding vehicle-mounted ranging sensor. Relative pose offset characteristics;

When three or more calibration laser point feature points can be extracted, the relative pose offset feature of one or two calibration laser point feature points, or the area or normal direction of a triangle defined by any three points, or any point The perimeter of the constituted side lines or the angle between the side lines is used as the relative pose offset characteristic between the corresponding vehicle-mounted ranging sensors.

Further, when three or more calibration laser point feature points can be extracted, the relative pose offset feature of one or two calibration laser point feature points is used as the first relative pose offset feature, and any three points are Statistical features that define the area or normal direction of the triangle, the perimeter of the sides formed by any points, or the angle between the sides are used as the second relative pose offset feature, and the first relative pose offset feature or the second relative pose offset feature is used. The statistical characteristics of the shift feature are used as the final relative pose shift feature between the corresponding vehicle ranging sensors.

Further, after step 4 is completed, based on the relative pose offset characteristics of the vehicle-mounted ranging sensor, it is determined whether the spatial pose offset of the corresponding vehicle-mounted ranging sensor exceeds the standard.

Furthermore, the criterion for judging whether the spatial pose deviation of the corresponding vehicle-mounted ranging sensor exceeds the standard is comprehensively set based on vehicle speed, measured point location, road conditions and other conditions.

An online detection system for vehicle ranging sensor pose and attitude deviation, including:

Vehicle-mounted ranging sensor, used to collect calibration laser point signals;

Processing module: Process the calibration laser point signal collected by the vehicle-mounted ranging sensor to obtain the three-dimensional coordinates of the calibration laser point coverage area;

Offset feature solving module: Based on the three-dimensional coordinates of the calibrated laser point coverage area, the relative pose offset feature of the vehicle ranging sensor is solved.

Compared with the prior art, the present invention at least has the following beneficial technical effects:

1. Due to the use of laser projection active marking, the present invention avoids the dependence of existing methods on objects with significant features on the road, is not affected by road conditions, and can theoretically adapt to the online detection of spatial pose deviations of vehicle-mounted ranging sensors in different scenarios. .

2. The relative pose offset characteristics of the vehicle-mounted ranging sensor obtained by the present invention are composed of the ranging deviations between different sensors, and can directly quantify the ranging error caused by the spatial pose offset of the vehicle-mounted ranging sensor, thus contributing to More accurately determine the health status of the vehicle ranging sensor, including but not limited to its posture deviation.

3. The cost of the calibration laser required for implementation in the present invention is very low, no expensive standard parts are required, and the calibration cost is reduced.

Description of drawings

Figure 1 shows a vehicle equipped with a main control system, camera, lidar, and calibration laser;

Figure 2 is a schematic diagram of a vehicle-mounted ranging sensor pose offset online detection system.

In the attached picture: 1. Calibration laser, 2. First camera, 3. Lidar, 4. Second camera, 5. Vehicle main control system, 6. Calibration laser point, 7. Vehicle.

Detailed ways

In order to make the purpose and technical solution of the present invention clearer and easier to understand. The present invention will be further described in detail below with reference to the accompanying drawings and examples. The specific embodiments described here are only used to explain the present invention and are not intended to limit the present invention.

In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "upper", "lower", "front", "rear", "left", "right", " The orientations or positional relationships indicated by "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. are based on the orientations or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and The simplified description is not intended to indicate or imply that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation, and therefore should not be construed as a limitation of the present invention. In addition, the terms “first” and “second” are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the quantity of indicated technical features. Therefore, features defined as "first" and "second" may explicitly or implicitly include one or more of these features. In the description of the present invention, unless otherwise stated, "plurality" means two or more. In the description of the present invention, it should be noted that, unless otherwise clearly stated and limited, the terms "installation", "connection" and "connection" should be understood in a broad sense. For example, it can be a fixed connection or a detachable connection. Connection, or integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium; it can be an internal connection between two components. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood on a case-by-case basis.

Example 1

Referring to Figure 1, an online detection method for pose offset of a vehicle-mounted ranging sensor includes the following steps:

Step 1: Fix the calibration laser at the selected position of the vehicle 7, and the vehicle main control system controls the opening and closing of the calibration laser;

Step 2: When the calibration laser is turned on, the vehicle main control system controls the work of the vehicle-mounted ranging sensor, and the vehicle-mounted ranging sensor collects the calibration laser point signal;

Step 3: Process the calibration laser point signal collected by the vehicle-mounted ranging sensor to obtain the three-dimensional coordinates of the area covered by the calibration laser point;

Step 4: Solve the relative pose offset characteristics of the vehicle ranging sensor based on the three-dimensional coordinates of the calibrated laser point coverage area;

Step 5: Based on the solved relative pose offset characteristics of the vehicle-mounted ranging sensor, determine whether the spatial pose offset of the corresponding vehicle-mounted ranging sensor exceeds the standard, and feed it back to the vehicle main control system.

Further, in step 1, the number of calibration lasers can be either one or multiple. The laser form emitted by the calibration laser must have spatial discrimination, such as point laser, line laser, cross laser, etc. The calibration laser is in the vehicle. The orientation needs to be selected based on the spatial field of view of the detected displacement sensor, and it must be rigidly connected to the vehicle.

Figure 1 shows a case of loading one calibration laser, and the laser emitted is a point laser. The vehicle main control system controls the opening and closing of the calibration laser 1.

Further, in step 2, the vehicle-mounted ranging sensor includes a lidar and two cameras, and there is at least one landing point on the ground (referred to as the calibration laser point) of the laser emitted by the calibration laser within the common field of view of any two ranging sensors. .

In Figure 1, the laser emitted by the calibration laser 1 falls on the ground to form a calibration laser point 6, and the calibration laser point 6 is located within the effective field of view of the first camera 2, the second camera 4, and the lidar 3 at the same time.

Further, in step 3, the lidar achieves indirect measurement of the three-dimensional coordinates of the area covered by the calibration laser point by detecting the interference caused by the laser emitted by the calibration laser.

The specific implementation of indirect measurement is as shown in Figure 1. During the scanning measurement process, the lidar 3 senses the sharp increase in light intensity caused by the calibration laser point 6 or the abnormality in the ranging laser flight time, and then realizes the measurement of the area. For the measurement of logos and three-dimensional coordinates, if necessary, the point cloud difference can be used to solve the three-dimensional coordinates of the area covered by the calibration laser point. Among them: the sharp increase in light intensity refers to: the reflected light of the calibration laser point is received by the lidar 3 and superimposed with the laser light emitted and reflected back by the lidar 3 itself, resulting in a significant increase in the actual detection light intensity of the lidar, which is manifested as the detection light intensity At least 50% more than adjacent areas. Among them, the abnormal ranging laser flight time refers to: the ranging laser flight time is less than the theoretical minimum value of the ranging laser flight time during lidar ranging.

After the first camera 2 and the second camera 4 collect the calibration laser point signal, the geometric center of the calibration laser point 6 is extracted through signal processing, and then the internal and external parameters of the first camera 2 and the second camera 4 obtained by pre-calibration can be calculated. The coordinates (x _c1 , y _c1 , z _c1 ) of the calibration laser point 6 in the coordinate system of the first camera 2 and the coordinates (x _c2 , y _c2 , z _c2 ) of the second camera 4 coordinate system are obtained. Of course, if the triangular relationship between the first camera 2 and the calibration laser 1 and the triangular relationship between the second camera 4 and the calibration laser 1 have been calibrated, then with the help of this triangular relationship, it can also be found that the calibration laser point 6 is located between the first camera 2 and the calibration laser 1. The coordinates (x _c1 , y _c1 , z _c1 ) and (x _c2 , y _c2 , z _c2 ) in the second camera 4 coordinate system. Since the lidar 3 itself has a three-dimensional sensing function, the measured coordinates of the calibration laser point 6 are set to (x _l1 , y _l1 , z _l1 ).

Further, in step 4, the relative pose offset characteristics of the vehicle ranging sensor have the following typical categories:

With the help of camera and lidar calibration parameters, the measured three-dimensional coordinates of the feature points of at least two sets of calibration laser points in the camera or lidar are unified into the same coordinate system; when only one calibration laser point feature point can be extracted, the same coordinate system can be The Euclidean distance between the three-dimensional coordinates of the calibrated laser point feature points of different vehicle-mounted ranging sensors is used as the relative pose offset feature between the corresponding vehicle-mounted ranging sensors;

Among them: in each measurement, the three-dimensional coordinates of all laser feature points measured by each ranging sensor constitute a set of feature points for calibrating laser points; because the laser points are actually distributed in blocks, their geometric feature centers need to be extracted, These geometric feature centers constitute laser point feature points.

When two calibration laser point feature points can be extracted, the offset feature of one calibration laser point feature point can be used, or the length difference between the two points measured by different vehicle-mounted ranging sensors can be used as the corresponding vehicle-mounted ranging Relative pose offset characteristics between sensors;

When three or more calibration laser point feature points can be extracted, the relative pose offset feature of one or two calibration laser point feature points can be used, or the area or method of the triangle defined by any three points can be used. direction, the perimeter of the edge or the angle between the edges formed by any points, etc., as the relative pose offset characteristics between the corresponding vehicle ranging sensors; when conditions permit, the statistics of the above relative pose offset features can be further used Features are used as relative pose offset features to improve the robustness of relative pose offset features.

Taking the case in Figure 1 where there is only one calibration laser point, an explanation of the relative pose offset representation is given.

In actual operations, the relative poses of the first camera 2, the second camera 4, and the lidar 3 must be known, that is, the transformation matrices of their coordinate systems are known. Assume that the coordinate system of the first camera 2 is converted to the coordinate system of LiDAR 3 as [R ₁₁ | T ₁₁ ], and the coordinate system of the second camera 4 is converted to the coordinate system of LiDAR 3 as [R ₂₁ | T ₂₁ ], then (x _c1 , y _c1 ,z _c1 ) and (x _c2 ,y _c2 ,z _c2 ) can be unified into the lidar 3 coordinate system. The conversion formula is as follows:

in, is the calibrated laser point coordinate measured by the first camera 2 after the unified coordinate system,/> is the calibrated laser point coordinate measured by the second camera 4 after the unified coordinate system, R ₁₁ is the spatial rotation matrix between the first camera 2 measurement coordinate system and the lidar 3 coordinate system, T ₁₁ is the first camera 2 measurement coordinate system and the spatial translation matrix between the second camera 4 measurement coordinate system and the lidar 3 coordinate system, R ₂₁ is the spatial rotation matrix between the second camera 4 measurement coordinate system and the lidar 3 coordinate system, T ₂₁ is the second camera 4 measurement coordinate system and the lidar 3 coordinate system Spatial rotation matrix between coordinate systems.

Furthermore, the Euclidean distance d c1_c2 between the first camera 2 and the second camera 4 in the same coordinate system, the Euclidean distance d _{c1_l1} between the first camera 2 and the lidar, and the Euclidean distance d _{c1_l1} between the first camera 2 and the lidar in the same coordinate system can be obtained The Euclidean distance d _{c2_l1} between 4 and the calibration laser point 6 measured by lidar 3:

Furthermore, in step 5, the criterion for determining whether the spatial pose deviation of the vehicle-mounted ranging sensor exceeds the standard needs to be comprehensively set based on conditions such as vehicle speed, measured point location, and road conditions.

In this embodiment, assuming that the pose offset threshold of the vehicle-mounted ranging sensor is d _max , then the conditions for determining whether the space pose offset of the vehicle-mounted ranging sensor exceeds the standard can be set as follows:

max(d _{c1_c2} ,d _{c1_l1} ,d _{c2_l1} )>d _max (4)

In the formula, max(*) represents the maximum value of *.

Once it is detected that the pose offset of the vehicle ranging sensor exceeds the set threshold, the status is fed back to the vehicle main control system and appropriate actions are taken.

Example 2

Referring to Figure 2, a vehicle-mounted ranging sensor pose offset online detection system includes:

Vehicle-mounted ranging sensor, used to collect calibration laser point signals;

Processing module: Process the calibration laser point signal collected by the vehicle-mounted ranging sensor to obtain the three-dimensional coordinates of the calibration laser point coverage area;

Offset feature solving module: Based on the three-dimensional coordinates of the calibrated laser point coverage area, the relative pose offset feature of the vehicle ranging sensor is solved.

The above contents are only for illustrating the technical ideas of the present invention and cannot be used to limit the protection scope of the present invention. Any changes made based on the technical ideas proposed by the present invention and based on the technical solutions shall fall within the scope of the claims of the present invention. within the scope of protection.

Claims (10)

1. An online detection method for pose offset of a vehicle-mounted ranging sensor, which is characterized by including the following steps: Step 1: Fix the calibration laser at the selected position of the vehicle; Step 2: When the calibration laser is turned on, the vehicle-mounted ranging sensor collects the calibration laser point signal; Step 3: Process the calibration laser point signal collected by the vehicle-mounted ranging sensor to obtain the three-dimensional coordinates of the calibration laser point coverage area; Step 4: Based on the three-dimensional coordinates of the calibration laser point coverage area, solve the relative pose offset characteristics of the vehicle ranging sensor. 2. The vehicle-mounted ranging sensor pose offset online detection method according to claim 1, characterized in that, in the step 1, the laser form emitted by the calibration laser has spatial discrimination, and the calibration laser in the vehicle The orientation is selected based on the spatial field of view of the detected displacement sensor and is rigidly connected to the vehicle. 3. The vehicle-mounted ranging sensor pose offset online detection method according to claim 1, characterized in that, in the step 2, the vehicle-mounted ranging sensor includes a laser radar and a camera, and any two ranging sensors There is at least one ground landing point of the laser emitted by the calibration laser within the public field of view. 4. The vehicle-mounted ranging sensor pose offset online detection method according to claim 1, characterized in that in step 3, the lidar realizes coverage of the calibration laser point by detecting the interference caused by the laser emitted by the calibration laser. Indirect measurement of the three-dimensional coordinates of the area; the indirect measurement process is: during the scanning measurement process, the lidar senses the sharp increase in light intensity caused by the calibration laser point or the abnormal flight time of the ranging laser, thereby realizing the identification of the area Measurement with three-dimensional coordinates. 5. The vehicle-mounted ranging sensor pose offset online detection method according to claim 1, characterized in that in step 3, after the camera collects the calibration laser point signal, the geometric center of the calibration laser point signal is extracted, and then the geometric center of the calibration laser point signal is extracted. The three-dimensional coordinates of the area covered by the calibration laser point are obtained with the help of the internal and external parameters of the camera obtained by pre-calibration or the triangular relationship between the camera and the calibration laser. 6. The vehicle-mounted ranging sensor pose offset online detection method according to claim 1, characterized in that, in step 4, the relative pose offset characteristics of the vehicle-mounted ranging sensor have the following typical categories: With the help of camera and lidar calibration parameters, the measured three-dimensional coordinates of at least two sets of calibration laser point feature points in the camera or lidar are unified into the same coordinate system: When only one calibration laser point feature point can be extracted, the distance between the three-dimensional coordinates of the calibration laser point feature points of different vehicle-mounted ranging sensors in the same coordinate system is used as the relative pose offset feature between the corresponding vehicle-mounted ranging sensors; When two calibration laser point feature points can be extracted, the offset feature of one calibration laser point feature point, or the length difference between the two points measured by different vehicle-mounted ranging sensors, is used as the corresponding vehicle-mounted ranging sensor. Relative pose offset characteristics; When three or more calibration laser point feature points can be extracted, the relative pose offset feature of one or two calibration laser point feature points, or the area or normal direction of a triangle defined by any three points, or any point The perimeter of the constituted side lines or the angle between the side lines is used as the relative pose offset characteristic between the corresponding vehicle-mounted ranging sensors. 7. The vehicle-mounted ranging sensor pose offset online detection method according to claim 6, characterized in that when three or more calibration laser point feature points can be extracted, one or two calibration laser point features are The relative posture offset characteristics of the points are used as the first relative posture offset characteristics, and the area or normal direction of the triangle defined by any three points, the perimeter of the sides formed by any points, or the statistical characteristics of the angle between the sides are used as the second relative posture shift characteristics. For pose offset features, the statistical features of the first relative pose offset feature or the second relative pose offset feature are used as the final relative pose offset features between the corresponding vehicle-mounted ranging sensors. 8. The vehicle-mounted ranging sensor pose offset online detection method according to claim 1, characterized in that after step 4 is completed, the corresponding vehicle-mounted ranging sensor spatial position is determined according to the relative pose offset characteristics of the vehicle-mounted ranging sensor. Whether the attitude deviation exceeds the standard. 9. The vehicle-mounted ranging sensor pose offset online detection method according to claim 8, characterized in that the criterion for judging whether the corresponding vehicle-mounted ranging sensor spatial pose offset exceeds the standard is based on vehicle speed, measured point position, Traffic conditions and other conditions are comprehensively set. 10. An online detection system for pose and attitude deviation of a vehicle-mounted ranging sensor, which is characterized by including: Vehicle-mounted ranging sensor, used to collect calibration laser point signals; Processing module: Process the calibration laser point signal collected by the vehicle-mounted ranging sensor to obtain the three-dimensional coordinates of the calibration laser point coverage area; Offset feature solving module: Based on the three-dimensional coordinates of the calibrated laser point coverage area, the relative pose offset feature of the vehicle ranging sensor is solved.