CN116758166A

CN116758166A - Parameter calibration method, device, equipment, vehicle and medium of vehicle sensor

Info

Publication number: CN116758166A
Application number: CN202310804105.1A
Authority: CN
Inventors: 张晓�
Original assignee: Chongqing Changan Automobile Co Ltd
Current assignee: Chongqing Changan Automobile Co Ltd
Priority date: 2023-06-30
Filing date: 2023-06-30
Publication date: 2023-09-15

Abstract

The invention relates to a parameter calibration method, device, equipment, vehicle and medium of a vehicle sensor, wherein the method comprises the following steps: presenting the relative position information of the first image data and the first point cloud data which are currently acquired in a vehicle-mounted engineering interface; the first image data or the first point cloud data are determined by utilizing the first calibration parameters of the sensor to be calibrated in a contraposition mode; under the condition that the relative position information does not meet the first preset condition, responding to a parameter correction instruction acting on the vehicle-mounted engineering interface, and performing parameter compensation on the first calibration parameter according to the space deviation vector of the sensor to be calibrated to obtain a second calibration parameter, thereby completing the parameter calibration of the sensor to be calibrated. The method and the device can improve timeliness of finding the abnormal calibration parameters, and further realize rapid parameter calibration.

Description

Parameter calibration method, device, equipment, vehicle and medium of vehicle sensor

Technical Field

The invention relates to the technical field of automobile driving, in particular to a parameter calibration method, device, equipment, a vehicle and a medium of a vehicle sensor.

Background

The automatic driving technology requires huge data in the research and development process, and the required data magnitude increases in geometric multiples with the increase of the automatic driving grade. In general, after the data acquisition of the acquisition vehicle is completed, the acquisition vehicle needs to be cleaned and completely marked with data to acquire high-quality images, and the development process of the automatic driving algorithm is to use the marked high-quality images to feed the algorithm in a large amount, so that a good training effect can be obtained.

However, because the acquisition vehicle is interfered by unstable factors such as driving shake, artificial interference, unstable fixation of the modified bracket and the like in the long-time running process, the sensor in the acquisition vehicle is often displaced, and related personnel cannot sense deviation of calibration parameters of the sensor, so that the calibration parameters cannot be modified in real time.

Disclosure of Invention

The invention aims to provide a parameter calibration method of a vehicle sensor, which aims to solve the problems that in the prior art, abnormal calibration parameters of the vehicle sensor cannot be found in time and the abnormal calibration parameters cannot be modified in time in the process of collecting data; the second aim is to provide a parameter calibration device of the vehicle sensor; a third object is to provide an electronic device; the fourth object is to provide a vehicle; it is a fifth object to provide a computer-readable storage medium.

In order to achieve the above purpose, the technical scheme adopted by the invention is as follows:

a method of parameter calibration of a vehicle sensor, the method comprising:

presenting the relative position information of the first image data and the first point cloud data which are currently acquired in a vehicle-mounted engineering interface; the first image data or the first point cloud data are determined by utilizing the first calibration parameters of the sensor to be calibrated in a contraposition mode;

under the condition that the relative position information does not meet the first preset condition, responding to a parameter correction instruction acting on the vehicle-mounted engineering interface, and performing parameter compensation on the first calibration parameter according to the space deviation vector of the sensor to be calibrated to obtain a second calibration parameter, thereby completing the parameter calibration of the sensor to be calibrated.

According to the technical means, the relative position information of the first image data and the first point cloud data is visually displayed in real time in the vehicle-mounted engineering interface, so that the time for finding the abnormal calibration parameters can be effectively shortened, and the timeliness for finding the abnormal calibration parameters and modifying the abnormal calibration parameters is improved.

Further, early warning prompt information is displayed in the vehicle-mounted engineering interface, and the early warning prompt information is used for indicating that the first calibration parameter of the sensor to be calibrated is in an abnormal state.

According to the technical means, under the condition that the relative position information does not meet the first preset condition, the first calibration parameters of related personnel can be timely reminded to be in an abnormal state through the early warning prompt information, and timeliness of finding the abnormal calibration parameters is guaranteed.

Further, the first preset condition is: the target pixel deviation of the first image data and the first point cloud data is smaller than or equal to a preset deviation threshold value; the method further comprises the steps of: gridding the first image data and the first point cloud data, and determining pixel deviation between the pixel point of each grid in the first image data and the pixel point of the corresponding grid in the first point cloud data; the maximum pixel deviation among the pixel deviations is set as the target pixel deviation.

According to the technical means, the maximum target pixel deviation in the pixel deviations corresponding to the grids in the first image data and the first point cloud data is selected for comparison, so that early warning prompt information can be accurately made, and timeliness and accuracy of finding out abnormal calibration parameters by related personnel are improved.

Further, with the preset shaft center of the vehicle as a calibration origin, aligning the reference sensor and the sensor to be calibrated based on the first spatial position information of the reference sensor and the second spatial position information of the sensor to be calibrated, so that the second image data and the second point cloud data meet the first preset condition, and the first calibration parameter is determined.

According to the technical means, the second image data and the second point cloud data acquired through the first calibration parameters can meet the first preset condition, and the accuracy of the second image data and the second point cloud data is guaranteed.

Further, according to the first spatial position information and the second spatial position information, determining a first spatial vector from a first position of a target point in the calibration tool to a second position of the reference sensor and a second spatial vector from the first position to a third position of the sensor to be calibrated; aligning the reference sensor and the sensor to be calibrated according to the first space vector and the second space vector so that a first offset vector from a first position point to a calibration origin point and a second offset vector from a second position point to the calibration origin point meet a second preset condition, and taking the second offset vector as a first calibration parameter; the second preset condition is as follows: the vector after the first space vector and the first offset vector are added is equal to the target space vector from the locus of the target point to the calibration origin, and the vector after the second space vector and the second offset vector are added is equal to the target space vector.

According to the technical means, the alignment is performed according to the reference sensor and the sensor to be calibrated, so that the first offset vector and the second offset vector meet the second preset condition, and the accuracy of the first calibration parameter is ensured.

Further, responding to a recalibration instruction acting on the vehicle-mounted engineering interface, and realigning the sensor to be calibrated to obtain a space deviation vector of the sensor to be calibrated.

According to the technical means, under the condition that the relative position information is determined not to meet the first preset condition, the spatial deviation vector corresponding to the calibration parameter in the abnormal state can be timely calculated by responding to the recalibration instruction acted on the vehicle-mounted engineering interface, so that parameter compensation can be timely conducted on the abnormal calibration parameter.

Further, in response to a recalibration instruction acting on the vehicle-mounted engineering interface, determining a third space vector from a first position of a target point in the calibration tool to a fourth position of a sensor to be calibrated by taking the center of a rear axle of the vehicle as a calibration origin; and subtracting the second space vector from the third space vector to obtain a space deviation vector.

According to the technical means, the accuracy of the space deviation vector is ensured.

Further, the space deviation vector is added with the first calibration parameters, so that parameter compensation of the first calibration parameters is completed, and the second calibration parameters are obtained.

According to the technical means, compensation of calibration parameters can be quickly, simply, conveniently and quickly realized by superposing linear deviation, and the complexity and time consumption of all the sensors for recalibration are avoided.

Further, the content displayed in the on-board engineering interface includes at least one of: the method comprises the steps of relative position information of first image data and first point cloud data, early warning prompt information, a spatial deviation vector, target pixel deviation, a preset deviation threshold value, first calibration parameters and second calibration parameters.

By displaying the parameters in the vehicle-mounted engineering interface, a user can be prompted in advance, data analysis after acquisition is completed is avoided, and the waste of data and manpower caused by parameter deviation is not found in time, so that the complexity and time consumption of all the sensors for recalibration are avoided.

A parameter calibration device for a vehicle sensor, the device comprising:

the display unit is used for presenting the relative position information of the first image data and the first point cloud data which are currently acquired in the vehicle-mounted engineering interface; the first image data or the first point cloud data are determined by utilizing the first calibration parameters of the sensor to be calibrated in a contraposition mode;

the determining unit is used for responding to a parameter correction instruction acting on the vehicle-mounted engineering interface under the condition that the relative position information does not meet a first preset condition, performing parameter compensation on the first calibration parameter according to the space deviation vector of the sensor to be calibrated to obtain a second calibration parameter, and completing parameter calibration of the sensor to be calibrated.

An electronic device comprising a vehicle body, the electronic device comprising: the system comprises a processor and a memory, wherein the memory is used for storing a computer program, and the processor is used for calling and running the computer program stored in the memory to execute the parameter calibration method of the vehicle sensor.

A vehicle comprises the parameter calibration device of the vehicle sensor.

A computer readable storage medium storing a computer program which when executed by at least one processor implements a method of calibrating parameters of a vehicle sensor as described above.

The invention has the beneficial effects that:

(1) According to the method, the relative position information of the first image data and the first point cloud data is visually displayed in real time in the vehicle-mounted engineering interface, so that the time for finding the abnormal calibration parameters can be effectively shortened, and the timeliness for finding the abnormal calibration parameters is improved;

(2) According to the invention, by responding to the parameter correction instruction acting on the vehicle-mounted engineering interface, the first calibration parameter can be subjected to parameter compensation according to the space deviation vector of the sensor to be calibrated, so that the deviation of the calibration parameter can be corrected by one key, and the rapid parameter calibration can be realized.

Drawings

FIG. 1 is a schematic flow chart of an alternative method for calibrating parameters of a vehicle sensor according to the present invention;

FIG. 2 is a schematic diagram of an alternative on-board engineering interface provided by the present invention;

FIG. 3 is a second flow chart of an alternative method for calibrating parameters of a vehicle sensor according to the present invention;

FIG. 4 is a schematic illustration of parameter calibration of an alternative vehicle sensor provided by the present invention;

FIG. 5 is a flowchart illustrating an alternative method for calibrating parameters of a vehicle sensor according to the present invention;

FIG. 6 is a flowchart illustrating an alternative method for calibrating parameters of a vehicle sensor according to the present invention;

FIG. 7 is a schematic diagram of the constitution of an alternative parameter calibration device for a vehicle sensor according to the present invention;

fig. 8 is a schematic diagram of an alternative composition structure of an electronic device according to the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present embodiment more apparent, a specific technical solution of the present invention will be described in further detail with reference to the accompanying drawings in the present embodiment. The following examples are illustrative of the invention and are not intended to limit the scope of the invention.

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terminology used herein is for the purpose of describing the embodiments only and is not intended to be limiting of the invention.

In the following description reference is made to "some embodiments," "this embodiment," and examples, etc., which describe a subset of all possible embodiments, but it is to be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments and can be combined with one another without conflict.

If a similar description of "first/second" appears in the application document, the following description is added, in which the terms "first/second/third" are merely distinguishing between similar objects and not representing a particular ordering of the objects, it being understood that the "first/second/third" may be interchanged with a particular order or precedence, where allowed, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein.

The term "and/or" in this embodiment is merely an association relationship describing an associated object, and indicates that three relationships may exist, for example, object a and/or object B may indicate: there are three cases where object a alone exists, object a and object B together, and object B alone exists.

Currently, the amount of data required in the development process of autopilot technology is enormous, and as the level of autopilot increases, the magnitude of the required data increases geometrically. The development of the automatic driving algorithm requires multiple incremental training to improve the performance of the vehicle-end algorithm. After the data acquisition of the acquisition vehicle is completed, the high-quality images are required to be acquired through cleaning and complete data labeling, and the development process of the automatic driving algorithm is to use the labeled high-quality images to feed the algorithm in a large amount so as to acquire a good training effect.

The acquisition vehicle acquires images of dynamic and static characteristics of a road and point cloud data on a large scale through a large number of public roads for perception algorithm training, the traditional single-view perception algorithm training is to continuously feed the images acquired through the single view into an algorithm model for incremental training after the images are completely marked by marking tools, such as a front-view algorithm model, a round-view algorithm model, a 3D laser algorithm model and the like, but the accuracy growth rate of the algorithm is basically kept level along with the increase of the data quantity, and the high cost investment is not improved well.

Therefore, the conventional single View algorithm training cannot meet the requirement of the perception precision, and further higher order perception training modes, such as BEV perception algorithm (Bird-Eye View algorithm) training, are proposed in the industry to break through the iteration bottleneck of the conventional single View algorithm. The method comprises the steps that 2D image information is formed by front view, peripheral view and rear view, 3D information is formed by overlapping 360-degree laser radar point clouds and is input to a 2D image, an image labeling result is provided with real 3D space information, and then a BEV perception algorithm model is trained by using labeled 2D image data carrying the real 3D information, so that the BEV algorithm has a more accurate recognition effect compared with a traditional single-view 2D algorithm model. However, the information required by the BEV algorithm comprises real 3D information and all 2D visual angle information, the information is acquired by relying on a manual post-loading camera and a laser radar, and the parameter calibration precision of the camera and the laser radar is higher. Therefore, in the installation process, the associated parameters between the image and the laser radar sensor need to be accurately calibrated, wherein the purpose of parameter calibration is to ensure the consistency and the accuracy of the data acquired by the camera and the sensor. However, after the initial calibration, the sensor is easily displaced due to the problems of driving shake, artificial interference, unstable fixation of a modified bracket and the like in the running process of the vehicle along with the time, so that the sensor is deviated from the initial calibration parameters, point cloud and image misalignment are formed, and the acquired data usability is poor.

The sensor position movement caused by the problems of shaking of driving environment, artificial interference, unstable fixation of a modified bracket and the like currently occurs, so that errors are generated between the sensor position movement and the previous calibration parameters, and the parameter calibration of the sensor is required to be immediately carried out again. However, from the occurrence of deviation of parameters to the occurrence of the problem that the deviation is found to cause the unavailability of collected data, at least links such as data collection, data copy uploading, data cleaning and the like are needed, and a period of time of nearly one week is needed to be spent for judging the unavailability by the data analysis software through the links in the process of normalized collection, so that the waste of one week of data is caused. Meanwhile, the calibration process is complex, more manpower and working hours are required to be consumed, and the acquisition task is seriously delayed when the acquisition vehicle stops running. The available data can not be obtained from the time of normal acquisition and finding of the problem to the time of stopping again for at least one week, which is unacceptable to data enterprises serving for algorithm development, and in addition, a method can be provided for the rapid calibration of low-cost sensors on mass production vehicles. Therefore, a visual rapid parameter calibration method is needed to shorten the calibration period, and meanwhile, the calibration parameter errors are found in time to reduce the data waste to the greatest extent, so that the method becomes an important problem to be solved by each automatic driving algorithm research and development enterprise.

In the prior art, a calibration method based on automatic driving sensor parameters is specifically implemented by establishing a sensor measurement equation through time delay between an actual measurement state of a sensor and a measurement unit, establishing an observation equation through a conversion relation between a sensor coordinate system and an inertial measurement unit coordinate system, and continuously iterating parameters according to the measurement equation and the observation equation by applying a Kalman filtering model to enable conditions to converge and calculate a calibration parameter value as a new calibration result. The method needs to monitor the state of the sensor in real time, requires complex iterative computation, has higher requirements on the controller, and has higher real-time monitoring cost and higher popularization difficulty; the method comprises the steps of obtaining calibration parameters and initial values thereof stored in an initial configuration file of an automatic driving vehicle of a terminal; calibrating parameters to be calibrated according to a preset calibration rule to obtain a real-time calibration value; and updating the initial configuration file by the parameter to be calibrated and the real-time calibration value to obtain a real-time configuration file, and sending the real-time configuration file to the automatic driving vehicle to update the parameter to be calibrated of the automatic driving vehicle, thereby completing the whole automatic driving parameter calibration process and solving the problems that parameters in the automobile controller cannot be modified in real time and the like. The calibration method relies on the conversion relation constructed by the actual mapping relation of the data to be calibrated, the process has higher requirements on the capability and environment of engineers, the method universality is not achieved, meanwhile, the data precision error caused by the calibration parameter deviation can be found after the data analysis, and the parameter error can not be calibrated quickly, so that the timely calibration can be achieved.

In summary, the automatic driving calibration is performed by performing reassignment in a deviation correction manner, and one way is to calculate new calibration parameters by recalibrating, that is, recalculating the position and angle parameters of the camera and the laser radar is completed by specific calibration software, and the image quality abnormality, such as ghost problem, is found in the acquisition process, so that the vehicle with problems can travel back to the calibration site, and the new calibration parameters are calculated by repeating the primary calibration operation in the whole process after the primary calibration environment is restored; the other type is to calculate and correct through error functions of actual values and observed values, and new parameters can be calculated after the functions are converged. In either way, the old parameters are replaced by the new parameters to obtain a relatively good calibration effect. The first mode is simple to operate, but takes a lot of time, the acquisition task cannot be executed in the time, the second mode consumes the controller computing resources through real-time computation, the vehicle frequency in the development stage is high, calibration is basically needed once a month, and a lot of energy is consumed through the mode, so that the cost is high.

Based on the above, the invention provides a parameter calibration method of a vehicle sensor, which is applied to electronic equipment with a display screen, wherein the electronic equipment is installed in a vehicle. The main idea of the method is that: firstly, the electronic equipment presents the relative position information of the first image data and the first point cloud data which are currently collected in a vehicle-mounted engineering interface; the first image data or the first point cloud data are determined by utilizing first calibration parameters of the sensor to be calibrated in a contraposition mode; and then, under the condition that the relative position information does not meet the first preset condition, the electronic equipment responds to a parameter correction instruction acting on the vehicle-mounted engineering interface, and performs parameter compensation on the first calibration parameter according to the space deviation vector of the sensor to be calibrated to obtain a second calibration parameter, so that parameter calibration of the sensor to be calibrated is completed. On the one hand, the relative position information of the first image data and the first point cloud data is visually displayed in real time in the vehicle-mounted engineering interface, so that the time for finding the abnormal calibration parameters can be effectively shortened, and the timeliness for finding the abnormal calibration parameters is improved; in still another aspect, by responding to a parameter correction instruction applied to the vehicle-mounted engineering interface, parameter compensation can be performed on the first calibration parameter according to the space deviation vector of the sensor to be calibrated, so that deviation of the calibration parameter is corrected by one key, and quick parameter calibration is realized.

The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings.

Fig. 1 is a flow chart of an alternative parameter calibration method of a vehicle sensor according to the present invention, as shown in fig. 1, including S101 to S102:

s101, presenting the relative position information of the first image data and the first point cloud data which are currently acquired in a vehicle-mounted engineering interface; the first image data or the first point cloud data are determined by utilizing first calibration parameters of the sensor to be calibrated.

In this embodiment, in the process of collecting data by the vehicle, the relative position information of the collected first image data and the first point cloud data is displayed in real time in the vehicle-mounted engineering interface of the electronic device.

In this embodiment, before data is collected by the vehicle, the electronic device displays an in-vehicle engineering interface in a display screen of the electronic device in response to a trigger operation acting on the engineering mode control.

In the embodiment, the vehicle-mounted engineering interface is similar to a developer interface of terminal equipment in function through being used for debugging calibration parameters of the vehicle sensor or collecting data, so that engineering personnel can conveniently debug the equipment.

In this embodiment, the triggering operation of the engineering mode control may be an interaction manner such as clicking, double clicking, touching, long pressing, preset gesture, remote control, voice control, and the like, which is not limited in this embodiment.

In this embodiment, the shape of the display area of the vehicle-mounted engineering interface may be rectangular, square, diamond, trapezoid, rounded rectangle or cut angle rectangle, and the shape, size and specific position of the display area of the vehicle-mounted engineering interface are not limited.

In this embodiment, the vehicle-mounted engineering interface may also be presented in the display screen in a floating window, a shorthand window, or the like, and the present embodiment does not limit the presentation manner of the vehicle-mounted engineering interface.

In this embodiment, the vehicle-mounted engineering interface may be presented in the form of a floating window, and by way of example, an icon of the vehicle-mounted engineering interface is displayed on a display interface of the vehicle-mounted terminal, the vehicle-mounted engineering interface may be expanded by clicking the icon, and by clicking a shrink control in the vehicle-mounted engineering interface, the interface may retract in the form of an icon or a small window, so that a user may conveniently view other application programs.

It will be appreciated that the electronic device presents the in-vehicle engineering interface in the display interface in the form of a small floating window in response to a triggering operation acting on the zoom-out window control. Therefore, the user can check other system application programs, such as a music system and the like, while collecting data, and further experience of the user is improved.

In this embodiment, the first image data is two-dimensional image data, and the first point cloud data is three-dimensional point cloud data.

In the present embodiment, the first image data and the first point cloud data are data acquired by the current vehicle.

In this embodiment, generally, the first image data is acquired by a camera in the vehicle, and the first point cloud data is acquired by a lidar in the vehicle.

In this embodiment, the first image data and the first point cloud data have a correspondence relationship, and specifically, the first image data and the first point cloud data are both for the same target object at the same viewing angle. For example, the first image data may be collected by a forward looking camera in the vehicle and the first point cloud data may be collected by a forward looking lidar in the vehicle.

In this embodiment, the number of the first image data may be one or more, and likewise, the number of the first point cloud data may be one or more. The first image data and the first point cloud data are present in pairs, that is, the number of the first image data and the first point cloud data present in the in-vehicle engineering interface is kept identical.

In the present embodiment, the relative positional relationship between the plurality of pairs of first image data and the first point cloud data may be displayed in the in-vehicle engineering interface. For example, in the first display area of the on-vehicle engineering interface, a relative positional relationship between the first image data collected by the front-view camera and the first point cloud data collected by the front-view lidar may be displayed; in the second display area of the vehicle-mounted engineering interface, the relative positional relationship between the first image data acquired by the rearview camera and the first point cloud data acquired by the rearview laser radar can be displayed.

In the present embodiment, the relative positional relationship of the first image data and the first point cloud data refers to the image coincidence degree between the first image data and the first point cloud data. The ideal states of the first image data and the first point cloud data are: the first image data and the image of the first point cloud data are completely coincident. The larger the coincidence ratio of the sensor to be calibrated and the laser radar, the larger the deviation between the camera and the laser radar is, and the first calibration parameter of the sensor to be calibrated is in an abnormal state.

In this embodiment, the first image data or the first point cloud data is determined by performing alignment using a first calibration parameter of the sensor to be calibrated. It should be noted that the sensor to be calibrated is one of a camera or a laser radar.

That is, for cameras and lidars, one of the sensors is used as a reference sensor and the other is used as a sensor to be calibrated. In this way, the first calibration parameter is determined by aligning the sensor to be calibrated with the reference sensor, and the first calibration parameter has the following functions: and the coincidence ratio of the acquired image data and the point cloud data meets the preset condition.

S102, under the condition that the relative position information does not meet the first preset condition, responding to a parameter correction instruction acting on a vehicle-mounted engineering interface, and performing parameter compensation on the first calibration parameter according to the space deviation vector of the sensor to be calibrated to obtain a second calibration parameter, thereby completing the parameter calibration of the sensor to be calibrated.

In this embodiment, in the process of collecting data for a long time by a vehicle, the vehicle sensor (camera or laser radar) is easily displaced due to the problems of driving shake, interference, unstable fixing of a modified bracket and the like in the running process of the vehicle over time, so that the first calibration parameter for the first calibration is biased, and the relative position information of the first point cloud data and the first image data collected subsequently does not meet the first preset condition (i.e. do not overlap), which results in poor availability of the collected data.

In this embodiment, the electronic device determines in real time whether the first image data and the first point cloud data that are currently collected meet the first preset condition, and when the electronic device determines that the first image data and the first point cloud data do not meet the first preset condition, it is stated that the first calibration parameter of the sensor to be calibrated may receive the problem of position displacement at this time, which results in inaccurate first calibration parameters that are determined before, and further results in poor availability of the collected first image data and the first point cloud data. Under the condition that the electronic equipment judges that the first image data and the first point cloud data meet the first preset condition, the first calibration parameters of the sensor to be calibrated are in a normal state, and the follow-up data can be continuously collected by utilizing the first calibration parameters determined before.

In this embodiment, the first preset condition may be that the image overlap ratio between the first image data and the first point cloud data is greater than or equal to a preset threshold, for example, the preset threshold is 95%. The first preset condition may also be that a target pixel deviation of the first image data and the first point cloud data is less than or equal to a preset deviation threshold, for example, the preset deviation threshold is 5 pixels. The first preset condition may be that a pixel deviation between the first image data and the first point cloud data is less than or equal to a preset pixel threshold. The setting of the first preset condition is not limited in this embodiment, and may specifically be selected according to an actual application scenario.

In this embodiment, when the electronic device determines that the relative positional relationship between the first image data and the first point cloud data does not meet the first preset condition, early warning prompt information is displayed in the vehicle-mounted engineering interface, where the early warning prompt information is used to indicate that the first calibration parameter of the sensor to be calibrated is in an abnormal state.

In this embodiment, the early warning prompt information may be displayed in the original display interface or in the early warning popup window, which is not limited in this embodiment.

In this embodiment, the early warning prompt information is always displayed in the vehicle-mounted engineering interface when the relative position information does not satisfy the first preset condition, until the relative position information satisfies the first preset condition.

For example, when the electronic device determines that the relative positional relationship between the first image data and the first point cloud data does not meet the first preset condition, an early warning popup window is displayed on the vehicle-mounted engineering interface. The early warning popup window can display early warning prompt information, and the early warning prompt information is used for prompting a user that a first calibration parameter of a sensor to be calibrated is in an abnormal state. For example, the early warning prompt information may be: "alert: the calibration parameters of the current sensor to be calibrated are in an abnormal state, and one-key correction operation can be selected to perform parameter compensation on the calibration parameters. "

It can be understood that in the process of data acquisition of the vehicle, the electronic device judges whether the acquired first image data and first point cloud data meet the first preset condition in real time, and when the first image data and the first point cloud data do not meet the first preset condition, early warning prompt information is timely displayed in the vehicle-mounted display interface to remind relevant personnel that the current calibration parameters of the sensor to be calibrated are in an abnormal state, so that the time for the abnormality of the calibration parameters can be greatly shortened, the relevant personnel can timely perform parameter compensation on the calibration parameters in the abnormal state, and further follow-up data acquisition is performed by using the calibration data after the parameter compensation.

It can be understood that abnormal states of calibration parameters of related personnel can be informed in advance by timely presenting early warning prompt information on a vehicle-mounted engineering interface, so that time consumption of data analysis after the vehicle collects the data can be effectively avoided, and waste of data and manpower caused by failure to timely find parameter deviation is effectively avoided.

In this embodiment, when the electronic device determines that the relative position information between the first image data and the first point cloud data does not meet the first preset condition, the electronic device responds to a parameter correction instruction acting on the vehicle-mounted engineering interface, and performs parameter compensation on the first calibration parameter according to the spatial deviation vector of the sensor to be calibrated to obtain the second calibration parameter, thereby completing parameter calibration of the sensor to be calibrated at this time.

In this embodiment, the parameter correction instruction may be an interaction manner such as a single click, a double click, a long press, a preset gesture, a remote control, a voice control, etc., which is not limited in this embodiment.

In this embodiment, when the early warning prompt information is displayed on the vehicle-mounted engineering interface, the pixel deviation, the target pixel deviation and the preset deviation threshold value corresponding to each grid in the first image data and the second point cloud data are presented on the vehicle-mounted engineering interface, so that relevant personnel can select whether to perform parameter compensation on the calibration parameters in the abnormal state according to the actual situation. For example, if the relevant person considers the current target pixel deviation to be within the allowable receiving range, data may be continuously collected, and if the relevant person considers the current target pixel deviation to be within the unacceptable range, parameter compensation may be performed on the first calibration parameter.

In some embodiments, in response to a parameter correction instruction applied to the vehicle engineering interface in S102, performing parameter compensation on the first calibration parameter according to the spatial deviation vector of the sensor to be calibrated, to obtain the implementation before the second calibration parameter, and further including: and responding to a recalibration instruction acting on the vehicle-mounted engineering interface, and realigning the sensor to be calibrated to obtain a space deviation vector of the sensor to be calibrated.

In this embodiment, when a related person can determine to perform parameter compensation on calibration parameters in an abnormal state according to an actual situation, the electronic device responds to a recalibration instruction acting on the vehicle-mounted engineering interface to recalibrate the sensor to be calibrated, so as to obtain a spatial deviation vector of the sensor to be calibrated.

In this embodiment, the recalibration instruction may be an interaction manner such as a single click, a double click, a long press, a preset gesture, a remote control, a voice control, etc., which is not limited in this embodiment.

In this embodiment, the spatial deviation vector of the sensor to be calibrated is obtained by aligning the sensor to be calibrated again. Wherein the spatial deviation vector characterizes a deviation vector between the last calibrated first calibration parameter and the second calibration parameter after realignment.

In some embodiments, performing parameter compensation on the first calibration parameter according to the spatial deviation vector of the sensor to be calibrated to obtain the implementation of the second calibration parameter may include: and adding the space deviation vector and the first calibration parameter, thereby completing the parameter compensation of the first calibration parameter and obtaining a second calibration parameter.

In this embodiment, the spatial deviation vector of the sensor to be calibrated may be denoted as Δp, and the second calibration parameter may be denoted as p ^* The first calibration parameter may be denoted as p, and the three satisfy the following linear relationship: p is p ^* ＝Δp+p。

It can be understood that the calibration parameters in the abnormal state are linearly compensated, and the compensation of the calibration parameters can be quickly, simply, conveniently and quickly realized by means of superposition of linear deviation by means of the parameter values of the initial calibration, so that the complexity and time consumption of all the sensors for recalibration are avoided.

In this embodiment, after the electronic device responds to the recalibration instruction acting on the vehicle-mounted engineering interface to calculate the space deviation vector, the deviation compensation is performed through the one-key reset operation of the vehicle-mounted display interface, the compensated result early warning prompt is eliminated, and the value of the new deviation vector (i.e. the second calibration parameter) p is stored and displayed on the vehicle-mounted engineering interface, so that the next linear compensation reference is facilitated.

The invention provides a parameter calibration method of a vehicle sensor, which is applied to electronic equipment with a display screen, wherein the electronic equipment is installed in a vehicle. The main idea of the method is that: firstly, the electronic equipment presents the relative position information of the first image data and the first point cloud data which are currently collected in a vehicle-mounted engineering interface; the first image data or the first point cloud data are determined by utilizing first calibration parameters of the sensor to be calibrated in a contraposition mode; and then, under the condition that the relative position information does not meet the first preset condition, the electronic equipment responds to a parameter correction instruction acting on the vehicle-mounted engineering interface, and performs parameter compensation on the first calibration parameter according to the space deviation vector of the sensor to be calibrated to obtain a second calibration parameter, so that parameter calibration of the sensor to be calibrated is completed. On the one hand, the relative position information of the first image data and the first point cloud data is visually displayed in real time in the vehicle-mounted engineering interface, so that the time for finding the abnormal calibration parameters can be effectively shortened, and the timeliness for finding the abnormal calibration parameters is improved; in still another aspect, by responding to a parameter correction instruction applied to the vehicle-mounted engineering interface, parameter compensation can be performed on the first calibration parameter according to the space deviation vector of the sensor to be calibrated, so that deviation of the calibration parameter is corrected by one key, and quick parameter calibration is realized.

In some embodiments, the content displayed in the on-board engineering interface includes at least one of: the method comprises the steps of relative position information of first image data and first point cloud data, early warning prompt information, a spatial deviation vector, target pixel deviation, a preset deviation threshold value, first calibration parameters and second calibration parameters.

Fig. 2 is a schematic diagram of an alternative vehicle engineering interface provided by the present invention, where, as shown in fig. 2, a display interface of the vehicle engineering interface at least includes: the display device comprises a first display area, a second display area and a third display area. The first display area is used for displaying the relative position relation between the first image data and the first point cloud data, the second display area is used for displaying early warning prompt information, and the third display area is used for displaying at least one of the following: the system comprises a space deviation vector, a target pixel deviation, a preset deviation threshold value, a first calibration parameter and a second calibration parameter.

It can be understood that by displaying the parameters in the vehicle engineering interface, the user can be prompted in advance, so that data analysis after acquisition is completed is avoided, and the waste of data and manpower caused by parameter deviation is not found in time, and the complexity and time consumption of all the sensors for recalibration are avoided.

In some embodiments, the first preset condition is: the target pixel deviation of the first image data and the first point cloud data is smaller than or equal to a preset deviation threshold value; the implementation of the parameter calibration method of the vehicle sensor further includes S201 to S202:

s201, gridding the first image data and the first point cloud data, and determining pixel deviation between the pixel point of each grid in the first image data and the pixel point of the corresponding grid in the first point cloud data.

In this embodiment, the electronic device performs gridding processing on the collected first image data and the first point cloud data at the same time, and determines a pixel deviation between a pixel point of each grid in the first image data and a pixel point of a corresponding grid in the first point cloud data. Illustratively, the pixel point of the 1 st grid in the first image data is different from the pixel point of the 1 st grid in the first point cloud data by 3 pixels, that is, the pixel deviation corresponding to the grid is 3 pixels.

In this embodiment, the process of performing the gridding processing on the first image data and the first point cloud data by the electronic device is to perform pixel-level gridding processing on a display area in which the first image data and the first point cloud data are displayed in the electronic device.

For example, the pixel deviation of the corresponding grid in the first image data and the first point cloud data may be expressed as: x is x ₁ px,x ₂ px,x ₃ px,……。

In this embodiment, the larger the pixel deviation of the grid is, the smaller the image overlap ratio between the first image data and the first point cloud data is; the smaller the pixel deviation of the grid, the larger the image overlap between the first image data and the first point cloud data.

S202, taking the maximum pixel deviation in the pixel deviations as a target pixel deviation.

In this embodiment, after the electronic device determines the pixel deviation between the pixel point of each grid in the first image data and the pixel point of the corresponding grid in the first point cloud data, the electronic device takes the maximum pixel deviation among the respective pixel deviations as the target pixel deviation.

In this embodiment, the electronic device may represent, as the target pixel deviation, the maximum pixel deviation among the pixel deviations corresponding to the respective grids as: max (x) ₁ px,x ₂ px,x ₃ px,……)。

In this embodiment, the preset deviation threshold is preset, and the preset deviation threshold may be represented as npx, for example, the preset deviation threshold is 5 pixels.

In the present embodiment, when the target pixel deviation is less than or equal to the preset deviation threshold (max (x ₁ px,x ₂ px,x ₃ px, … …) is less than or equal to npx), determining, at the electronic device, that the relative positional relationship of the first image data and the first point cloud data satisfies a first preset condition; when the target pixel deviation is greater than the preset deviation threshold (max (x ₁ px,x ₂ px,x ₃ px,……)>npx), determining, at the electronic device, that the relative positional relationship of the first image data and the first point cloud data does not satisfy the first preset condition.

Note that the electronic device may use an average value, a median value, or the like of the pixel deviations corresponding to the respective grids as the target pixel deviation, which is not limited in this embodiment.

It can be understood that the electronic device determines whether the relative position information of the first image data and the first point cloud data meets the first preset condition according to the target pixel deviation of the first image data and the first point cloud data which are acquired currently, and the electronic device can accurately make early warning prompt information by selecting the largest target pixel deviation in the pixel deviations corresponding to the grids in the first image data and the first point cloud data for comparison, so that timeliness and accuracy of finding abnormal calibration parameters by related personnel are improved.

In some embodiments, the method for calibrating parameters of the vehicle sensor further comprises:

and aligning the reference sensor and the sensor to be calibrated based on the first spatial position information of the reference sensor and the second spatial position information of the sensor to be calibrated by taking the preset shaft center of the vehicle as a calibration origin, so that the second image data and the second point cloud data meet a first preset condition, and further, a first calibration parameter is determined.

In this embodiment, the preset axle center of the vehicle is a preset center point, for example, the preset axle center may be a rear axle center of the vehicle, a front axle center of the vehicle, or the like, and the setting of the preset axle center is not limited in any way.

In this embodiment, the reference sensor is also referred to as a reference sensor, calibration parameters of the reference sensor do not need to be corrected (parameter compensation), and the sensor to be calibrated is a sensor whose calibration parameters need to be corrected.

In this embodiment, the first spatial position information is the spatial position coordinate of the reference sensor, and the second spatial position information is the spatial position coordinate of the sensor to be calibrated.

In this embodiment, the center of the rear axle of the vehicle is taken as the calibration origin (usually, the center of the rear axle) and the first calibration parameter p of the sensor to be calibrated is formed according to the spatial position and the angle (i.e. the spatial position vector) of the sensor to be calibrated, and the first calibration parameter can meet the superposition requirement of the point cloud and the image.

In the embodiment, a first calibration parameter is determined by aligning a reference sensor and a sensor to be calibrated; the second image data and the second point cloud data can meet a first preset condition through the first calibration parameters. The second image data and the second point cloud data are data acquired when the reference sensor and the sensor to be calibrated are aligned.

In some embodiments, as shown in fig. 3, with the preset axis center of the vehicle as the calibration origin, aligning the reference sensor and the sensor to be calibrated based on the first spatial position information of the reference sensor and the second spatial position information of the sensor to be calibrated, and determining the implementation of the first calibration parameter may include S301 to S303:

s301, determining a first space vector from a first position point of a target point in the calibration tool to a second position point of a reference sensor and a second space vector from the first position point to a third position point of the sensor to be calibrated according to the first space position information and the second space position information.

S302, aligning the reference sensor and the sensor to be calibrated according to the first space vector and the second space vector so that a first offset vector from a first position point to a calibration origin point and a second offset vector from a second position point to the calibration origin point meet a second preset condition, and taking the second offset vector as a first calibration parameter; the second preset condition is as follows: the vector after the first space vector and the first offset vector are added is equal to the target space vector from the locus of the target point to the calibration origin, and the vector after the second space vector and the second offset vector are added is equal to the target space vector.

For example, fig. 4 is a schematic diagram of parameter calibration of an alternative vehicle sensor provided by the present invention, taking a front-view camera and a front-view laser radar as examples, as shown in fig. 4, the calibration origin is the center of the rear axle of the vehicle (shown by "o" in fig. 4), point a is the front-view camera (reference sensor) of the vehicle, and point B is the front-view laser radar (sensor to be calibrated) of the vehicle. The electronic device determines a first space vector (indicated by "Ob" in fig. 4) from a first position of a target point (indicated by "Ob" in fig. 4) in the calibration tool to a second position of the reference sensor a based on the first space position information of the forward looking camera a and the second space position information of the forward looking lidar B ₁ "shown") and a second spatial vector (indicated by "a" in fig. 4) of the first locus to a third locus of the sensor B to be calibrated ₂ "show".

Further, electronsThe device is based on a first spatial vector a ₁ And a second space vector a ₂ Alignment of the reference sensor A and the sensor B to be calibrated is performed, and a first offset vector (shown as "B" in FIG. 4, from the first position to the calibration origin o is determined ₁ "shown) and a second offset vector (shown as" b "in FIG. 4) of the second locus to the calibration origin o ₂ "shown") such that the first offset vector b ₁ And a second offset vector b ₂ The second preset condition is satisfied.

In this embodiment, taking fig. 4 as an example, the second preset condition is: first space vector a ₁ And a first offset vector b ₁ The vector after addition is equal to the target space vector Obo from the point of the target point Ob to the calibration origin o, and the second space vector a ₂ And a second offset vector b ₂ The vector after addition is equal to the target space vector Obo, i.e.: a, a ₁ +b ₁ ＝Obo，a ₂ +b ₂ = Obo. In the case where the above condition is satisfied, the object Ob representing that the second image data and the second point cloud data are referenced to the spatial origin of coordinates are coincident.

In some embodiments, in response to a recalibration instruction acting on the vehicle engineering interface, realigning the sensor to be calibrated to obtain a spatial deviation vector of the sensor to be calibrated may include S401 to S402:

s401, responding to a recalibration instruction acting on a vehicle-mounted engineering interface, and determining a third space vector from a first position of a target point in a calibration tool to a fourth position of a sensor to be calibrated by taking the center of a rear axle of the vehicle as a calibration origin; wherein the vector after the addition of the third spatial vector and the second offset vector is not equal to the target spatial vector.

S402, subtracting the second space vector from the third space vector to obtain a space deviation vector.

In this embodiment, the sensor is easily displaced due to the problems of driving shake, interference, unstable fixation of the modified bracket, etc. during the running process of the vehicle over time, so that the sensor is deviated from the primary calibration parameters, and thus point cloud and image misalignment are formed, and further acquisition is causedThe data availability of the set is poor. Exemplary, as shown in FIG. 4, it is assumed that forward looking lidar B is laterally offset from original site B to site B due to lateral displacement bias of road test shock ^′ In this way, the spatial positions of the 2D image and the 3D point cloud of the same view angle of the vehicle are not overlapped, as shown in fig. 4, the image data collected by the front-view camera is the target point Ob, and the point cloud data collected by the front-view laser radar is the target point Ob ^′ In this way, the image Ob displayed on the in-vehicle engineering interface may appear ghost Ob ^′ 。

In the present embodiment, as shown in fig. 4, a third space vector a ^′ And a second offset vector b ₂ The vector after addition is not equal to the target space vector Obo, i.e. a ^′ +b ₂ ≠Ob ^′ o，Ob ^′ o+. Obo. At this time, ghost Ob of Ob appears on the on-vehicle engineering interface display screen ^′ 。

In this embodiment, in the recalibration process, the vehicle needs to be driven to the same calibration site as the previous calibration site, and the same calibration tool as the previous parameter calibration is adopted to realize the realignment. Taking fig. 4 as an example, taking the center of the rear axle of the vehicle as the calibration origin o, the first position of the target point Ob in the calibration tool is determined to the fourth position B of the sensor to be calibrated ^′ Is shown in fig. 4 (with "a" in fig. 4 ^′ "show".

Further, the electronic device is based on the third space vector a ^′ And a second space vector a ₂ A spatial deviation vector Δp is determined.

In this embodiment, the electronic device uses the second space vector a ₂ And a third space vector a ^′ The subtraction is performed to obtain a spatial deviation vector Δp, and the above process can be expressed as: Δp=a ₂ -a ^′ 。

It can be understood that in the process of acquiring data through the vehicle, under the condition that the electronic device determines that the relative position information of the first image data and the first point cloud data acquired currently does not meet the first preset condition, the sensor to be calibrated is aligned again in response to a recalibration instruction acting on the vehicle-mounted engineering interface, so as to obtain a space deviation vector of the sensor to be calibrated, and further, parameter compensation is performed on the first calibration parameter in an abnormal state according to the space deviation vector, so that the time for finding out that the calibration parameter is abnormal can be greatly shortened, the first calibration parameter is corrected quickly and accurately, and quick parameter calibration is realized.

The following explains the parameter calibration method of the vehicle sensor provided in this embodiment in some specific embodiments:

(one), example 1

In this embodiment, a method for calibrating a fast parameter based on visual linear deviation compensation (i.e. the method for calibrating a parameter of a vehicle sensor) is provided, and the core idea of the method is as follows: the position relation (corresponding to relative position information) between point cloud (corresponding to point cloud data) and image (corresponding to image data) is displayed through a vehicle-mounted engineering mode interface (corresponding to a vehicle-mounted engineering interface), deviation values (corresponding to space deviation vectors) are directly displayed through a digital display mode, an early warning prompt is triggered after the vehicle-mounted engineering mode interface enters an early warning range, early warning is executed through the display screen engineering interface, parameter compensation operation is timely carried out after early warning, and data can be continuously collected after early warning elimination is completed.

Specifically, the method mainly comprises performing early warning prompt within an early warning range (pixel error npx is provided by algorithm, namely preset deviation threshold value) after error (corresponding to space deviation vector) is generated, calculating position and angle deviation deltap (corresponding to space deviation vector) of calibration parameters of a vehicle through a calibration plate (corresponding to calibration tool) in advance according to the early warning prompt when idle, further performing linear compensation on the basis accumulated deviation deltap of the last calibration value (corresponding to first calibration parameter), and finally obtaining new calibration parameter p ^* (corresponding to the second calibration parameter).

In this embodiment, as shown in fig. 5, the parameter calibration method of the vehicle sensor mainly includes the following steps: the steps mentioned above are explained below, namely, primary calibration, display of a vehicle display screen, deviation early warning, linear deviation compensation, deviation resetting and storage:

s501, primary calibration.

In this embodiment, the center of the rear axle of the vehicle is used as the calibration origin, and the primary calibration parameter p (corresponding to the first calibration parameter) is formed according to the spatial position and the angle of the sensor to be calibrated, where the primary calibration parameter p can meet the superposition requirement of the point cloud data and the image data.

S502, displaying on a display screen of the car machine.

In this embodiment, the image and the alignment point cloud signal (the point cloud data after alignment by using the calibration parameter) are visually displayed on the vehicle working interface screen, and after initial calibration, the point cloud data and the target object shown by the image data are substantially coincident.

S503, deviation early warning.

In this embodiment, pixel-level gridding is implemented on the display screen of the vehicle, images of all objects in front of the vehicle and the opposite point cloud pixel deviation xpx are calculated based on the grids, the calculation rule is max { x1px, x2px, x3px, … }, the maximum pixel deviation is taken as the early warning deviation, and when max { x1px, x2px, x3px, … } > npx, the deviation early warning prompt is executed. All target deviation results are displayed at the right lower part of the display screen (pixel deviation corresponding to each network is displayed), so that research, development, viewing and problem backtracking are facilitated. The target deviation condition can be obtained through checking, so that whether the quick parameter calibration is executed is judged, if the deviation is in a controllable range, the data can be continuously collected without being influenced, and otherwise, the quick parameter calibration is needed to be executed for the vehicle.

S504, linear deviation compensation.

In this embodiment, after the early warning and prompting, if the rapid parameter calibration is required to be arranged, a safety officer opens the vehicle to a calibration site, and positions the sensor sites (namely, the position information and the angle) rapidly through a calibration plate, so as to calculate the position and angle deviation deltap, and a vector calculation rule: Δp is equal to vector a minus vector a ', where vector a is the spatial vector of the original sensor (i.e., the vector to be calibrated) and vector a' is the shifted sensor spatial vector.

S505, deviation resetting and storing.

In the present embodiment, after calculating the linear deviation, the deviation Δp (i.e., the spatial deviation vector) is displayed on a displayOn the screen, the user manually triggers a reset switch (after calculation is completed, a popup window with offset compensation calculation is displayed, or an original interface is displayed, and a switch button is displayed at the same time), the calibration parameters are updated by clicking, and when data is acquired next time, the updated calibration parameters are used for realizing parameter reset, and the reset logic is as follows: p is p ^* The new parameter after reset is stored in the system as the basis for the next calibration.

Through the process, the image of the target object and the point cloud deviation (corresponding to relative position information) can be directly displayed on the vehicle working interface (corresponding to the vehicle engineering interface), a vehicle user is prompted to calibrate parameters according to the deviation degree in a deviation early warning mode (namely, one-key deviation correction is performed), and meanwhile, the calibration process is that linear deviation calculation is performed in a simple sensor alignment mode. Compared with the traditional mode, the embodiment of the invention has the following beneficial effects:

1) The embodiment can quickly display and early warn deviation, prompt a user to timely perform parameter compensation on the calibration parameters in an abnormal state in advance, and avoid the waste of data and manpower caused by the fact that the parameter deviation is not found in time after the data acquisition is completed;

2) The linear compensation mode related to the embodiment is simple, convenient and quick, compensation of calibration parameters can be rapidly realized by means of superposition of linear deviation by means of the parameter values of primary calibration, and complexity and time consumption of all the sensors in recalibration are avoided.

(II) example 2

In this embodiment, taking forward looking image and forward laser radar point cloud calibration as examples, illustrating a linear compensation process after the laser radar has a lateral displacement, fig. 6 is a flowchart illustrating an alternative parameter calibration method for a vehicle sensor according to the present invention, as shown in fig. 6, where the method includes S601 to S603:

s601, obtaining calibration parameters through primary calibration, and displaying based on the deviation after gridding treatment.

The calibration parameters obtained by the primary calibration correspond to the first calibration parameters in the previous description.

In the present embodiment, the deviation display refers to the relative position information of the first image data and the first point cloud data in the foregoing.

In the present embodiment, S601 includes S6011 to S6013:

s6011, performing primary calibration, wherein the center of a rear axle of the vehicle is used as a calibration origin, and a primary calibration parameter p is formed according to the space position and the angle of a sensor to be calibrated.

The first calibration parameter p corresponds to the first calibration parameter described above.

In this embodiment, after the front-view camera and the laser radar are post-assembled and modified and then are subjected to detailed calibration on a calibration site, initial calibration parameters are formed according to the spatial position and angle of the sensor to be calibrated based on the center of the rear axle of the vehicle as the spatial coordinate origin.

S6012, displacement and vibration cause deviation, visual images of the same visual angle of the vehicle and the machine are not overlapped with the point cloud, and the vehicle-mounted engineering interface displays original calibration parameters, the point cloud and the image projection.

The original calibration parameters are equivalent to the first calibration parameters, and the point cloud and the image projection are equivalent to the first image data and the second point cloud data.

In the embodiment, it is assumed that the laser radar generates lateral displacement deviation due to road test vibration, so that the 2D image and the 3D point cloud space positions of the same visual angle of the vehicle and the machine are not overlapped, and the result is displayed on the engineering interface of the vehicle through a front-end image rendering technology, and meanwhile, the original calibration parameter p is displayed.

S6013, pixel meshing processing, wherein when the maximum value of the misaligned pixels of the point cloud and the image exceeds an early warning threshold value, an early warning prompt is displayed, and meanwhile, the numerical value of meshing calculation deviation is displayed on a vehicle-mounted display screen, so that the problem is convenient to view.

The early warning threshold value is equivalent to the preset deviation threshold value.

In this embodiment, pixel gridding processing is performed on the rendered image, gridding calculation deviation xpx values are displayed at the right lower side of a display screen of a vehicle, so that research, development and inspection and problem backtracking are facilitated, early warning prompt is performed when the maximum value max { x1px, x2px, x3px, … } of the misaligned pixels of the point cloud and the image exceeds the early warning npx, and according to the possibility that prompt engineers comprehensively judge parameters need recalibration, if the parameters do not influence, data are continuously collected, calibration is performed at a selected period, otherwise, rapid parameter calibration is performed through calibration plates (equivalent to calibration tools) in alignment immediately if acquisition conditions are not met.

S602, calculating parameter deviation, and rapidly compensating calibration parameters based on the deviation value.

Wherein the deviation value corresponds to the spatial deviation vector described above.

In the present embodiment, S602 includes S6021 to S6022:

s6021, calculating the position and angle deviation deltap of the coordinate space according to the position relation of the calibration plates.

S6022, displaying the calculated deviation on a display screen, performing deviation compensation by one key, and performing p+Deltap on the primary calibration parameters by the compensation logic.

In this embodiment, the vehicle to be calibrated is driven into the calibration site, and the calibration board checkerboard (i.e., calibration board) is used for positioning the vehicle sensor. According to the example of FIG. 4, o is the spatial origin of coordinates, A (corresponding to the reference sensor) is the forward looking camera site, and the spatial vector to the target Ob (corresponding to the target point in the calibration tool) is denoted as a ₁ (corresponding to the first space vector), B (corresponding to the sensor to be calibrated) is the forward laser radar site, and the space vector from the target Ob is denoted as a ₂ (corresponding to the second space vector), the corresponding offset vector after the primary calibration is b ₁ (corresponding to the first offset vector) and b ₂ (corresponding to the second offset vector), at this time a ₁ +b ₁ ＝Obo，a ₂ +b ₂ = Obo, meaning that the image coincides with the object Ob of the point cloud referenced to the origin of the spatial coordinates. And (5) calibrating again: when the position of the laser radar B is laterally shifted to B ^′ Determining a space vector from a new position to Ob of the laser radar as a by using a calibration plate checkerboard ^′ (corresponding to the third space vector), then an offset vector b based on the initial calibration ₂ At this time a ^′ +b ₂ ≠Ob ^′ o，Ob ^′ o is not equal to Obo, in-vehicle engineering Ghost Ob of Ob appears on interface display screen ^′ 。

After realignment, the deviation is calculated through the vector, and the reasoning process is as follows through the offset vector of the primary calibration and the deviation vector deltap calculated after realignment:

1) Primary calibration result p=b ₂ Offset vector Δp=a after alignment ₂ -a ^′ ；

2) New offset vector p ^* ＝Δp+p＝b ₂ +(a ₂ -a ^′ )。

S603, parameter deviation resetting and parameter storage.

In this embodiment, after execution is completed, the maximum deviation does not exceed the npx early warning range, the early warning prompt is cleared, and new calibration parameters p are stored.

In this embodiment, after the deviation vector is calculated, the deviation compensation is performed through a one-key reset operation of the display screen of the vehicle machine, and the compensated result early warning prompt is eliminated, and the new p-type value of the deviation vector is stored and displayed on the display screen of the working interface of the vehicle machine, so that the next linear compensation reference is facilitated.

The preferred embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to the specific details of the above embodiments, and various simple modifications can be made to the technical solution of the present invention within the scope of the technical concept of the present invention, and all the simple modifications belong to the protection scope of the present invention. For example, the specific features described in the above embodiments may be combined in any suitable manner, and in order to avoid unnecessary repetition, various possible combinations are not described further. As another example, any combination of the various embodiments of the present invention may be made without departing from the spirit of the present invention, which should also be regarded as the disclosure of the present invention. For example, on the premise of no conflict, the embodiments described in the present invention and/or technical features in the embodiments may be combined with any other embodiments in the prior art, and the technical solutions obtained after combination should also fall into the protection scope of the present invention.

It should be understood that, in the various method embodiments of the present invention, the sequence number of each process described above does not mean that the execution sequence of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of this embodiment.

Based on the same inventive concept as the previous embodiment, fig. 7 is a schematic diagram of the composition structure of an alternative parameter calibration device for a vehicle sensor according to the present invention, as shown in fig. 7, the parameter calibration device 10 for a vehicle sensor includes a display unit 11 and a determining unit 11; wherein,

the display unit 11 is configured to present, in a vehicle-mounted engineering interface, relative position information of first image data and first point cloud data that are currently collected; the first image data or the first point cloud data are determined by utilizing the first calibration parameters of the sensor to be calibrated in a contraposition mode;

the determining unit 11 is configured to perform parameter compensation on the first calibration parameter according to the spatial deviation vector of the to-be-calibrated sensor in response to a parameter correction instruction applied to the vehicle-mounted engineering interface when the relative position information does not meet a first preset condition, so as to obtain a second calibration parameter, and complete parameter calibration on the to-be-calibrated sensor at this time.

In some embodiments, the display unit 11 is further configured to display early warning prompt information in the vehicle engineering interface, where the early warning prompt information is used to indicate that the first calibration parameter of the sensor to be calibrated is in an abnormal state.

In some embodiments, the first preset condition is: the target pixel deviation of the first image data and the first point cloud data is smaller than or equal to a preset deviation threshold value; the determining unit 11 is further configured to perform gridding processing on the first image data and the first point cloud data, and determine a pixel deviation between a pixel point of each grid in the first image data and a pixel point of a corresponding grid in the first point cloud data; and taking the maximum pixel deviation in each pixel deviation as the target pixel deviation.

In some embodiments, the determining unit 11 is further configured to determine the first calibration parameter by using a preset axle center of the vehicle as a calibration origin, and aligning the reference sensor and the sensor to be calibrated based on the first spatial position information of the reference sensor and the second spatial position information of the sensor to be calibrated, so that the second image data and the second point cloud data satisfy the first preset condition.

In some embodiments, the determining unit 11 is further configured to determine, according to the first spatial position information and the second spatial position information, a first spatial vector from a first location of the target point in the calibration tool to a second location of the reference sensor, and a second spatial vector from the first location to a third location of the sensor to be calibrated; aligning the reference sensor and the sensor to be calibrated according to the first space vector and the second space vector so that a first offset vector from the first position to the calibration origin and a second offset vector from the second position to the calibration origin meet a second preset condition, and taking the second offset vector as the first calibration parameter; wherein the second preset condition is: the vector after the first space vector and the first offset vector are added is equal to a target space vector from the locus of the target point to the calibration origin, and the vector after the second space vector and the second offset vector are added is equal to the target space vector.

In some embodiments, the second preset condition is: the vector after the first space vector and the first offset vector are added is equal to a target space vector from the locus of the target point to the calibration origin, and the vector after the second space vector and the second offset vector are added is equal to the target space vector.

In some embodiments, the determining unit 11 is further configured to, in response to a recalibration instruction applied to the vehicle engineering interface, realign the sensor to be calibrated to obtain a spatial deviation vector of the sensor to be calibrated.

In some embodiments, the determining unit 11 is further configured to determine, in response to a recalibration instruction acting on the on-vehicle engineering interface, a third space vector from a first location of a target point in a calibration tool to a fourth location of the sensor to be calibrated, with a rear axle center of the vehicle as a calibration origin; and subtracting the second space vector from the third space vector to obtain the space deviation vector.

In some embodiments, the determining unit 11 is further configured to add the spatial deviation vector to the first calibration parameter, thereby completing parameter compensation for the first calibration parameter, and obtaining the second calibration parameter.

In some embodiments, the content displayed in the on-board engineering interface includes at least one of: the relative position information, the early warning prompt information, the spatial deviation vector, the target pixel deviation, the preset deviation threshold value, the first calibration parameter and the second calibration parameter of the first image data and the first point cloud data.

It will be appreciated by those skilled in the art that the above description of the vehicle-mounted terminal of the present embodiment can be understood with reference to the description of the data processing method of the present embodiment.

Fig. 8 is a schematic diagram of the composition structure of an alternative electronic device provided by the present invention. As shown in fig. 8, the electronic device 20 includes a processor 21 and a memory 22, the memory 22 may store a computer program, and the processor 21 may call and run the computer program from the memory 22 to implement the method in the present embodiment.

The memory 22 may be a separate device independent of the processor 21 or may be integrated in the processor 21.

In some embodiments, as shown in fig. 8, the electronic device 20 may further include a transceiver 23, and the processor 21 may control the transceiver 23 to communicate with other devices, and in particular, may send information or data to other devices, or receive information or data sent by other devices.

The transceiver 23 may include a transmitter and a receiver, among others. The transceiver 23 may further include antennas, the number of which may be one or more.

In some embodiments, a vehicle is provided that includes the parameter calibration device 10 of the vehicle sensor described above.

It will be appreciated that the processor of the present embodiment may be an integrated circuit chip having information processing capabilities. In implementation, the steps of the above method embodiments may be implemented by integrated logic circuits of hardware in a processor or instructions in software form. The processor may be a general purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), an off-the-shelf programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components. The methods, steps and logic blocks disclosed in the present embodiment may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the present embodiment may be embodied directly in hardware, in a decoded processor, or in a combination of hardware and software modules in a decoded processor. The software modules may be located in a random access memory, flash memory, read only memory, programmable read only memory, or electrically erasable programmable memory, registers, etc. as well known in the art. The storage medium is located in a memory, and the processor reads the information in the memory and, in combination with its hardware, performs the steps of the above method.

It will also be appreciated that the memory in this embodiment can be either volatile memory or nonvolatile memory, or can include both volatile and nonvolatile memory. The nonvolatile Memory may be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically Erasable EPROM (EEPROM), or a flash Memory. The volatile memory may be random access memory (Random Access Memory, RAM) which acts as an external cache. By way of example, and not limitation, many forms of RAM are available, such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (Double Data Rate SDRAM), enhanced SDRAM (ESDRAM), synchronous DRAM (SLDRAM), and Direct RAM (DR RAM). It should be noted that the memory described herein is intended to comprise, without being limited to, these and any other suitable types of memory.

It is also understood that the above memory is illustrative but not restrictive, and for example, the memory in the present embodiment may be Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (Double Data Rate SDRAM), enhanced SDRAM (ESDRAM), synchronous DRAM (SLDRAM), direct RAM (DR RAM), and the like. That is, the memory in this embodiment is intended to comprise, without being limited to, these and any other suitable types of memory.

The present embodiment also provides a computer-readable storage medium storing a computer program.

In some embodiments, the computer readable storage medium may be applied to the server in the present embodiment, and when the computer program is executed by at least one processor, the corresponding flow implemented by the server in each method of the present embodiment is implemented, which is not described herein for brevity.

In some embodiments, the computer readable storage medium may be applied to the graphics processor in this embodiment, and when the computer program is executed by at least one processor, the corresponding flow implemented by the graphics processor in each method of this embodiment is implemented, which is not described herein for brevity.

The present embodiment also provides a computer program product comprising computer program instructions.

In some embodiments, the computer program product may be applied to the server in this embodiment, and the computer program instructions cause the computer to execute the corresponding flow implemented by the server in each method of this embodiment, which is not described herein for brevity.

In some embodiments, the computer program product may be applied to the graphics processor in this embodiment, and the computer program instructions cause the computer to execute the corresponding processes implemented by the graphics processor in the methods of this embodiment, which are not described herein for brevity.

The present embodiment also provides a computer program.

In some embodiments, the computer program may be applied to the server in this embodiment, and when the computer program runs on the computer, the computer is caused to execute the corresponding flow implemented by the server in each method in this embodiment, which is not described herein for brevity.

In some embodiments, the computer program may be applied to the graphics processor in this embodiment, and when the computer program runs on the computer, the computer program makes the computer execute the corresponding flow implemented by the graphics processor in each method of this embodiment, which is not described herein for brevity.

Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.

It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the apparatus and units described above may refer to corresponding procedures in the foregoing method embodiments, which are not described herein again.

In the several embodiments provided by the present invention, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of the units is merely a logical function division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.

The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit.

The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.

It should be noted that, in the present invention, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

The foregoing embodiment numbers are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments.

The methods disclosed in the method embodiments provided by the invention can be arbitrarily combined under the condition of no conflict to obtain a new method embodiment.

The features disclosed in the several product embodiments provided by the invention can be combined arbitrarily under the condition of no conflict to obtain new product embodiments.

The features disclosed in the embodiments of the method or the apparatus provided by the invention can be arbitrarily combined without conflict to obtain new embodiments of the method or the apparatus.

The foregoing is merely illustrative of the present invention, and the present invention is not limited thereto, and any person skilled in the art will readily recognize that variations or substitutions are within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A method for calibrating parameters of a vehicle sensor, the method comprising:

and under the condition that the relative position information does not meet a first preset condition, responding to a parameter correction instruction acting on the vehicle-mounted engineering interface, and carrying out parameter compensation on the first calibration parameter according to the space deviation vector of the sensor to be calibrated to obtain a second calibration parameter, thereby completing the parameter calibration of the sensor to be calibrated.

2. The method according to claim 1, wherein the method further comprises:

And displaying early warning prompt information in the vehicle-mounted engineering interface, wherein the early warning prompt information is used for indicating that the first calibration parameter of the sensor to be calibrated is in an abnormal state.

3. The method according to claim 1, wherein the first preset condition is: the target pixel deviation of the first image data and the first point cloud data is smaller than or equal to a preset deviation threshold value; the method further comprises the steps of:

gridding the first image data and the first point cloud data, and determining pixel deviation between the pixel point of each grid in the first image data and the pixel point of the corresponding grid in the first point cloud data;

and taking the maximum pixel deviation in each pixel deviation as the target pixel deviation.

4. The method according to claim 1, wherein the method further comprises:

and aligning the reference sensor and the sensor to be calibrated based on the first spatial position information of the reference sensor and the second spatial position information of the sensor to be calibrated by taking the preset shaft center of the vehicle as a calibration origin, so that the second image data and the second point cloud data meet the first preset condition, and the first calibration parameter is determined.

5. The method according to claim 4, wherein the aligning the reference sensor and the sensor to be calibrated based on the first spatial position information of the reference sensor and the second spatial position information of the sensor to be calibrated with the preset axis center of the vehicle as the calibration origin, so that the second image data and the second point cloud data satisfy the first preset condition, thereby determining the first calibration parameter, includes:

determining a first space vector from a first position point of a target point in a calibration tool to a second position point of a reference sensor and a second space vector from the first position point to a third position point of the sensor to be calibrated according to the first space position information and the second space position information;

aligning the reference sensor and the sensor to be calibrated according to the first space vector and the second space vector so that a first offset vector from the first position to the calibration origin and a second offset vector from the second position to the calibration origin meet a second preset condition, and taking the second offset vector as the first calibration parameter; wherein,

The second preset condition is: the vector after the first space vector and the first offset vector are added is equal to a target space vector from the locus of the target point to the calibration origin, and the vector after the second space vector and the second offset vector are added is equal to the target space vector.

6. The method of claim 1, wherein the responding to the parameter correction command applied to the vehicle-mounted engineering interface performs parameter compensation on the first calibration parameter according to the spatial deviation vector of the sensor to be calibrated, and before obtaining the second calibration parameter, the method further comprises:

and responding to a recalibration instruction acting on the vehicle-mounted engineering interface, and realigning the sensor to be calibrated to obtain a space deviation vector of the sensor to be calibrated.

7. The method of claim 6, wherein the realigning the sensor to be calibrated in response to a recalibration instruction acting on the on-board engineering interface to obtain a spatial deviation vector of the sensor to be calibrated comprises:

responding to a recalibration instruction acting on the vehicle-mounted engineering interface, and determining a third space vector from a first position of a target point in a calibration tool to a fourth position of the sensor to be calibrated by taking the center of a rear axle of the vehicle as a calibration origin;

And subtracting the second space vector from the third space vector to obtain the space deviation vector.

8. The method according to claim 1, wherein the performing parameter compensation on the first calibration parameter according to the spatial deviation vector of the sensor to be calibrated to obtain a second calibration parameter includes:

and adding the space deviation vector and the first calibration parameter, thereby completing parameter compensation of the first calibration parameter and obtaining the second calibration parameter.

9. The method of any one of claims 1 to 8, wherein the content displayed in the on-board engineering interface comprises at least one of:

the relative position information, the early warning prompt information, the spatial deviation vector, the target pixel deviation, the preset deviation threshold value, the first calibration parameter and the second calibration parameter of the first image data and the first point cloud data.

10. A parameter calibration device for a vehicle sensor, the device comprising:

11. An electronic device, the electronic device comprising: a processor and a memory for storing a computer program, the processor being adapted to invoke and run the computer program stored in the memory for performing the parameter calibration method of a vehicle sensor according to any of claims 1 to 9.

12. A vehicle, characterized in that it comprises a parameter calibration device of a vehicle sensor according to claim 10.

13. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program which, when executed by at least one processor, implements a method for calibrating parameters of a vehicle sensor according to any of claims 1 to 9.