CN117268450A - Calibration quality online verification method and device for vehicle-mounted sensor and electronic equipment - Google Patents

Calibration quality online verification method and device for vehicle-mounted sensor and electronic equipment Download PDF

Info

Publication number
CN117268450A
CN117268450A CN202311195999.5A CN202311195999A CN117268450A CN 117268450 A CN117268450 A CN 117268450A CN 202311195999 A CN202311195999 A CN 202311195999A CN 117268450 A CN117268450 A CN 117268450A
Authority
CN
China
Prior art keywords
target
vehicle
current frame
sensing
mounted sensor
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202311195999.5A
Other languages
Chinese (zh)
Inventor
李健
杜明
姜南
刘新宇
鲁小伟
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Mushroom Car Union Information Technology Co Ltd
Original Assignee
Mushroom Car Union Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Mushroom Car Union Information Technology Co Ltd filed Critical Mushroom Car Union Information Technology Co Ltd
Priority to CN202311195999.5A priority Critical patent/CN117268450A/en
Publication of CN117268450A publication Critical patent/CN117268450A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01D—MEASURING NOT SPECIALLY ADAPTED FOR A SPECIFIC VARIABLE; ARRANGEMENTS FOR MEASURING TWO OR MORE VARIABLES NOT COVERED IN A SINGLE OTHER SUBCLASS; TARIFF METERING APPARATUS; MEASURING OR TESTING NOT OTHERWISE PROVIDED FOR
    • G01D18/00—Testing or calibrating apparatus or arrangements provided for in groups G01D1/00 - G01D15/00
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30244—Camera pose
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D30/00—Reducing energy consumption in communication networks
    • Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Theoretical Computer Science (AREA)
  • Traffic Control Systems (AREA)

Abstract

The application discloses an on-line calibration quality verification method and device for a vehicle-mounted sensor and electronic equipment. The method comprises the following steps: the method comprises the steps of obtaining sensing data and true value data of a current frame, wherein the sensing data is obtained according to sensing information of a vehicle-mounted sensor on an obstacle, and comprises more than one sensing target and sensing positions and sensing attributes of each sensing target; the truth value data is obtained through information of the intelligent road side terminal sent by the intelligent road side terminal and comprises a truth value target, and a truth value position and a truth value attribute of the truth value target; determining whether a road side intelligent terminal sensing target belonging to the same road side intelligent terminal as the true value target exists in the current frame sensing target according to the sensing data and the true value data of the current frame; if the real-value target exists, verifying the calibration quality of the vehicle-mounted sensor according to the real-value target and the road side intelligent terminal perception target; and determining a maintenance scheme of the vehicle-mounted sensor according to the calibration quality check result so as to maintain the vehicle-mounted sensor according to the maintenance scheme of the vehicle-mounted sensor.

Description

Calibration quality online verification method and device for vehicle-mounted sensor and electronic equipment
Technical Field
The application relates to the technical field of automatic driving, in particular to an on-line calibration quality verification method and device for a vehicle-mounted sensor and electronic equipment.
Background
With the development of artificial intelligence and 5G technology, more and more companies and universities have focused on the study of automated driving automobiles (Autonomous Vehicles). The automatic driving automobile is an intelligent automobile system for realizing unmanned through a vehicle-mounted computer system, which is also called an unmanned automobile, a computer driving automobile or a wheel type mobile robot, and comprises a sensing system, a decision system, an execution system and a communication system. The sensing system is an intelligent core of automatic driving, and the calibration of the sensor is a basis for maintaining the normal operation of the sensing system.
Currently, the calibration method of the vehicle-mounted sensor is mostly off-line calibration, for example, calibration workshop or manual calibration is used for calibrating initial external parameters, but the relative pose of the sensor changes due to various factors such as vibration during running of an automatic driving vehicle or after a period of time.
Aiming at the problems, when the calibration parameter problem of the vehicle appears in the long-term use process, the prior art directly returns to the factory for re-calibration or adopts an online calibration scheme to recalculate the calibration parameter. However, the factory re-calibration time is long, the cost is high, the user experience is poor, most of the existing online calibration schemes are based on the matching of the linear characteristics of the image data and the point cloud data, and the existing matching algorithm is difficult to obtain an online calibration result with higher precision.
Disclosure of Invention
Aiming at the problems in the prior art, the embodiment of the application provides an on-line calibration method, device and electronic equipment for the calibration quality of a vehicle-mounted sensor, which are used for carrying out quantitative calibration on the calibration quality of the vehicle-mounted sensor so as to formulate a corresponding maintenance scheme based on a quantitative calibration result.
The embodiment of the application adopts the following technical scheme:
in a first aspect, an embodiment of the present application provides an online calibration method for calibration quality of a vehicle-mounted sensor, where the method includes:
obtaining perception data and true value data of a current frame, wherein the perception data is obtained according to the perception information of a vehicle-mounted sensor on an obstacle, and the perception data comprises more than one perception target and the perception position and the perception attribute of each perception target; the truth data is obtained through information of the intelligent road side terminal sent by the intelligent road side terminal, and the truth data comprises a truth target, and a truth position and a truth attribute of the truth target;
determining whether a road side intelligent terminal sensing target belonging to the same road side intelligent terminal as the true value target of the current frame exists in the sensing target of the current frame according to the sensing data and the true value data of the current frame;
When a road side intelligent terminal sensing target belonging to the same road side intelligent terminal as a true value target of the current frame exists in a sensing target of the current frame, checking the calibration quality of the vehicle-mounted sensor according to the true value target belonging to the same road side intelligent terminal in the current frame and the road side intelligent terminal sensing target to obtain a calibration quality checking result;
and determining a maintenance scheme of the vehicle-mounted sensor according to the calibration quality check result so as to maintain the vehicle-mounted sensor according to the maintenance scheme of the vehicle-mounted sensor.
In a second aspect, an embodiment of the present application further provides an on-line calibration quality calibration device for an on-vehicle sensor, where the device includes:
the data acquisition unit is used for acquiring the perception data and the true value data of the current frame, wherein the perception data are acquired according to the perception information of the vehicle-mounted sensor on the obstacle, and the perception data comprise more than one perception target and the perception position and the perception attribute of each perception target; the truth data is obtained through information of the intelligent road side terminal sent by the intelligent road side terminal, and the truth data comprises a truth target, and a truth position and a truth attribute of the truth target;
The target matching unit is used for determining whether a road side intelligent terminal sensing target belonging to the same road side intelligent terminal as the true value target of the current frame exists in the sensing target of the current frame according to the sensing data and the true value data of the current frame;
the calibration quality verification unit is used for verifying the calibration quality of the vehicle-mounted sensor according to the true value target belonging to the same road side intelligent terminal in the current frame and the road side intelligent terminal sensing target when the road side intelligent terminal sensing target belonging to the same road side intelligent terminal as the true value target of the current frame exists in the sensing target of the current frame, so as to obtain a calibration quality verification result;
and the maintenance scheme determining unit is used for determining the maintenance scheme of the vehicle-mounted sensor according to the calibration quality check result so as to maintain the vehicle-mounted sensor according to the maintenance scheme of the vehicle-mounted sensor.
In a third aspect, embodiments of the present application further provide an electronic device, including:
a processor; and
a memory arranged to store computer executable instructions that, when executed, cause the processor to perform a calibration quality on-line verification method for an on-board sensor.
In a fourth aspect, embodiments of the present application also provide a computer-readable storage medium storing one or more programs that, when executed by an electronic device that includes a plurality of application programs, cause the electronic device to perform a calibration quality online verification method of an in-vehicle sensor.
The above-mentioned at least one technical scheme that this application embodiment adopted can reach following beneficial effect:
according to the method and the device, the road side intelligent terminal information sent by the road side intelligent terminal is used as true value data, obstacle sensing information provided by the vehicle-mounted sensor is used as sensing data to be checked, and calibration quality of the vehicle-mounted sensor is checked according to the true value data and the sensing data, so that the checking accuracy of the calibration quality is improved through the multi-device collaborative sensing scheme, in addition, the maintenance scheme suitable for the vehicle-mounted sensor is determined according to the calibration quality checking result, and the recalibration efficiency of the vehicle-mounted sensor is improved to a certain extent.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the application and do not constitute an undue limitation to the application. In the drawings:
FIG. 1 is a flow chart of an on-line calibration quality verification method for a vehicle-mounted sensor in an embodiment of the application;
fig. 2 is a schematic diagram of a data packet structure based on a multi-device collaborative information interaction protocol according to an embodiment of the present application;
FIG. 3 is a schematic flow chart of calibration quality online verification and replenishment shown in an embodiment of the present application;
fig. 4 is a schematic structural diagram of an on-line calibration quality checking device for a vehicle-mounted sensor according to an embodiment of the present application;
fig. 5 is a schematic structural diagram of an electronic device in an embodiment of the present application.
Detailed Description
For the purposes, technical solutions and advantages of the present application, the technical solutions of the present application will be clearly and completely described below with reference to specific embodiments of the present application and corresponding drawings. It will be apparent that the described embodiments are only some, but not all, of the embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present disclosure.
The following describes in detail the technical solutions provided by the embodiments of the present application with reference to the accompanying drawings.
The embodiment of the application provides an on-line calibration quality verification method for a vehicle-mounted sensor, as shown in fig. 1, and provides a flow chart diagram of the on-line calibration quality verification method for the vehicle-mounted sensor in the embodiment of the application, wherein the method comprises the following steps S110 to S140:
step S110, obtaining the perception data and the true value data of the current frame, wherein the perception data is obtained according to the perception information of the vehicle-mounted sensor on the obstacle, and the perception data comprises more than one perception target and the perception position and the perception attribute of each perception target; the truth data is obtained through information of the intelligent road side terminal sent by the intelligent road side terminal, and the truth data comprises a truth target, and a truth position and a truth attribute of the truth target.
According to the automatic driving vehicle, information interaction can be carried out between the automatic driving vehicle and the road side intelligent terminal through the road cooperative communication link, message data sent by the road side intelligent terminal are received, the road cooperative communication link is a 5G communication link, a C-V2X communication link and the like, the road side intelligent terminal comprises road side equipment and/or networking vehicles, the road side equipment comprises a road side intelligent rod, equipment which is fixed and can be perceived by a vehicle-mounted sensor and is arranged on the road side intelligent rod, such as a road side sensor, and the networking vehicles refer to vehicles which can provide information of real-time positions, vehicle types, vehicle sizes and the like for the automatic driving vehicle through the road cooperative communication link.
In the application scenario of the embodiment of the present application, the roadside intelligent terminal may send message data carrying its own information (for example, information of a position, an attribute, etc. of the roadside intelligent terminal) to the automatic driving vehicle, and the automatic driving vehicle receives the message data sent by each roadside intelligent terminal and uses the information of the position, the attribute, etc. of the roadside intelligent terminal in the message data as true value data of the current frame.
For example, the surrounding roadside terminals and the surrounding networked vehicles of the autonomous vehicle respectively transmit message data carrying information such as the position and the attribute thereof to the autonomous vehicle, and the autonomous vehicle analyzes the message data to use the information such as the position and the attribute of the surrounding roadside terminals and the surrounding networked vehicles as truth data.
In addition, install on the autopilot vehicle on-vehicle sensor such as camera, laser radar, millimeter wave radar, can be real-time normal acquisition data after accomplishing initial calibration. Each kind of vehicle-mounted sensor gathers the environmental data around the vehicle according to its settlement operating frequency to carry out the barrier detection to every frame environmental data that gathers, obtain barrier perception information, this application embodiment regard barrier perception information that every kind of vehicle-mounted sensor detected based on every frame environmental data as perception data, and the barrier in this application embodiment can understand as various traffic participants, for example is vehicle, traffic signal equipment (e.g. traffic signal lamp, roadblock etc.), pedestrian, road side equipment, road side building.
Therefore, the embodiment of the application can perform time synchronization verification on the road side intelligent terminal information sent by each road side intelligent terminal and the obstacle sensing information sent by each vehicle-mounted sensor, takes the road side intelligent terminal information meeting the time synchronization verification as true value data of the current frame, and takes the obstacle sensing information provided by each vehicle-mounted sensor meeting the time synchronization verification as sensing data of the current frame.
It should be noted that, each frame of true value data corresponds to one intelligent terminal at the road side, each frame of true value data includes the position and attribute of the intelligent terminal at the road side, and the attribute includes information such as size, type, etc.; each frame of sensing data corresponds to one type of vehicle-mounted sensor, each frame of sensing data generally comprises sensing positions and sensing data of all obstacles sensed by the vehicle-mounted sensor, sensing attributes comprise sensing information such as sensing size and sensing type, for convenience of description, the following embodiments take that the sensing data of a current frame only comprises one frame, and the sensing data of the current frame is from one type of vehicle-mounted sensor as an example, in practical application, the sensing data of the current frame can also comprise multiple frames and come from multiple types of vehicle-mounted sensors, and when the sensing data of the current frame comprises multiple frames, the processing process of the sensing data of other frames can refer to the related description of the embodiments of the application.
Step S120, determining whether there is a road side intelligent terminal perception target belonging to the same road side intelligent terminal as the true value target of the current frame in the perception targets of the current frame according to the perception data and the true value data of the current frame.
Through the method, whether the sensing targets of the road side intelligent terminal sensing targets exist in all the sensing targets sensed by the vehicle-mounted sensor or not can be detected by performing target matching on the sensing targets of the current frame and the truth targets, and if the sensing targets exist, the position sensing deviation of the vehicle-mounted sensor can be quantitatively detected according to the truth targets and the sensing targets which are matched with each other, so that a basis is provided for the subsequent calibration quality verification of the vehicle-mounted sensor.
Step S130, when a road side intelligent terminal sensing target belonging to the same road side intelligent terminal as a true value target of the current frame exists in the sensing target of the current frame, calibrating quality of the vehicle-mounted sensor according to the true value target belonging to the same road side intelligent terminal in the current frame and the road side intelligent terminal sensing target to obtain a calibration quality calibration result.
According to the foregoing steps, the truth data includes a truth position of a truth target, where the truth position is provided by the intelligent road side terminal, for example, when the road side device is installed and deployed, the road side device locally stores basic information of the road side device, where the basic information at least records its own position and attribute data including size and type; or the road side equipment is provided with the GPS and the like sensing equipment, and the road side equipment can acquire the information of the road side equipment through the local storage file or the GPS and the like sensing equipment. And the networking vehicle can acquire the information of the real-time position, the size, the type and the like of the self through a sensing system of the networking vehicle and provide the information for the automatic driving vehicle. Therefore, the true value data in the embodiment of the application is irrelevant to the vehicle-mounted sensor, and the calibration quality of the vehicle-mounted sensor is checked by taking the data as the true value data, so that the accuracy and the reliability of a check result can be improved.
In the embodiment of the application, the sensing data is provided by the vehicle-mounted sensor, taking the vehicle-mounted camera as an example, when the surrounding obstacles of the automatic driving vehicle are sensed by the vehicle-mounted camera, the sensing positions of the obstacles around the vehicle are required to be calculated through the calibration parameters of the vehicle-mounted camera, so that when a certain sensing target sensed by the vehicle-mounted sensor and a truth target are determined to be the same object, the deviation condition of the sensing target can be measured based on the sensing positions of the sensing target and the truth target, and further the calibration quality of the vehicle-mounted sensor is quantitatively verified according to the deviation condition, so that the current calibration quality of the vehicle-mounted sensor is applicable to which maintenance scheme can be determined.
And step S140, determining a maintenance scheme of the vehicle-mounted sensor according to the calibration quality check result so as to maintain the vehicle-mounted sensor according to the maintenance scheme of the vehicle-mounted sensor.
According to the embodiment of the application, various maintenance schemes can be formulated in advance, for example, when the calibration quality check result indicates that the current calibration quality of the vehicle-mounted sensor is qualified, the maintenance scheme at the moment can be continuous monitoring, namely, the calibration quality of the vehicle-mounted sensor can be continuously monitored; when the calibration quality check result indicates that the current calibration quality of the vehicle-mounted sensor is unqualified, the maintenance scheme at the moment is online compensation, namely, the sensing position of each sensing target of the vehicle-mounted sensor is online compensated; when the calibration quality check result indicates that the current calibration quality of the vehicle-mounted sensor alarms, the maintenance scheme at the moment is off-line recalibration, for example, maintenance personnel are informed to carry out factory return calibration on the vehicle-mounted sensor.
The main difference between the calibration quality alarm and the calibration quality disqualification is that when the calibration quality is disqualified, the vehicle-mounted sensor has smaller position sensing deviation on the obstacle, and the position sensing deviation caused by the calibration parameter error can be overcome by an online compensation mode; when the calibration quality alarms, the vehicle-mounted sensor has larger position sensing deviation on the obstacle, and at the moment, the position sensing deviation caused by the calibration parameter error cannot be overcome by an online compensation mode, and the recalibration can only be carried out by an offline mode.
As can be seen from the calibration quality online verification method of the vehicle-mounted sensor shown in fig. 1, in the embodiment of the present application, road side intelligent terminal information sent by the road side intelligent terminal is used as true value data, obstacle sensing information provided by the vehicle-mounted sensor is used as sensing data to be verified, and the calibration quality of the vehicle-mounted sensor is verified according to the true value data and the sensing data, so that the verification accuracy of the calibration quality can be improved through the multi-device collaborative sensing scheme, in addition, the maintenance scheme suitable for the vehicle-mounted sensor is determined according to the calibration quality verification result, and the calibration efficiency of the vehicle-mounted sensor is improved to a certain extent.
In some embodiments of the present application, in the step S110, the obtaining the perceived data and the true value data of the current frame specifically includes:
firstly, analyzing a first data packet from a road side intelligent terminal and a second data packet from a vehicle-mounted sensor according to a preset multi-device cooperative information interaction protocol to obtain a first analysis result corresponding to the first data packet and a second analysis result corresponding to the second data packet.
The first analysis result comprises a first timestamp of a first data packet and the road side intelligent terminal information, the road side intelligent terminal information comprises an identifier of the road side intelligent terminal and positions and attributes of the road side intelligent terminal, the second analysis result comprises a second timestamp of a second data packet and more than one obstacle perception information, and the obstacle perception information comprises an identifier of an obstacle, a perception position and a perception attribute of the obstacle.
And secondly, determining whether the first data packet and the second data packet are time synchronous or not according to the first time stamp and the second time stamp.
For example, calculating a time stamp difference value of the first data packet and the second data packet, comparing the time stamp difference value with a preset time difference threshold, if the time stamp difference value is not larger than the time difference threshold, determining that the first data packet is time-synchronized with the second data packet, otherwise, determining that the first data packet is not time-synchronized with the second data packet, when the first data packet is not time-synchronized with the second data packet, ending the online verification, generating a time difference alarm log, continuously collecting true value data and perception data of the next frame, and if continuous multi-frame data do not meet the time synchronization, generating a time difference alarm reminder. Wherein the time difference threshold may be empirically set.
When the first data packet is time-synchronized with the second data packet, obtaining the true value data of the current frame according to the first analysis result, wherein the road side intelligent terminal is used as a true value target, and the position and the attribute of the road side intelligent terminal are used as the true value position and the true value attribute of the true value target; and obtaining the perception data of the current frame according to the second analysis result, wherein each obstacle is taken as a perception target, and the perception position and the perception attribute of each obstacle are taken as the perception position and the perception attribute of the perception target.
In some scenarios, a multi-device collaborative information interaction protocol may be formulated in advance, where the interaction protocol specifies a data packet format, where the data packet format is shown with reference to fig. 2, and the data header carries a timestamp, device information, a data packet ID, and the like, and the data body carries a truth value target or a perception target, and when the data packet is generated by the intelligent road side terminal, the data body carries the truth value target, where the truth value target is specifically an identifier of the intelligent road side terminal; when the data packet is generated by the vehicle-mounted sensor or the sensing system of the automatic driving vehicle, the data body carries a sensing target, and the sensing target is specifically the identification of all the obstacles sensed by the vehicle-mounted sensor, wherein the data body also carries information such as the position, the attribute and the like of the truth target or the sensing target. The data packet further includes a data end identifier, which may be flexibly set, which is not particularly limited in this application.
It should be noted that, in this embodiment, the information security transmission may be ensured through a custom multi-device cooperative information interaction protocol, and in other embodiments, the intelligent terminal on the road side and the automatic driving vehicle may also perform message interaction based on the existing communication protocol.
In some embodiments of the present application, the truth data of the current frame includes one or more frames, each frame of the truth data includes a different truth target, for example, the truth data of the current frame includes three frames, one frame is from a roadside intelligent pole, one frame is from a roadside sensor mounted on the roadside intelligent pole, and the other frame is from a surrounding networked vehicle, so that the truth target of the truth data of the first frame in the current frame corresponds to the roadside intelligent pole, the truth target of the truth data of the second frame corresponds to the roadside sensor, and the truth target of the truth data of the third frame corresponds to the networked vehicle.
Based on the application scenario, in the step S120, according to the perceived data and the truth data of the current frame, it is determined whether there is a road side intelligent terminal perceived target belonging to the same road side intelligent terminal as the truth target of the current frame in the perceived target of the current frame, which specifically includes:
acquiring multidimensional distance features of each truth value target in the current frame and each perception target in the current frame according to the perception position and the perception attribute of each perception target in the current frame and according to the truth value position and the truth value attribute of each truth value target in the current frame;
According to the multidimensional distance characteristics of each truth value target in the current frame and each perception target in the current frame, obtaining the similarity of each truth value target in the current frame and each perception target in the current frame;
and determining whether a road side intelligent terminal perception target belonging to the same road side intelligent terminal as the true value target of the current frame exists in the perception targets of the current frame according to the similarity between each true value target in the current frame and each perception target in the current frame.
For example, in some scenarios, a euclidean distance_o between each true value target in the current frame and each perceived target in the current frame may be calculated by a nearest neighbor matching algorithm according to the true value position of each true value target in the current frame and the perceived position of each perceived target in the current frame; calculating the Markov distance_m between each true value target in the current frame and each perceived target in the current frame according to the true value position and the true value attribute (such as the true value size and/or the true value type) of each true value target in the current frame and the perceived position and the perceived attribute (such as the perceived size and/or the perceived type) of each perceived target in the current frame; based on the assumption, three sets of data can be obtained, wherein the actual object corresponding to the first set of data is a road side intelligent rod, the actual object corresponding to the second set of data is a road side sensor, and the actual object corresponding to the third set of data is a networking vehicle.
Then, performing similarity calculation on the three groups of data according to a formula of similarity=w1×distance_o+w2×distance_m, and sequencing the similarity calculation results of the three groups of data to obtain a similarity maximum value corresponding to each data; here w1, w2 are weight factors, w1+w2=1.
It should be noted that, in order to avoid the influence of the elevation information of the intelligent terminal at the road side on the target matching and the position sensing deviation of the vehicle-mounted sensor, the position of the embodiment of the application may be understood as the road bed position, for example, the Z-axis data may be uniformly set to a determined value under the XYZ coordinate system.
And finally, judging whether a road side intelligent rod sensing target, a road side sensor sensing target and a networking vehicle sensing target exist in the sensing targets of the current frame according to the maximum value of the similarity in the three groups of data.
In some implementations of this embodiment, determining, according to the similarity between each truth target in the current frame and each perceived target in the current frame, whether there is a perceived target of a roadside intelligent terminal that belongs to the same roadside intelligent terminal as the truth target of the current frame in the perceived targets of the current frame specifically includes:
determining a similarity maximum value similarity_max from the similarity between the true value target in the current frame and each perception target of the current frame, and comparing the similarity maximum value similarity_max with a preset similarity threshold value Match_Thresh;
If the similarity maximum value similarity_max is larger than the similarity threshold value Match_Thresh, determining that a perception target corresponding to the similarity maximum value and the true value target of the current frame belong to the same-path-side intelligent terminal;
and if the similarity maximum value similarity_max is not greater than the similarity threshold value Match_Thresh, determining that no sensing target belonging to the same intelligent terminal as the true target of the current frame exists in the current frame.
Still based on the assumption, if the maximum value of the similarity corresponding to each set of data in the three sets of data is greater than the similarity threshold, that is, the similarity_max > match_thresh, it can be stated that the true value target corresponding to the maximum value of the similarity is matched with the perceived target, and the true value target and the perceived target are the same object, and at this time, it can be determined that the perceived target of the current frame has the perceived target of the intelligent pole on the road side, the perceived target of the sensor on the road side and the perceived target of the networked vehicle. In contrast, if the maximum value of the similarity corresponding to each set of data in the three sets of data is not greater than the similarity threshold, that is, the similarity_max is less than or equal to match_thresh, it can be stated that no matched sensing target exists in the current frame, and at this time, it can be determined that no roadside intelligent rod sensing target, roadside sensor sensing target and networking vehicle sensing target exist in the sensing target of the current frame.
In this way, through the above embodiment, it may be determined whether there is a road side intelligent terminal sensing target belonging to the same road side intelligent terminal as the true value target of the current frame in the sensing target of the current frame, and when there is a road side intelligent terminal sensing target belonging to the same road side intelligent terminal as the true value target of the current frame in the sensing target of the current frame, the calibration quality of the vehicle-mounted sensor is checked according to the true value target belonging to the same road side intelligent terminal as the road side intelligent terminal sensing target in the current frame in step S130, so as to obtain a calibration quality check result, which specifically includes:
acquiring the current position sensing deviation of the vehicle-mounted sensor according to the true value position of the true value target belonging to the same road side intelligent terminal in the current frame and the sensing position of the road side intelligent terminal sensing target;
and acquiring a calibration quality check result of the vehicle-mounted sensor according to the current position sensing deviation of the vehicle-mounted sensor.
According to the method and the device for calculating the current position sensing deviation of the vehicle-mounted sensor, when the current position sensing deviation of the vehicle-mounted sensor is calculated, the multi-point position sensing results of the vehicle-mounted sensor can be counted, the current position sensing deviation of the vehicle-mounted sensor is measured based on the counted results of a period of time, and the accuracy and the reliability of calculating the position sensing deviation are improved.
Specifically, in some possible implementation solutions of this embodiment, the obtaining the current position sensing deviation of the vehicle-mounted sensor according to the true value position of the true value target belonging to the same roadside intelligent terminal and the sensing position of the roadside intelligent terminal sensing target in the current frame specifically includes:
firstly, converting the true value position of a true value target belonging to the same intelligent terminal at the same road side in a current frame and the perception position of a perception target of the intelligent terminal at the road side into the same coordinate system according to the calibration parameters of the vehicle-mounted sensor; the same coordinate system here may be a world coordinate system, a vehicle body coordinate system or a vehicle-mounted sensor coordinate system.
For example, in some scenarios, assuming that the current vehicle-mounted sensor is a vehicle-mounted camera, a truth target in truth data of each frame in the current frame may be projected into an image coordinate system according to calibration parameters and a truth position of the vehicle-mounted camera, so that a positional deviation between the truth target and a perceived target of the roadside intelligent terminal may be calculated under the image coordinate system, where the coordinate conversion scheme may refer to the prior art, and embodiments of the present application are not illustrated in detail herein.
And then, acquiring the intra-frame position sensing deviation of the vehicle-mounted sensor in the current frame according to the positions of the true value target belonging to the same road side intelligent terminal and the road side intelligent terminal sensing target in the current frame under the same coordinate system.
If n pairs of true-value targets belonging to the same intelligent terminal at the road side and intelligent terminal perception targets at the road side exist in the current frame, the position perception deviation of the vehicle-mounted sensor in the current frame can be calculated according to the average value or the weighted value of the position deviation between the n pairs of targets. Taking the example that three target pairs exist in the current frame assumed in the foregoing, namely, a target pair corresponding to a road side intelligent rod, a target pair corresponding to a road side sensor and a target pair corresponding to a networking vehicle, the mean value of the position deviations of the three target pairs can be used as the mean value of the position deviations of the vehicle-mounted camera in the current frame. For example, the mean value of the position deviation of the vehicle-mounted camera in the current frame can be calculated according to the formula (1).
In the above formula (1), pixel _ base _ i is the position coordinate of the true value target in the image coordinate system,Pixelthe _persistence_i is the position coordinate of the perceived target in the image coordinate system, and n is the target pair that the current frame shares n pairs of mutually matched targets.
And then, carrying out deviation statistics on the intra-frame position sensing deviation of the continuous M frames including the current frame, and obtaining an intra-frame position sensing deviation statistical result of the vehicle-mounted sensor in the current frame, wherein the intra-frame position sensing deviation statistical result comprises an average value and a variance value of the intra-frame position sensing deviation, and M is a positive integer larger than 1.
And finally, acquiring the current position sensing deviation of the vehicle-mounted sensor according to the intra-frame position sensing deviation statistical result of the vehicle-mounted sensor in the current frame.
Therefore, when the intra-frame position sensing deviation of the vehicle-mounted sensor is obtained, the embodiment can also count the intra-frame position sensing deviation of the multiple points within a period of time, obtain the average value ym_mean and the variance value ym_diff corresponding to the intra-frame position sensing deviation of the multiple points, and measure the current position sensing deviation of the vehicle-mounted sensor through the intra-frame position sensing deviation statistics result of the continuous M frames.
Thus, after obtaining the current position sensing deviation of the vehicle-mounted sensor, in other implementations of the present embodiment, the obtaining the calibration quality check result of the vehicle-mounted sensor according to the current position sensing deviation of the vehicle-mounted sensor specifically includes:
comparing the variance value YM_diff with a preset safety variance threshold value YTHresh_diff, and if the variance value YM_diff is larger than the safety variance threshold value YTHresh_diff, determining that a calibration quality check result of the vehicle-mounted sensor is a calibration quality alarm; here, the safety variance threshold may be set empirically, and when the variance value ym_diff is greater than the safety variance threshold ythresh_diff, it is indicated that the fluctuation degree of the position sensing deviation of the vehicle-mounted sensor to the obstacle is severe, and in this case, the calibration parameter error of the vehicle-mounted sensor is large, and offline recalibration is required.
If the variance value YM_diff is not larger than the safety variance threshold value YTHresh_diff, the fluctuation degree of the position sensing deviation of the vehicle-mounted sensor to the obstacle is relatively gentle, at the moment, the average value YM_mean and a preset safety average threshold value YTHresh_mean can be compared, if the average value YM_mean is larger than the safety average threshold value YTHresh_mean, the calibration quality check result of the vehicle-mounted sensor is determined to be that the calibration quality is unqualified, and at the moment, the sensing position of each sensing target of the vehicle-mounted sensor is compensated online; and if the average value YM_mean is not greater than the safe average threshold value of YTHresh_mean, determining that the calibration quality check result of the vehicle-mounted sensor is qualified in calibration quality, and continuously monitoring the calibration quality of the vehicle-mounted sensor at the moment.
According to the above embodiment, in the embodiment of the present application, the calibration quality check results about the vehicle-mounted sensor include three types, that is, the calibration quality alarm, the calibration quality failure, and the calibration quality qualification, respectively.
In some embodiments of the present application, the determining, in step S140, a maintenance scheme of the vehicle-mounted sensor according to the calibration quality check result specifically includes:
When the calibration quality check result of the vehicle-mounted sensor is a calibration quality alarm, determining that the maintenance scheme of the vehicle-mounted sensor is off-line recalibration;
when the calibration quality check result of the vehicle-mounted sensor is that the calibration quality is qualified, determining that the maintenance scheme of the vehicle-mounted sensor is continuous monitoring;
and when the calibration quality check result of the vehicle-mounted sensor is that the calibration quality is unqualified, determining that the maintenance scheme of the vehicle-mounted sensor is online compensation.
In some possible implementations of this embodiment, when the calibration quality check result of the vehicle-mounted sensor is that the calibration quality is qualified, the accumulated number of times that the calibration quality is qualified may be updated, and if the updated accumulated number of times that the calibration quality is qualified is greater than the corresponding number of times threshold, the maintenance scheme of the vehicle-mounted sensor is determined to be continuously monitored, so that accuracy of the determined maintenance scheme may be improved.
In other possible implementations of the embodiment, when the calibration quality check result of the vehicle-mounted sensor is that the calibration quality is not qualified, the accumulated number of times that the calibration quality is not qualified may be updated, and if the updated accumulated number of times that the calibration quality is not qualified is greater than a preset number of times threshold, it is determined that the maintenance scheme of the vehicle-mounted sensor is offline recalibration; that is, in some scenarios, there may be a situation that the sensing deviation caused by the calibration parameter error still cannot be overcome after multiple online compensations, and in this case, the off-line recalibration is performed on the vehicle-mounted sensor in this embodiment.
And if the updated accumulated times of unqualified calibration quality are not greater than the times threshold, determining that the maintenance scheme of the vehicle-mounted sensor is online compensation. Thus, the accuracy of the determined maintenance scheme can be improved by combining the accumulated times of unqualified calibration quality.
It will be appreciated that the two thresholds may be the same or different, and that one skilled in the art may set the thresholds empirically.
In some embodiments of the present application, when the maintenance scheme of the in-vehicle sensor is determined to be online compensation, the online compensation may be performed by:
acquiring a position compensation value; for example, in some scenarios, the position compensation value may be obtained according to the intra-frame position sensing deviation of the vehicle-mounted sensor in the current frame and a preset scaling factor, for example, a product of the scaling factor and the intra-frame position sensing deviation of the vehicle-mounted sensor in the current frame is used as the position compensation value, where the scaling factor is a positive number smaller than 1, and the compensation accuracy of the sensing position is controlled through the scaling factor, so that adverse effects on the driving safety of the automatic driving vehicle caused by excessive single compensation value are avoided; it will be appreciated that in other scenarios, the position compensation value may also be obtained according to an average value of intra-frame position sensing deviations of the in-vehicle sensor and a preset scaling factor.
And generating a position compensation command according to the position compensation value, and controlling the vehicle-mounted sensor to perform position compensation on obstacle sensing information in the sensing data of the next frame according to the position compensation command.
It should be noted that, the automatic driving vehicle in the embodiment of the present application may execute the calibration quality on-line calibration method for the on-vehicle sensor in a non-driving state such as a temporary stop when meeting a red light, so as to avoid adverse effects on the driving state of the automatic driving vehicle when executing the on-line compensation scheme.
An online verification process of calibration quality of the vehicle-mounted camera and an online compensation process of perceived positions of each perceived target of the vehicle-mounted camera according to the embodiment of the present application are described in detail below with reference to fig. 3.
When the current running state of the automatic driving vehicle is determined to be in accordance with the on-line verification condition, for example, the automatic driving vehicle is in a non-running state currently, the automatic driving vehicle can be determined to be in accordance with the on-line verification condition, at the moment, the on-line verification of the calibration quality of the vehicle-mounted camera can be started, as shown in fig. 3, a calibration detection system node is started, the sending data of the intelligent terminal at the road side and the sending data of the vehicle-mounted camera are monitored, the time synchronization verification is carried out on the received data frames, when the two frames of data pass the time synchronization verification, the target matching is carried out on the true value target and the perception target in the two frames of data, and if the matching is unsuccessful, the next frame of data is continuously monitored; if the matching is successful, the calibration quality of the vehicle-mounted camera is checked according to the position information of the true value target and the sensing target which are matched with each other, when the calibration quality check result is that the calibration quality is unqualified, the true value position of the true value target is mapped into an image coordinate system of the vehicle-mounted camera, the current position sensing deviation of the vehicle-mounted camera is calculated according to the mapping position of the true value target and the sensing position of the sensing target, the position compensation value is used for carrying out position compensation on each sensing target in the next frame of the vehicle-mounted camera according to the fineness of the position compensation value, and therefore, when the calibration parameter error of the vehicle-mounted camera is smaller, the position sensing precision of the vehicle-mounted camera can be improved through closed-loop online position compensation, and the adverse effect caused by the calibration parameter error is compensated.
Based on the above embodiments of the present application, the present application provides an online calibration scheme for calibration quality based on vehicle-road coordination, which can utilize a road side intelligent terminal to perform real-time calibration on the calibration quality of a vehicle-mounted sensor in the running or waiting process of an automatic driving vehicle, and determine a maintenance scheme suitable for the vehicle-mounted sensor according to the real-time calibration result, so as to avoid direct factory return calibration under the condition of small calibration parameter error; when the on-line compensation of the vehicle-mounted sensor is determined, the position compensation value is determined by using the vehicle-road cooperation scheme, so that the problems of low accuracy of the calibration result and unreliable calibration result caused by calculating the calibration result by using only the environmental data collected by the vehicle-mounted sensor can be avoided; when on-line compensation is performed, the obstacle sensing position of the vehicle-mounted sensor is finely adjusted through the scaling coefficient, so that the influence of on-line compensation on the running safety of the automatic driving vehicle can be reduced; in addition, the information interaction between the intelligent road side terminal and the automatic driving vehicle is realized through the self-defined multi-equipment cooperative information interaction protocol, so that the safety and convenience of information transmission are ensured; and in the target matching stage of the truth value data and the perception data, correlation calculation is carried out based on Euclidean distance and Mahalanobis distance as multi-dimensional distance characteristics, and the accuracy of matching between the truth value target and the perception target is improved, so that the accuracy of the calibration quality of the vehicle-mounted sensor is improved.
The embodiment of the application also provides an on-line calibration quality verification device 400 of the vehicle-mounted sensor, as shown in fig. 4, and provides a schematic structural diagram of the on-line calibration quality verification device of the vehicle-mounted sensor in the embodiment of the application, where the device 400 includes: a data acquisition unit 410, a target matching unit 420, a calibration quality checking unit 430, and a maintenance scheme determination unit 440, wherein:
the data acquisition unit 410 is configured to acquire sensing data and true value data of a current frame, where the sensing data is acquired according to sensing information of an obstacle by a vehicle-mounted sensor, and the sensing data includes more than one sensing target and a sensing position and a sensing attribute of each sensing target; the truth data is obtained through information of the intelligent road side terminal sent by the intelligent road side terminal, and the truth data comprises a truth target, and a truth position and a truth attribute of the truth target;
the target matching unit 420 is configured to determine, according to the perceived data and the truth data of the current frame, whether there is a road side intelligent terminal perceived target that belongs to the same road side intelligent terminal as the truth target of the current frame in the perceived targets of the current frame;
The calibration quality verification unit 430 is configured to, when there is a road side intelligent terminal sensing target belonging to the same road side intelligent terminal as the true value target of the current frame in the sensing target of the current frame, verify the calibration quality of the vehicle-mounted sensor according to the true value target belonging to the same road side intelligent terminal in the current frame and the road side intelligent terminal sensing target, and obtain a calibration quality verification result;
and a maintenance scheme determining unit 440, configured to determine a maintenance scheme of the vehicle-mounted sensor according to the calibration quality check result, so as to maintain the vehicle-mounted sensor according to the maintenance scheme of the vehicle-mounted sensor.
In one embodiment of the present application, the truth data of the current frame includes one or more frames, each frame of the truth data includes different truth targets, and the target matching unit 420 is specifically configured to obtain a multidimensional distance feature between each truth target in the current frame and each sense target in the current frame according to the sense location and the sense attribute of each sense target in the current frame, and according to the truth location and the truth attribute of each truth target in the current frame; according to the multidimensional distance characteristics of each truth value target in the current frame and each perception target in the current frame, obtaining the similarity of each truth value target in the current frame and each perception target in the current frame; and determining whether a road side intelligent terminal perception target belonging to the same road side intelligent terminal as the true value target of the current frame exists in the perception targets of the current frame according to the similarity between each true value target in the current frame and each perception target in the current frame.
In one embodiment of the present application, the object matching unit 420 is specifically configured to determine a similarity maximum value from the similarity between the true value object in the current frame and each perceived object of the current frame, and compare the similarity maximum value with a preset similarity threshold; if the maximum value of the similarity is larger than the similarity threshold value, determining that a perception target corresponding to the maximum value of the similarity and the true value target of the current frame belong to the same intelligent terminal at one road side; and if the maximum value of the similarity is not greater than the similarity threshold value, determining that the sensing target belonging to the same road side intelligent terminal as the true target of the current frame does not exist in the current frame.
In one embodiment of the present application, the calibration quality checking unit 430 is specifically configured to obtain a current position sensing deviation of the vehicle-mounted sensor according to a true value position of a true value target belonging to the same roadside intelligent terminal and a sensing position of a roadside intelligent terminal sensing target in a current frame; and acquiring a calibration quality check result of the vehicle-mounted sensor according to the current position sensing deviation of the vehicle-mounted sensor.
In one embodiment of the present application, the calibration quality verification unit 430 is specifically configured to convert, according to the calibration parameter of the vehicle-mounted sensor, a true value position of a true value target belonging to the same roadside intelligent terminal in a current frame and a perceived position of a perceived target of the roadside intelligent terminal into the same coordinate system; acquiring the intra-frame position sensing deviation of the vehicle-mounted sensor in the current frame according to the positions of a true value target belonging to the same road side intelligent terminal and a road side intelligent terminal sensing target in the current frame under the same coordinate system; performing deviation statistics on the intra-frame position sensing deviation of continuous M frames including the current frame to obtain an intra-frame position sensing deviation statistical result of the vehicle-mounted sensor in the current frame, wherein the intra-frame position sensing deviation statistical result comprises an average value and a variance value of the intra-frame position sensing deviation, and M is a positive integer larger than 1; and acquiring the current position sensing deviation of the vehicle-mounted sensor according to the intra-frame position sensing deviation statistical result of the vehicle-mounted sensor in the current frame.
In one embodiment of the present application, the calibration quality checking unit 430 is specifically configured to compare the variance value with a preset safety variance threshold, and if the variance value is greater than the safety variance threshold, determine that the calibration quality checking result of the vehicle-mounted sensor is a calibration quality alarm; and if the variance value is not greater than the safety variance threshold, comparing the average value with a preset safety average threshold, if the average value is greater than the safety average threshold, determining that the calibration quality check result of the vehicle-mounted sensor is not qualified in calibration quality, and if the average value is not greater than the safety average threshold, determining that the calibration quality check result of the vehicle-mounted sensor is qualified in calibration quality.
In one embodiment of the present application, the maintenance scheme determining unit 440 is specifically configured to determine that the maintenance scheme of the vehicle-mounted sensor is offline recalibration when the calibration quality check result of the vehicle-mounted sensor is a calibration quality alarm; when the calibration quality check result of the vehicle-mounted sensor is that the calibration quality is qualified, determining that the maintenance scheme of the vehicle-mounted sensor is continuous monitoring; and when the calibration quality check result of the vehicle-mounted sensor is that the calibration quality is unqualified, determining that the maintenance scheme of the vehicle-mounted sensor is online compensation.
In one embodiment of the present application, the maintenance scheme determining unit 440 is further configured to update the cumulative number of times of failed calibration quality when the calibration quality check result of the vehicle-mounted sensor is failed, and determine that the maintenance scheme of the vehicle-mounted sensor is offline recalibration if the updated cumulative number of times of failed calibration quality is greater than a preset frequency threshold; if the updated accumulated times of unqualified calibration quality are not greater than the times threshold, determining that the maintenance scheme of the vehicle-mounted sensor is online compensation, acquiring a position compensation value, generating a position compensation command according to the position compensation value, and controlling the vehicle-mounted sensor to perform position compensation on obstacle sensing information in sensing data of the next frame according to the position compensation command.
In one embodiment of the present application, the data obtaining unit 410 is specifically configured to parse a first data packet from a road side intelligent terminal and a second data packet from a vehicle-mounted sensor according to a preset multi-device collaborative information interaction protocol, so as to obtain a first parsing result corresponding to the first data packet and a second parsing result corresponding to the second data packet, where the first parsing result includes a first timestamp of the first data packet and information of the road side intelligent terminal, the information of the road side intelligent terminal includes an identifier of the road side intelligent terminal and a position and an attribute of the road side intelligent terminal, the second parsing result includes a second timestamp of the second data packet and more than one obstacle sensing information, and the obstacle sensing information includes an identifier of an obstacle and a sensing position and a sensing attribute of the obstacle; determining whether the first data packet is time-synchronized with the second data packet according to the first timestamp and the second timestamp; when the first data packet is time-synchronized with the second data packet, obtaining the true value data of the current frame according to the first analysis result, wherein the road side intelligent terminal is used as a true value target, and the position and the attribute of the road side intelligent terminal are used as the true value position and the true value attribute of the true value target; and obtaining the perception data of the current frame according to the second analysis result, wherein each obstacle is taken as a perception target, and the perception position and the perception attribute of each obstacle are taken as the perception position and the perception attribute of the perception target.
It can be understood that the above-mentioned calibration quality online verification device for the vehicle-mounted sensor can implement each step of the calibration quality online verification method for the vehicle-mounted sensor provided in the foregoing embodiment, and the relevant explanation about the calibration quality online verification method for the vehicle-mounted sensor is applicable to the calibration quality online verification device for the vehicle-mounted sensor, which is not repeated herein.
Fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present application. Referring to fig. 5, at the hardware level, the electronic device includes a processor, and optionally an internal bus, a network interface, and a memory. The Memory may include a Memory, such as a Random-Access Memory (RAM), and may further include a non-volatile Memory (non-volatile Memory), such as at least 1 disk Memory. Of course, the electronic device may also include hardware required for other services.
The processor, network interface, and memory may be interconnected by an internal bus, which may be an ISA (Industry Standard Architecture ) bus, a PCI (Peripheral Component Interconnect, peripheral component interconnect standard) bus, or EISA (Extended Industry Standard Architecture ) bus, among others. The buses may be classified as address buses, data buses, control buses, etc. For ease of illustration, only one bi-directional arrow is shown in FIG. 5, but not only one bus or type of bus.
And the memory is used for storing programs. In particular, the program may include program code including computer-operating instructions. The memory may include memory and non-volatile storage and provide instructions and data to the processor.
The processor reads the corresponding computer program from the nonvolatile memory to the memory and then operates the computer program to form the calibration quality on-line calibration device of the vehicle-mounted sensor on a logic level. The processor is used for executing the programs stored in the memory and is specifically used for executing the following operations:
obtaining perception data and true value data of a current frame, wherein the perception data is obtained according to the perception information of a vehicle-mounted sensor on an obstacle, and the perception data comprises more than one perception target and the perception position and the perception attribute of each perception target; the truth data is obtained through information of the intelligent road side terminal sent by the intelligent road side terminal, and the truth data comprises a truth target, and a truth position and a truth attribute of the truth target;
determining whether a road side intelligent terminal sensing target belonging to the same road side intelligent terminal as the true value target of the current frame exists in the sensing target of the current frame according to the sensing data and the true value data of the current frame;
When a road side intelligent terminal sensing target belonging to the same road side intelligent terminal as a true value target of the current frame exists in a sensing target of the current frame, checking the calibration quality of the vehicle-mounted sensor according to the true value target belonging to the same road side intelligent terminal in the current frame and the road side intelligent terminal sensing target to obtain a calibration quality checking result;
and determining a maintenance scheme of the vehicle-mounted sensor according to the calibration quality check result so as to maintain the vehicle-mounted sensor according to the maintenance scheme of the vehicle-mounted sensor.
The method executed by the on-line calibration quality verification device of the vehicle-mounted sensor disclosed in the embodiment shown in fig. 1 of the present application can be applied to a processor or implemented by the processor. The processor may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware in a processor or by instructions in the form of software. The processor may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU), a network processor (Network Processor, NP), etc.; but also digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), field programmable gate arrays (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The disclosed methods, steps, and logic blocks in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of a method disclosed in connection with the embodiments of the present application may be embodied directly in hardware, in a decoded processor, or in a combination of hardware and software modules in a decoded processor. The software modules may be located in a random access memory, flash memory, read only memory, programmable read only memory, or electrically erasable programmable memory, registers, etc. as well known in the art. The storage medium is positioned in the memory, the processor reads the information in the memory, and the steps of the calibration quality on-line verification method of the vehicle-mounted sensor are completed by combining hardware of the processor.
The electronic device may further execute the method executed by the calibration quality online verification device of the vehicle-mounted sensor in fig. 1, and implement the function of the calibration quality online verification device of the vehicle-mounted sensor in the embodiment shown in fig. 1, which is not described herein.
The embodiments of the present application also provide a computer readable storage medium storing one or more programs, where the one or more programs include instructions, which when executed by an electronic device including a plurality of application programs, enable the electronic device to perform a method performed by the calibration quality online verification apparatus of an in-vehicle sensor in the embodiment shown in fig. 1, and specifically are configured to perform the following operations:
obtaining perception data and true value data of a current frame, wherein the perception data is obtained according to the perception information of a vehicle-mounted sensor on an obstacle, and the perception data comprises more than one perception target and the perception position and the perception attribute of each perception target; the truth data is obtained through information of the intelligent road side terminal sent by the intelligent road side terminal, and the truth data comprises a truth target, and a truth position and a truth attribute of the truth target;
Determining whether a road side intelligent terminal sensing target belonging to the same road side intelligent terminal as the true value target of the current frame exists in the sensing target of the current frame according to the sensing data and the true value data of the current frame;
when a road side intelligent terminal sensing target belonging to the same road side intelligent terminal as a true value target of the current frame exists in a sensing target of the current frame, checking the calibration quality of the vehicle-mounted sensor according to the true value target belonging to the same road side intelligent terminal in the current frame and the road side intelligent terminal sensing target to obtain a calibration quality checking result;
and determining a maintenance scheme of the vehicle-mounted sensor according to the calibration quality check result so as to maintain the vehicle-mounted sensor according to the maintenance scheme of the vehicle-mounted sensor.
It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In one typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include volatile memory in a computer-readable medium, random Access Memory (RAM) and/or nonvolatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of computer-readable media.
Computer readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. Computer-readable media, as defined herein, does not include transitory computer-readable media (transmission media), such as modulated data signals and carrier waves.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises the element.
It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The foregoing is merely exemplary of the present application and is not intended to limit the present application. Various modifications and changes may be made to the present application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc. which are within the spirit and principles of the present application are intended to be included within the scope of the claims of the present application.

Claims (11)

1. The on-line calibration quality verification method for the vehicle-mounted sensor is characterized by comprising the following steps of:
obtaining perception data and true value data of a current frame, wherein the perception data is obtained according to the perception information of a vehicle-mounted sensor on an obstacle, and the perception data comprises more than one perception target and the perception position and the perception attribute of each perception target; the truth data is obtained through information of the intelligent road side terminal sent by the intelligent road side terminal, and the truth data comprises a truth target, and a truth position and a truth attribute of the truth target;
determining whether a road side intelligent terminal sensing target belonging to the same road side intelligent terminal as the true value target of the current frame exists in the sensing target of the current frame according to the sensing data and the true value data of the current frame;
when a road side intelligent terminal sensing target belonging to the same road side intelligent terminal as a true value target of the current frame exists in a sensing target of the current frame, checking the calibration quality of the vehicle-mounted sensor according to the true value target belonging to the same road side intelligent terminal in the current frame and the road side intelligent terminal sensing target to obtain a calibration quality checking result;
And determining a maintenance scheme of the vehicle-mounted sensor according to the calibration quality check result so as to maintain the vehicle-mounted sensor according to the maintenance scheme of the vehicle-mounted sensor.
2. The method for online calibration quality verification according to claim 1, wherein the truth data of the current frame includes one or more frames, each frame of truth data includes different truth targets, and determining whether there is a roadside intelligent terminal sensing target belonging to the same roadside intelligent terminal as the truth target of the current frame in the sensing targets of the current frame according to the sensing data and the truth data of the current frame includes:
acquiring multidimensional distance features of each truth value target in the current frame and each perception target in the current frame according to the perception position and the perception attribute of each perception target in the current frame and according to the truth value position and the truth value attribute of each truth value target in the current frame;
according to the multidimensional distance characteristics of each truth value target in the current frame and each perception target in the current frame, obtaining the similarity of each truth value target in the current frame and each perception target in the current frame;
and determining whether a road side intelligent terminal perception target belonging to the same road side intelligent terminal as the true value target of the current frame exists in the perception targets of the current frame according to the similarity between each true value target in the current frame and each perception target in the current frame.
3. The method for online calibration quality verification according to claim 2, wherein determining whether there is a roadside intelligent terminal sensing target belonging to the same roadside intelligent terminal as the truth target of the current frame in the sensing targets of the current frame according to the similarity between each truth target of the current frame and each sensing target of the current frame comprises:
determining a similarity maximum value from the similarity between the true value target in the current frame and each perception target of the current frame, and comparing the similarity maximum value with a preset similarity threshold;
if the maximum value of the similarity is larger than the similarity threshold value, determining that a perception target corresponding to the maximum value of the similarity and the true value target of the current frame belong to the same intelligent terminal at one road side;
and if the maximum value of the similarity is not greater than the similarity threshold value, determining that the sensing target belonging to the same road side intelligent terminal as the true target of the current frame does not exist in the current frame.
4. The method for on-line calibration quality verification according to claim 1, wherein the verifying the calibration quality of the vehicle-mounted sensor according to the true value target belonging to the same roadside intelligent terminal and the roadside intelligent terminal perceived target in the current frame to obtain the calibration quality verification result comprises:
Acquiring the current position sensing deviation of the vehicle-mounted sensor according to the true value position of the true value target belonging to the same road side intelligent terminal in the current frame and the sensing position of the road side intelligent terminal sensing target;
and acquiring a calibration quality check result of the vehicle-mounted sensor according to the current position sensing deviation of the vehicle-mounted sensor.
5. The method for online calibration of calibration quality according to claim 4, wherein the obtaining the current position sensing deviation of the vehicle-mounted sensor according to the true position of the true value target belonging to the same intelligent terminal on the same road side and the sensing position of the intelligent terminal sensing target on the road side in the current frame comprises:
converting the true value position of the true value target belonging to the same road side intelligent terminal in the current frame and the perception position of the road side intelligent terminal perception target into the same coordinate system according to the calibration parameters of the vehicle-mounted sensor;
acquiring the intra-frame position sensing deviation of the vehicle-mounted sensor in the current frame according to the positions of a true value target belonging to the same road side intelligent terminal and a road side intelligent terminal sensing target in the current frame under the same coordinate system;
performing deviation statistics on the intra-frame position sensing deviation of continuous M frames including the current frame to obtain an intra-frame position sensing deviation statistical result of the vehicle-mounted sensor in the current frame, wherein the intra-frame position sensing deviation statistical result comprises an average value and a variance value of the intra-frame position sensing deviation, and M is a positive integer larger than 1;
And acquiring the current position sensing deviation of the vehicle-mounted sensor according to the intra-frame position sensing deviation statistical result of the vehicle-mounted sensor in the current frame.
6. The method for on-line calibration quality verification according to claim 5, wherein the obtaining the calibration quality verification result of the on-vehicle sensor according to the current position sensing deviation of the on-vehicle sensor comprises:
comparing the variance value with a preset safety variance threshold, and if the variance value is larger than the safety variance threshold, determining that the calibration quality check result of the vehicle-mounted sensor is a calibration quality alarm;
and if the variance value is not greater than the safety variance threshold, comparing the average value with a preset safety average threshold, if the average value is greater than the safety average threshold, determining that the calibration quality check result of the vehicle-mounted sensor is not qualified in calibration quality, and if the average value is not greater than the safety average threshold, determining that the calibration quality check result of the vehicle-mounted sensor is qualified in calibration quality.
7. The method for on-line calibration quality verification according to claim 1, wherein the calibration quality verification result includes a calibration quality alarm, a calibration quality failure, and a calibration quality qualification, and the determining the maintenance scheme of the vehicle-mounted sensor according to the calibration quality verification result includes:
When the calibration quality check result of the vehicle-mounted sensor is a calibration quality alarm, determining that the maintenance scheme of the vehicle-mounted sensor is off-line recalibration;
when the calibration quality check result of the vehicle-mounted sensor is that the calibration quality is qualified, determining that the maintenance scheme of the vehicle-mounted sensor is continuous monitoring;
and when the calibration quality check result of the vehicle-mounted sensor is that the calibration quality is unqualified, determining that the maintenance scheme of the vehicle-mounted sensor is online compensation.
8. The method for on-line calibration quality verification according to claim 7, wherein when the calibration quality verification result of the vehicle-mounted sensor is that the calibration quality is not acceptable, the method further comprises:
updating the accumulated times of unqualified calibration quality, and if the updated accumulated times of unqualified calibration quality is greater than a preset time threshold, determining that the maintenance scheme of the vehicle-mounted sensor is off-line recalibration;
if the updated accumulated times of unqualified calibration quality are not greater than the times threshold, determining that the maintenance scheme of the vehicle-mounted sensor is online compensation, acquiring a position compensation value, generating a position compensation command according to the position compensation value, and controlling the vehicle-mounted sensor to perform position compensation on the sensing position of each sensing target in the sensing data of the next frame according to the position compensation command.
9. The method for online verification of calibration quality according to any one of claims 1 to 8, wherein the obtaining the perceived data and the true value data of the current frame includes:
analyzing a first data packet from a road side intelligent terminal and a second data packet from a vehicle-mounted sensor according to a preset multi-device cooperative information interaction protocol to obtain a first analysis result corresponding to the first data packet and a second analysis result corresponding to the second data packet, wherein the first analysis result comprises a first timestamp of the first data packet and road side intelligent terminal information, the road side intelligent terminal information comprises an identifier of the road side intelligent terminal and the position and attribute of the road side intelligent terminal, the second analysis result comprises a second timestamp of the second data packet and more than one obstacle perception information, and the obstacle perception information comprises the identifier of an obstacle and the perception position and perception attribute of the obstacle;
determining whether the first data packet is time-synchronized with the second data packet according to the first timestamp and the second timestamp;
when the first data packet is time-synchronized with the second data packet, obtaining the true value data of the current frame according to the first analysis result, wherein the road side intelligent terminal is used as a true value target, and the position and the attribute of the road side intelligent terminal are used as the true value position and the true value attribute of the true value target; and obtaining the perception data of the current frame according to the second analysis result, wherein each obstacle is taken as a perception target, and the perception position and the perception attribute of each obstacle are taken as the perception position and the perception attribute of the perception target.
10. An on-line calibration quality verification device for a vehicle-mounted sensor, which is characterized by comprising:
the data acquisition unit is used for acquiring the perception data and the true value data of the current frame, wherein the perception data are acquired according to the perception information of the vehicle-mounted sensor on the obstacle, and the perception data comprise more than one perception target and the perception position and the perception attribute of each perception target; the truth data is obtained through information of the intelligent road side terminal sent by the intelligent road side terminal, and the truth data comprises a truth target, and a truth position and a truth attribute of the truth target;
the target matching unit is used for determining whether a road side intelligent terminal sensing target belonging to the same road side intelligent terminal as the true value target of the current frame exists in the sensing target of the current frame according to the sensing data and the true value data of the current frame;
the calibration quality verification unit is used for verifying the calibration quality of the vehicle-mounted sensor according to the true value target belonging to the same road side intelligent terminal in the current frame and the road side intelligent terminal sensing target when the road side intelligent terminal sensing target belonging to the same road side intelligent terminal as the true value target of the current frame exists in the sensing target of the current frame, so as to obtain a calibration quality verification result;
And the maintenance scheme determining unit is used for determining the maintenance scheme of the vehicle-mounted sensor according to the calibration quality check result so as to maintain the vehicle-mounted sensor according to the maintenance scheme of the vehicle-mounted sensor.
11. An electronic device, comprising:
a processor; and
a memory arranged to store computer executable instructions which, when executed, cause the processor to perform the calibration quality on-line calibration method of an in-vehicle sensor as claimed in any one of claims 1 to 9.
CN202311195999.5A 2023-09-15 2023-09-15 Calibration quality online verification method and device for vehicle-mounted sensor and electronic equipment Pending CN117268450A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311195999.5A CN117268450A (en) 2023-09-15 2023-09-15 Calibration quality online verification method and device for vehicle-mounted sensor and electronic equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311195999.5A CN117268450A (en) 2023-09-15 2023-09-15 Calibration quality online verification method and device for vehicle-mounted sensor and electronic equipment

Publications (1)

Publication Number Publication Date
CN117268450A true CN117268450A (en) 2023-12-22

Family

ID=89213585

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311195999.5A Pending CN117268450A (en) 2023-09-15 2023-09-15 Calibration quality online verification method and device for vehicle-mounted sensor and electronic equipment

Country Status (1)

Country Link
CN (1) CN117268450A (en)

Similar Documents

Publication Publication Date Title
CN111127931B (en) Vehicle road cloud cooperation method, device and system for intelligent networked automobile
CN113223317B (en) Method, device and equipment for updating map
US20200391766A1 (en) Multi-mode communication method for autonomous transport system of mining vehicle and apparatus thereof
CN113721621B (en) Vehicle control method, device, electronic equipment and storage medium
KR102587090B1 (en) Apparatus, system and method for recognizing object of vehicle
CN111752825A (en) Plug-and-play software platform for intelligent electric vehicle and detection method thereof
US11889341B2 (en) Method and apparatus for managing roadside device in vehicle road cooperation, and cloud control platform system
CN116534059B (en) Adaptive perception path decision method, device, computer equipment and storage medium
CN113985854B (en) Data processing method and device and electronic equipment
CN117268450A (en) Calibration quality online verification method and device for vehicle-mounted sensor and electronic equipment
EP4280573A1 (en) Method for returning data on vehicle, controller on vehicle, cloud server, and vehicle
CN115240453B (en) Driving control method, device and system for automatic driving vehicle and electronic equipment
CN115294771B (en) Monitoring method and device for road side equipment, electronic equipment and storage medium
CN116148821A (en) Laser radar external parameter correction method and device, electronic equipment and storage medium
US11490029B2 (en) Vehicle vision LED flicker interference mitigation system
CN111277953B (en) Communication network stability processing method and system and electronic equipment
CN115371719B (en) Parameter calibration method and device for detection equipment, storage medium and electronic device
CN112565364B (en) Cross-chain cooperation method and device based on highway path recognition result
US20200387189A1 (en) Synchronizing sensing systems
US11796345B2 (en) Method and system for optimized notification of detected event on vehicles
WO2024032348A1 (en) Automatic-driving perception method and system, and controller and computer-readable storage medium
US20210061285A1 (en) Method for generating a reference representation
WO2023103143A1 (en) Sensor inspection method and apparatus, electronic device, and readable storage medium
US20220189294A1 (en) Data integrity verification and misbehavior detection and reporting of connected vehicles through smart infrastructure
CN117278657A (en) Data processing method, data transmitting device, computer equipment and medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination