CN115235525B - Sensor detection method, sensor detection device, electronic equipment and readable storage medium - Google Patents
Sensor detection method, sensor detection device, electronic equipment and readable storage medium Download PDFInfo
- Publication number
- CN115235525B CN115235525B CN202111486395.7A CN202111486395A CN115235525B CN 115235525 B CN115235525 B CN 115235525B CN 202111486395 A CN202111486395 A CN 202111486395A CN 115235525 B CN115235525 B CN 115235525B
- Authority
- CN
- China
- Prior art keywords
- point cloud
- cloud data
- sensor
- calibration
- coordinate system
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01D—MEASURING NOT SPECIALLY ADAPTED FOR A SPECIFIC VARIABLE; ARRANGEMENTS FOR MEASURING TWO OR MORE VARIABLES NOT COVERED IN A SINGLE OTHER SUBCLASS; TARIFF METERING APPARATUS; MEASURING OR TESTING NOT OTHERWISE PROVIDED FOR
- G01D18/00—Testing or calibrating apparatus or arrangements provided for in groups G01D1/00 - G01D15/00
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Traffic Control Systems (AREA)
Abstract
The present disclosure provides a method, apparatus and readable storage medium for detecting sensor calibration anomalies, the method comprising: respectively acquiring first point cloud data of a plurality of sensors arranged on a vehicle under an initial coordinate system; based on a first conversion relation, respectively converting first point cloud data of the sensor into a vehicle coordinate system to obtain second point cloud data of the sensor, wherein the first conversion relation is obtained according to an initial setting pose of the sensor on the vehicle; combining second point cloud data of at least two sensors in the plurality of sensors to obtain combined point cloud data; and determining the calibration state of the sensor according to the second point cloud data of the sensor and the merging point cloud data. Through the technical scheme provided by the disclosure, under the condition that calibration abnormality occurs to the sensor, the abnormality can be detected in time, so that operation and maintenance personnel can process the abnormality conveniently, and the safety of the vehicle in operation is improved.
Description
Technical Field
The disclosure relates to the technical field of intelligent driving, and in particular relates to a sensor detection method, a sensor detection device, electronic equipment and a readable storage medium.
Background
The environment perception is a key part of the intelligent driving system, is the front-end input of planning decisions, and provides important basis for the planning decisions. The environment sensing module includes various sensors, such as a laser radar, a camera, a millimeter wave radar, etc., through which the vehicle senses the surrounding environment and then controls the travel of the autonomous vehicle according to the sensed environmental information. Because the detection range of a single sensor is limited, a plurality of sensors are arranged on the vehicle to reduce detection blind areas, and the point cloud coordinate systems respectively generated by the sensors are converted into a unified coordinate system through calibration. And when the relative position of the sensor and the vehicle changes, the point cloud merging result is abnormal. In the related art, the operation and maintenance personnel mainly conduct detection and investigation of calibration abnormality of the sensor in an off-line state when the vehicle finishes running, so that the calibration state of the sensor cannot be obtained in real time. And the calibration state of the sensor cannot be known in time in the running process of the vehicle, so that the vehicle can continue to run under the condition of abnormal calibration of the sensor, and safety risks exist.
Disclosure of Invention
In view of the above, the present disclosure provides a sensor detection method, a device, an electronic apparatus, and a readable storage medium, so as to realize automatic detection of a calibration state of a sensor by a vehicle, shorten a detection feedback time, and improve safety of the vehicle during running.
According to a first aspect of the present disclosure, there is provided a sensor detection method, the method comprising:
respectively acquiring first point cloud data of a plurality of sensors arranged on a vehicle under an initial coordinate system;
based on a first conversion relation, respectively converting first point cloud data of the sensor into a vehicle coordinate system to obtain second point cloud data of the sensor, wherein the first conversion relation is obtained according to an initial setting pose of the sensor on the vehicle;
combining second point cloud data of at least two sensors in the plurality of sensors to obtain combined point cloud data;
and determining the calibration state of the sensor according to the second point cloud data and the merging point cloud data.
In combination with any one of the embodiments provided in the present disclosure, the converting, based on the first conversion relation, the first point cloud data of the sensor to a vehicle coordinate system includes:
the first point cloud data of the sensor is respectively converted to a vehicle coordinate system based on a six-degree-of-freedom conversion matrix converted from an initial coordinate system of the sensor to the vehicle coordinate system, wherein the six-degree-of-freedom conversion matrix comprises a rotation matrix and a translation matrix.
In combination with any one of the embodiments provided in the present disclosure, the determining the calibration state of the sensor according to the first point cloud data, the second point cloud data, and the combined point cloud data of the sensor includes:
registering the first point cloud data of the first sensor with the merging point cloud data to obtain a second conversion relation, wherein the first sensor is any one of the plurality of sensors;
converting the first point cloud data of the first sensor into a coordinate system of a merging point cloud based on the second conversion relation to obtain third point cloud data of the first sensor, wherein the coordinate system of the merging point cloud is obtained by fitting the merging point cloud data;
and comparing the second point cloud data with the third point cloud data of the first sensor, and determining the calibration state of the first sensor according to the comparison result.
In combination with any one of the embodiments provided in the present disclosure, the comparing the second point cloud data and the third point cloud data of the first sensor, and determining the calibration state of the sensor according to the comparison result includes:
determining that the first sensor calibration is abnormal under the condition that the average position offset distance of the corresponding point coordinates in the second point cloud data and the third point cloud data exceeds a first set threshold value;
And under the condition that the average position offset distance of the corresponding point coordinates in the second point cloud data and the third point cloud data does not exceed a first set threshold value, determining that the first sensor is calibrated normally.
In combination with any one of the embodiments provided in the present disclosure, the determining, according to the second point cloud data and the combined point cloud data, the calibration state of the sensor includes:
registering the first point cloud data of the first sensor and the merging point cloud data to obtain a second conversion relation;
determining that the first sensor calibration is abnormal under the condition that the parameter deviation in the first conversion relation and the second conversion relation exceeds a second set threshold value;
and under the condition that the parameter deviation in the first conversion relation and the second conversion relation does not exceed a second set threshold value, determining that the first sensor is calibrated normally.
In combination with any one of the embodiments provided in the present disclosure, the merging the second point cloud data of at least two sensors in the plurality of sensors to obtain merged point cloud data includes:
and merging the second point cloud data of the other sensors except the first sensor in the plurality of sensors to obtain merged point cloud data.
In connection with any one of the embodiments provided by the present disclosure, the method further comprises:
carrying out multi-frame point cloud data combination on the combination point cloud data to obtain multi-frame combination point cloud data;
based on the multi-frame merging point cloud data, acquiring relative displacement of the vehicle to obtain the moving direction of the vehicle;
and comparing the positive direction orientation of the coordinate system of the merging point cloud with the moving direction of the vehicle, and determining the calibration state of the sensor according to the comparison result.
In combination with any one of the embodiments provided in the present disclosure, the comparing the positive direction of the coordinate system of the merging point cloud with the moving direction of the vehicle, and determining the calibration state of the sensor according to the comparison result includes:
determining that at least one sensor of the plurality of sensors is abnormal in calibration under the condition that the angle of the included angle between the two directions exceeds a first angle threshold;
and under the condition that the angle of the included angle between the two directions does not exceed a first angle threshold, determining that the calibration of the plurality of sensors is normal.
In connection with any one of the embodiments provided by the present disclosure, the method further comprises:
performing ground segmentation processing on the merging point cloud data to obtain ground segmentation point cloud data;
and comparing the merging point cloud data with the ground division point cloud data, and determining the calibration state of the sensor according to a comparison result.
In combination with any one of the embodiments provided in the present disclosure, the comparing the merging point cloud data with the ground segmentation point cloud data, and determining the calibration state of the sensor according to the comparison result includes:
determining that at least one sensor of the plurality of sensors is abnormal in calibration under the condition that the angle between the positive direction of the coordinate system of the merging point cloud data and the ground included angle acquired by the ground division point cloud exceeds a second angle threshold;
and under the condition that the angle between the positive direction of the coordinate system of the merging point cloud data and the ground included angle acquired by the ground division point cloud does not exceed a second angle threshold value, determining that the plurality of sensors are calibrated normally.
In connection with any one of the embodiments provided by the present disclosure, the method further comprises:
and under the condition that the sensor calibration abnormality is determined, generating a sensor calibration abnormality report, and sending the abnormality report to an operation and maintenance terminal.
In connection with any one of the embodiments provided by the present disclosure, the method further comprises:
and generating a sensor calibration detection report according to the calibration state of the sensor at set time intervals, and sending the calibration detection report to an operation and maintenance terminal.
According to a second aspect of the present disclosure, there is provided a sensor detection apparatus, the apparatus comprising:
the first point cloud acquisition module is used for respectively acquiring first point cloud data of a plurality of sensors arranged on the vehicle under an initial coordinate system;
the second point cloud generation module is used for respectively converting the first point cloud data of the sensor into a vehicle coordinate system based on a first conversion relation to obtain second point cloud data of the sensor, wherein the first conversion relation is obtained according to the initial setting pose of the sensor on the vehicle;
the merging point cloud generation module is used for merging second point cloud data of at least two sensors in the plurality of sensors to obtain merging point cloud data;
and the calibration detection module is used for determining the calibration state of the sensor according to the second point cloud data and the merging point cloud data.
In combination with any one of the embodiments provided in the present disclosure, the second point cloud generating module is configured to convert, based on a first conversion relationship, first point cloud data of the sensor to a vehicle coordinate system, respectively, and specifically configured to:
the first point cloud data of the sensor is respectively converted to a vehicle coordinate system based on a six-degree-of-freedom conversion matrix converted from an initial coordinate system of the sensor to the vehicle coordinate system, wherein the six-degree-of-freedom conversion matrix comprises a rotation matrix and a translation matrix.
In combination with any one of the embodiments provided in the present disclosure, the calibration detection module determines a calibration state of the sensor according to the first point cloud data, the second point cloud data, and the combined point cloud data of the sensor, and is specifically configured to:
registering the first point cloud data of the first sensor with the merging point cloud data to obtain a second conversion relation, wherein the first sensor is any one of the plurality of sensors;
converting the first point cloud data of the first sensor into a coordinate system of a merging point cloud based on the second conversion relation to obtain third point cloud data of the first sensor, wherein the coordinate system of the merging point cloud is obtained by fitting the merging point cloud data;
and comparing the second point cloud data with the third point cloud data of the first sensor, and determining the calibration state of the first sensor according to the comparison result.
In combination with any one of the embodiments provided in the present disclosure, the calibration detection module compares the second point cloud data and the third point cloud data of the first sensor, and determines a calibration state of the sensor according to a comparison result, including:
determining that the first sensor calibration is abnormal under the condition that the average position offset distance of the corresponding point coordinates in the second point cloud data and the third point cloud data exceeds a first set threshold value;
And under the condition that the average position offset distance of the corresponding point coordinates in the second point cloud data and the third point cloud data does not exceed a first set threshold value, determining that the first sensor is calibrated normally.
In combination with any one of the real-time modes provided by the disclosure, the calibration detection module determines a calibration state of the sensor according to the second point cloud data and the merging point cloud data, including:
registering the first point cloud data of the first sensor and the merging point cloud data to obtain a second conversion relation;
determining that the first sensor calibration is abnormal under the condition that the parameter deviation in the first conversion relation and the second conversion relation exceeds a second set threshold value;
and under the condition that the parameter deviation in the first conversion relation and the second conversion relation does not exceed a second set threshold value, determining that the first sensor is calibrated normally.
In combination with any one of the embodiments provided in the present disclosure, the merging point cloud generating module merges the second point cloud data of at least two sensors in the plurality of sensors to obtain merging point cloud data, which is specifically configured to:
and merging the second point cloud data of the other sensors except the first sensor in the plurality of sensors to obtain merged point cloud data.
In combination with any one of the embodiments provided in the present disclosure, the apparatus further includes a first angle comparison module configured to:
carrying out multi-frame point cloud data combination on the combination point cloud data to obtain multi-frame combination point cloud data;
based on the multi-frame merging point cloud data, acquiring relative displacement of the vehicle to obtain the moving direction of the vehicle;
and comparing the positive direction orientation of the coordinate system of the merging point cloud with the moving direction of the vehicle, and determining the calibration state of the sensor according to the comparison result.
In combination with any one of the embodiments provided in the present disclosure, the first angle comparing module compares the coordinate system positive direction of the merging point cloud with the vehicle moving direction, and determines, according to a comparison result, a calibration state of the sensor, where the first angle comparing module is specifically configured to:
determining that at least one sensor of the plurality of sensors is abnormal in calibration under the condition that the angle of the included angle between the two directions exceeds a first angle threshold;
and under the condition that the angle of the included angle between the two directions does not exceed a first angle threshold, determining that the calibration of the plurality of sensors is normal.
In combination with any one of the embodiments provided in the present disclosure, the apparatus further includes a second angle comparison module configured to:
Performing ground segmentation processing on the merging point cloud data to obtain ground segmentation point cloud data;
and comparing the merging point cloud data with the ground division point cloud data, and determining the calibration state of the sensor according to a comparison result.
In combination with any one of the embodiments provided in the present disclosure, the second angle comparison module compares the merging point cloud data with the ground division point cloud data, and determines a calibration state of the sensor according to a comparison result, where the second angle comparison module is specifically configured to:
determining that at least one sensor of the plurality of sensors is abnormal in calibration under the condition that the angle between the positive direction of the coordinate system of the merging point cloud data and the ground included angle acquired by the ground division point cloud exceeds a second angle threshold;
and under the condition that the angle between the positive direction of the coordinate system of the merging point cloud data and the ground included angle acquired by the ground division point cloud does not exceed a second angle threshold value, determining that the plurality of sensors are calibrated normally.
In combination with any one of the embodiments provided in the present disclosure, the apparatus further includes an anomaly reporting module configured to:
and under the condition that the sensor calibration abnormality exists, generating a sensor calibration abnormality report, and sending the abnormality report to an operation and maintenance terminal.
In combination with any one of the embodiments provided in the present disclosure, the apparatus further includes a detection report module configured to:
and generating a sensor calibration detection report according to the calibration state of the sensor at set time intervals, and sending the detection report to an operation and maintenance terminal.
According to a third aspect of the present disclosure, there is provided an electronic device comprising:
a memory for storing processor-executable instructions;
a processor configured to execute executable instructions in the memory to implement the steps of the method of any of the embodiments of the first aspect described above.
According to a fourth aspect of the present disclosure, there is provided a computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of the method according to any of the embodiments of the first aspect described above.
According to a fifth aspect of the present disclosure, there is provided an intelligent vehicle comprising the above-described electronic device.
The technical scheme provided by the disclosure can comprise the following beneficial effects:
through the first point cloud data, the second point cloud data and the merging point cloud data of the sensor, the automatic detection of the sensor calibration state of the vehicle is realized, the sensor calibration abnormality detection can be carried out in real time in the vehicle running process, the sensor calibration abnormality detection time is shortened, the feedback process from the occurrence of the calibration abnormality to the detection of the calibration abnormality is simplified, the operation and maintenance personnel can process at any time conveniently, and the safety of the vehicle in running is improved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this disclosure, illustrate embodiments consistent with the disclosure and together with the disclosure, serve to explain the principles of the disclosure.
Fig. 1A is a flow chart of a sensor detection method according to an exemplary embodiment of the present disclosure.
Fig. 1B is a flow chart of another sensor detection method illustrated by the present disclosure according to an exemplary embodiment.
Fig. 2 is a flow chart of another sensor detection method illustrated by the present disclosure according to an exemplary embodiment.
Fig. 3 is a flow chart of another sensor detection method illustrated by the present disclosure according to an exemplary embodiment.
Fig. 4 is a schematic diagram of a sensor detection device according to an exemplary embodiment of the present disclosure.
Fig. 5 is a block diagram of an electronic device according to an exemplary embodiment of the present disclosure.
Detailed Description
Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary examples are not representative of all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present disclosure as detailed in the accompanying claims.
The terminology used in the present disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. As used in this disclosure and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any or all possible combinations of one or more of the associated listed items.
It should be understood that although the terms first, second, third, etc. may be used in this disclosure to describe various information, these information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present disclosure. The word "if" as used herein may be interpreted as "at … …" or "at … …" or "responsive to a determination", depending on the context.
Fig. 1A illustrates a flow chart of a sensor detection method according to an exemplary embodiment of the present disclosure.
In step S101, first point cloud data of a plurality of sensors provided on a vehicle in an initial coordinate system is acquired, respectively.
In order to reduce the detection dead zone, a plurality of sensors may be provided on the vehicle, and the sensors may include at least one of commonly used sensors such as a laser radar, a multi-camera, a millimeter wave radar, and the like.
First point cloud data of a plurality of sensors on a vehicle in an initial coordinate system is acquired through electronic equipment arranged in the vehicle. The initial coordinate system may be set according to the position of the sensor on the vehicle, for example, the position of the sensor on the vehicle is set as the origin of the initial coordinate system. The initial coordinate system may also be obtained from configuration parameters of the vehicle.
In step S102, based on the first conversion relation, the first point cloud data of the sensor are respectively converted into a vehicle coordinate system, so as to obtain second point cloud data of the sensor.
The first conversion relation is obtained according to the initial setting pose of the sensor on the vehicle and the setting information of the vehicle coordinate system. The vehicle coordinate system may be set as required, for example, a midpoint of a front axis or a rear axis of the vehicle may be set as an origin of the vehicle coordinate system, the midpoint of the front axis or the rear axis may be set forward along a vehicle body direction as an X-axis positive direction, and left along the front axis or the rear axis may be set as a Y-axis positive direction and up as a Z-axis positive direction.
Converting the first point cloud data into the second point cloud data based on a first conversion relation, wherein when a sensor is calibrated normally, for example, parameters in the first conversion relation are correct, and the position and the direction of the sensor are not deviated from an initial setting pose, a coordinate system of the second point cloud data obtained after conversion is consistent with a coordinate system of the vehicle; under the condition that the sensor calibration is abnormal, for example, parameter errors in the first conversion relation or the position and the direction of the sensor deviate from the initial setting pose, the coordinate system of the second point cloud data obtained after conversion is inconsistent with the coordinate system of the vehicle.
In step S103, second point cloud data of at least two sensors of the plurality of sensors are combined to obtain combined point cloud data.
In step S104, a calibration state of the sensor is determined according to the second point cloud data and the merging point cloud data.
And fitting a coordinate system of second point cloud data obtained by the plurality of sensors to obtain the combined point cloud coordinate system. Although the coordinate system of the second point cloud data acquired by the individual sensor deviates from the vehicle coordinate system in the case of abnormal sensor calibration, the coordinate system of the merged point cloud data approaches the vehicle coordinate system after the point cloud data merging and the coordinate system fitting, and the coordinate system of the merged point cloud data can be regarded as a standard coordinate system similar to the vehicle coordinate system.
By comparing the second point cloud data with the merged point cloud data approximating the vehicle coordinate system or other data processed by the merged point cloud data, it is possible to determine whether the coordinate system of the second point cloud data is the vehicle coordinate system, that is, to determine the calibration state of the sensor.
According to the method and the device, the automatic detection of the calibration state of the sensor by the vehicle is realized through the first point cloud data, the second point cloud data and the merging point cloud data of the sensor, the abnormal detection of the calibration of the sensor can be carried out in real time in the running process of the vehicle, the abnormal detection time of the calibration of the sensor is shortened, the feedback process from the abnormal calibration to the abnormal calibration is simplified, the operation and maintenance personnel can process at any time, and the safety of the vehicle in running is improved.
In an optional embodiment, the converting the first point cloud data of the sensor to the vehicle coordinate system based on the first conversion relation includes: the first point cloud data of the sensor is respectively converted to a vehicle coordinate system based on a six-degree-of-freedom conversion matrix converted from an initial coordinate system of the sensor to the vehicle coordinate system, wherein the six-degree-of-freedom conversion matrix comprises a rotation matrix and a translation matrix.
And converting the first point cloud data into a vehicle coordinate system through a first conversion relation to obtain the second point cloud data, wherein the first conversion relation is a conversion relation between the sensor coordinate system and the vehicle coordinate system and can be an affine matrix or a six-degree-of-freedom conversion matrix containing pose parameters of the sensor and the vehicle. In one example shown in the present disclosure, the first conversion relationship includes a six degree-of-freedom conversion matrix composed of a rotation matrix and a translation matrix converted from an initial coordinate system of the sensor to a vehicle coordinate system.
In the running process of the vehicle, the initial coordinate system of the first point cloud and the vehicle coordinate system are three-dimensional space rectangular coordinate systems, the first point cloud data are calibrated through a six-degree-of-freedom conversion matrix, and the first point cloud data of the sensor are converted into the vehicle coordinate system to obtain second point cloud data of the sensor.
Fig. 1B illustrates a flow chart of another sensor detection method illustrated by the present disclosure according to an exemplary embodiment.
In step S104-1, the first point cloud data of the first sensor and the merging point cloud data are registered to obtain a second conversion relationship, where the first sensor is any one of the plurality of sensors.
In the present disclosure, the registration may be a process of acquiring a conversion relationship converted from an original point cloud to a target point cloud. In one example, by registering the first point cloud data of the first sensor and the merged point cloud data, a second conversion relationship capable of converting the first point cloud data coordinate system to the merged point cloud data coordinate system is obtained.
In one example, the electronic device in the vehicle may take each sensor on the vehicle as the first sensor in turn to detect the calibration state of each sensor separately; the specific sensor on the vehicle can also be independently controlled as the first sensor so as to specifically detect the calibration state of the sensor.
In step S104-2, based on the second conversion relationship, the first point cloud data of the first sensor is converted into a coordinate system of a merging point cloud, so as to obtain third point cloud data of the first sensor, and the coordinate system of the merging point cloud is obtained by fitting the merging point cloud data.
Since the plurality of sensors are not abnormal in the same time scale under normal conditions, the deviation of the coordinate system obtained by fitting the merging point cloud data from the vehicle coordinate system is generally small, and can be approximated as a standard coordinate system close to the vehicle coordinate system. The coordinate system of the third point cloud data is consistent with the coordinate system of the merging point cloud data, and is obtained by fitting the merging point cloud data, so that the coordinate system can also be used as a standard coordinate system similar to the coordinate system of a vehicle.
And converting the first point cloud data of the first sensor into a coordinate system of a merging point cloud based on the second conversion relation obtained in the step, and obtaining third point cloud data of the first sensor.
In step S104-3, the second point cloud data of the first sensor is compared with the third point cloud data, and the calibration state of the sensor is determined according to the comparison result.
Since the coordinate system of the third point cloud data is identical to the coordinate system of the merging point cloud data, the coordinate system may be a standard coordinate system similar to the coordinate system of the vehicle. Therefore, by comparing the second point cloud data with the third point cloud data, it is possible to determine whether the coordinate system of the second point cloud data is the vehicle coordinate system, that is, to determine the calibration state of the first sensor.
The second point cloud data and the third point cloud data of the first sensor are obtained by conversion of the first sensor, so that the comparison processing is convenient, the sensor calibration abnormality caused by the change of the relative positions of the sensor and the vehicle is detected in real time, operation and maintenance personnel can process the sensor at any time, and the safety of the vehicle in operation is improved.
In an alternative embodiment, the comparing the second point cloud data of the first sensor with the third point cloud data, and determining the calibration state of the sensor according to the comparison result includes: determining that the first sensor calibration is abnormal under the condition that the average position offset distance of the corresponding point coordinates in the second point cloud data and the third point cloud data exceeds a first set threshold value; and under the condition that the average position offset distance of the corresponding point coordinates in the second point cloud data and the third point cloud data does not exceed a first set threshold value, determining that the first sensor is calibrated normally.
The occurrence of calibration anomalies in the sensor may be caused by two conditions, in one of which sensor calibration anomalies may be caused by a change in the relative position of the sensor to the vehicle. Under the condition that the relative position of the first sensor and the vehicle changes so that calibration abnormality occurs to the first sensor, the average position of the point coordinates corresponding to the first point cloud data and the second point cloud data obtained by the first sensor has an offset distance. Determining that the first sensor calibration is abnormal under the condition that the offset distance exceeds a second set threshold value; and under the condition that the offset distance does not exceed a second set threshold, determining that the first sensor is calibrated normally, wherein the second set threshold can be set according to actual requirements.
According to the method, the sensor calibration abnormality caused by the change of the relative positions of the sensor and the vehicle can be detected in real time, so that operation and maintenance personnel can process the sensor at any time, and the safety of the vehicle during operation is improved.
In an optional embodiment, the determining the calibration state of the sensor according to the second point cloud data and the merging point cloud data includes: registering the first point cloud data of the first sensor and the merging point cloud data to obtain a second conversion relation; determining that the first sensor calibration is abnormal under the condition that the parameter deviation in the first conversion relation and the second conversion relation exceeds a second set threshold value;
and under the condition that the parameter deviation in the first conversion relation and the second conversion relation does not exceed a second set threshold value, determining that the first sensor is calibrated normally.
The sensor calibration abnormality may also be caused by parameter errors in the first relationship. In one case, the parameters include parameters in a six-degree-of-freedom conversion matrix composed of a rotation matrix and a translation matrix converted from an initial coordinate system of the sensor to a vehicle coordinate system. Under the condition that parameter errors in the first conversion relation cause abnormal calibration of the sensor, parameter deviations in the first conversion relation and the second conversion relation exceed a second set threshold; and under the condition that the parameter deviation does not exceed a first set threshold value, calibrating the first sensor normally. The second setting threshold can be set according to actual requirements.
According to the method, the sensor calibration abnormality caused by parameter errors in the first conversion relation can be detected in real time, so that operation and maintenance personnel can process the sensor calibration abnormality at any time, and the safety of a vehicle in operation is improved.
In an optional embodiment, the merging the second point cloud data of at least two sensors of the plurality of sensors to obtain merged point cloud data includes: and merging the second point cloud data of the other sensors except the first sensor in the plurality of sensors to obtain merged point cloud data.
In one example, the vehicle includes N sensors, and after determining the first sensor to be detected, second point cloud data of other (N-1) sensors except the first sensor are combined to obtain combined point cloud data.
According to the method, under the condition that the calibration of the first sensor is abnormal, the rest of sensors except the first sensor are combined, so that the influence on the calibration effect of the combined point cloud data due to the calibration abnormality of the first sensor is avoided, and the calibration state detection effect is improved.
Fig. 2 illustrates a flow chart of another sensor detection method illustrated by the present disclosure according to an exemplary embodiment.
In step S201, multi-frame point cloud data is combined with the combined point cloud data to obtain multi-frame combined point cloud data.
And in a set time period, acquiring the merging point cloud data obtained by the sensor at set time intervals, so as to obtain multi-frame data of the merging point cloud, and carrying out point cloud fusion on the multi-frame data of the merging point cloud to obtain multi-frame merging point cloud data. The point cloud fusion of the multi-frame data of the point cloud is a means in the related art, and the disclosure is not repeated here.
In step S202, based on the multi-frame merging point cloud data, a vehicle relative displacement is obtained, and a vehicle moving direction is obtained.
And determining the displacement condition of the merging point cloud in continuous time, namely the displacement condition of the vehicle in the set time period, by the multi-frame merging point cloud data, and acquiring the moving direction of the vehicle from the displacement condition.
In step S203, the coordinate system positive direction orientation of the merging point cloud is compared with the vehicle moving direction, and the calibration state of the sensor is determined according to the comparison result.
Under the condition that the sensor calibration is normal, the coordinate system positive direction of the merging point cloud is consistent with the vehicle moving direction, and the calibration abnormality can cause an included angle between the coordinate system positive direction of the merging point cloud and the vehicle moving direction, so that by comparing the coordinate system positive direction of the merging point cloud with the vehicle moving direction, whether the calibration abnormality exists in the plurality of sensors for acquiring the merging point cloud or not, namely, the calibration state of the sensors can be determined.
According to the method, the calibration state of the sensor is determined based on the direction in the running process of the vehicle, and the detection effect of the calibration state is improved. Through the calibration state detection of the vehicle in running, operation and maintenance personnel can timely know whether the vehicle needs sensor calibration optimization or maintenance, the time from the occurrence of calibration abnormality to the detection of the calibration abnormality is shortened, the operation and maintenance personnel can timely process the calibration abnormality, and the safety of the vehicle in running is improved.
In an optional embodiment, the comparing the positive direction of the coordinate system of the merging point cloud with the moving direction of the vehicle, and determining the calibration state of the sensor according to the comparison result includes: determining that at least one sensor of the plurality of sensors is abnormal in calibration under the condition that the angle of the included angle between the two directions exceeds a first angle threshold; and under the condition that the angle of the included angle between the two directions does not exceed a first angle threshold, determining that the calibration of the plurality of sensors is normal.
And under the condition that at least one sensor of the plurality of sensors is abnormal in calibration, an included angle exists between the positive direction of the coordinate system of the merging point cloud and the moving direction of the vehicle. Determining that at least one sensor of the plurality of sensors is abnormal in calibration under the condition that the angle of the included angle between the two directions exceeds a first angle threshold; and under the condition that the angle of the included angles of the two directions does not exceed a first angle threshold, determining that the plurality of sensors are calibrated normally, wherein the first angle threshold can be set according to actual requirements.
According to the method, through comparing the positive direction of the merging point cloud coordinate system with the included angle of the moving direction of the vehicle, sensor calibration abnormality detection is carried out in real time in the running process of the vehicle, the time for detecting the sensor calibration abnormality is shortened, the feedback process from the occurrence of the calibration abnormality to the detection of the calibration abnormality is simplified, operation and maintenance personnel can process at any time conveniently, and the safety of the running process of the vehicle is improved.
Fig. 3 illustrates a flow chart of another sensor detection method illustrated by the present disclosure according to an exemplary embodiment.
In step S301, ground segmentation processing is performed on the merged point cloud data to obtain ground segmentation point cloud data.
The ground segmentation processing is a method for obtaining a ground segmentation point cloud, for example, including a planar grid method, a point cloud normal vector, a model fitting method, a surface element grid method, and the like, and the specific ground segmentation method is a common means in the related art, which is not described herein in detail. In the embodiment of the disclosure, the ground segmentation point cloud can be obtained through the ground segmentation method.
In step S302, the merging point cloud data and the ground division point cloud data are compared, and the calibration state of the sensor is determined according to the comparison result.
Under the condition that the calibration of the sensor is normal, the coordinate system positive direction of the merging point cloud is in a parallel relation with the ground acquired by the ground dividing point cloud, and the calibration anomaly can lead to the fact that an included angle exists between the coordinate system positive direction of the merging point cloud and the ground acquired by the ground dividing point cloud, so that whether the calibration anomaly exists in the plurality of sensors acquiring the merging point cloud or not, namely the calibration state of the sensor can be determined by comparing the coordinate system positive direction of the merging point cloud with the included angle between the coordinate system positive direction of the merging point cloud and the ground.
By comparing the merging point cloud data with the ground partition point cloud data, whether the calibration abnormality exists in any sensor among the plurality of sensors for acquiring the merging point cloud can be determined in real time, namely, the calibration state of the sensor is determined. The sensor calibration abnormality detection time is shortened, the feedback process from the occurrence of calibration abnormality to the detection of the calibration abnormality is simplified, operation and maintenance personnel can process at any time, and the safety of the vehicle during operation is improved.
According to the method, the calibration state of the sensor is determined based on the ground state, and the detection effect of the calibration state is improved. Through the calibration state detection based on the ground, an operation and maintenance person can timely know whether the vehicle needs sensor calibration optimization or maintenance, the feedback time from the occurrence of calibration abnormality to the detection of the calibration abnormality is shortened, the operation and maintenance person can timely process the calibration abnormality, and the safety of the vehicle during operation is improved.
In an optional embodiment, the comparing the merging point cloud data with the ground segmentation point cloud data, and determining the calibration state of the sensor according to the comparison result includes: determining that at least one sensor of the plurality of sensors is abnormal in calibration under the condition that the angle between the positive direction of the coordinate system of the merging point cloud data and the ground included angle acquired by the ground division point cloud exceeds a second angle threshold; and under the condition that the angle between the positive direction of the coordinate system of the merging point cloud data and the ground included angle acquired by the ground division point cloud does not exceed a second angle threshold value, determining that the plurality of sensors are calibrated normally.
Determining that at least one sensor of the plurality of sensors is abnormal in calibration under the condition that the angle between the positive direction of the coordinate system of the merging point cloud data and the included angle between the ground exceeds a second angle threshold; and under the condition that the angle between the positive direction of the coordinate system of the merging point cloud data and the ground included angle does not exceed a second angle threshold, determining that the plurality of sensors are calibrated normally, wherein the second angle threshold can be set according to actual requirements.
According to the method, the positive direction of the merging point cloud coordinate system is compared with the included angle between the ground and the ground obtained by the ground division point cloud, and the sensor calibration abnormality detection is carried out in real time in the running process of the vehicle, so that the time for detecting the sensor calibration abnormality is shortened, the feedback process from the occurrence of the calibration abnormality to the detection of the calibration abnormality is simplified, the operation and maintenance personnel can process the operation and maintenance personnel at any time conveniently, and the safety of the running process of the vehicle is improved.
In an alternative embodiment, the calibration state of the sensor may also be determined directly by the ground segmentation process.
Under the condition that the ground is horizontal and flat, performing ground segmentation processing on first point cloud data acquired by any two sensors arranged on a vehicle respectively, and under the condition that the height coordinates of the ground segmentation point cloud data acquired by the any two sensors respectively exceed a third set threshold, determining that at least one sensor in the sensors is abnormal in calibration; and determining that the sensor calibration is normal under the condition that the ground partition point cloud data height coordinates respectively acquired by any two sensors of the vehicle do not exceed a third set threshold value.
According to the method, the ground division point cloud data height coordinates obtained by any two sensors are compared, the sensor calibration abnormality detection is carried out in real time in the vehicle running process, the sensor calibration abnormality detection time is shortened, the feedback process from the occurrence of the calibration abnormality to the detection of the calibration abnormality is simplified, the operation and maintenance personnel can process at any time conveniently, and the safety of the vehicle in running is improved.
In an alternative embodiment, the method further comprises: and under the condition that the sensor calibration abnormality is determined, generating a sensor calibration abnormality report, and sending the abnormality report to an operation and maintenance terminal. The anomaly report may include all the point cloud data acquired by the sensors acquired by the above method, and other vehicle operation-related data.
Under the condition that the calibration abnormality of a specific sensor in the vehicle can be determined, an operation and maintenance person can carry out maintenance treatment on the sensor with the calibration abnormality, and under the condition that the calibration abnormality exists in at least one sensor in the plurality of sensors in the vehicle, the operation and maintenance person can carry out integral optimization or maintenance on the plurality of sensors. In addition, under the condition that potential safety hazards exist in the vehicle due to abnormal sensor calibration, operation and maintenance personnel can timely inform the vehicle user of current risks.
According to the method, under the condition that the sensor is abnormal in calibration, the abnormality can be detected in time, the feedback time from the occurrence of the calibration abnormality to the detection of the calibration abnormality is shortened, so that operation and maintenance personnel can process in time, and the safety of a vehicle in running is improved.
In an alternative embodiment, the method further comprises: and generating a sensor calibration detection report according to the calibration state of the sensor at set time intervals, and sending the calibration detection report to an operation and maintenance terminal.
The set time interval can take daily or weekly as set time, a sensor calibration detection report is generated according to the calibration state of the sensor, and the calibration detection report is sent to the operation and maintenance terminal. Under the condition that calibration abnormality occurs to the sensor within a set time, the abnormality report is contained to the calibration detection report; and under the condition that calibration abnormality does not occur in the sensor within a set time, all the point cloud data acquired by the sensor and the current related operation data of the vehicle, which are acquired by the method, can be contained in the calibration detection report.
According to the method, the operation and maintenance personnel can acquire the calibration state of the sensor and the running state of the vehicle within the set time, so that the comprehensive evaluation of the vehicle is facilitated, and the safety of the vehicle in running is improved.
For the foregoing method embodiments, for simplicity of explanation, the methodologies are shown as a series of acts, but one of ordinary skill in the art will appreciate that the present disclosure is not limited by the order of acts described, as some steps may occur in other orders or concurrently in accordance with the disclosure.
Further, those skilled in the art will also appreciate that the embodiments described in the specification are all alternative embodiments, and that the acts and modules referred to are not necessarily required by the present disclosure.
Corresponding to the embodiment of the application function implementation method, the disclosure also provides an embodiment of the application function implementation device and a corresponding terminal.
Fig. 4 illustrates a schematic diagram of a sensor detection apparatus according to an exemplary embodiment of the present disclosure, which may include:
a first point cloud acquisition module 401 for respectively acquiring first point cloud data of a plurality of sensors provided on a vehicle in an initial coordinate system;
A second point cloud generating module 402, configured to convert, based on a first conversion relationship, first point cloud data of the sensor to a vehicle coordinate system, to obtain second point cloud data of the sensor, where the first conversion relationship is obtained according to an initial setting pose of the sensor on the vehicle;
a merging point cloud generating module 403, configured to merge second point cloud data of at least two sensors in the plurality of sensors to obtain merging point cloud data;
and the calibration detection module 404 is configured to determine a calibration state of the sensor according to the second point cloud data and the combined point cloud data.
In combination with any one of the embodiments provided in the present disclosure, the second point cloud generating module is configured to convert, based on a first conversion relationship, first point cloud data of the sensor to a vehicle coordinate system, respectively, and specifically configured to:
the first point cloud data of the sensor is respectively converted to a vehicle coordinate system based on a six-degree-of-freedom conversion matrix converted from an initial coordinate system of the sensor to the vehicle coordinate system, wherein the six-degree-of-freedom conversion matrix comprises a rotation matrix and a translation matrix.
In combination with any one of the embodiments provided in the present disclosure, the calibration detection module determines a calibration state of the sensor according to the first point cloud data, the second point cloud data, and the combined point cloud data of the sensor, and is specifically configured to:
Registering the first point cloud data of the first sensor with the merging point cloud data to obtain a second conversion relation, wherein the first sensor is any one of the plurality of sensors;
converting the first point cloud data of the first sensor into a coordinate system of a merging point cloud based on the second conversion relation to obtain third point cloud data of the first sensor, wherein the coordinate system of the merging point cloud is obtained by fitting the merging point cloud data;
and comparing the second point cloud data with the third point cloud data of the first sensor, and determining the calibration state of the first sensor according to the comparison result.
In combination with any one of the embodiments provided in the present disclosure, the calibration detection module compares the second point cloud data and the third point cloud data of the first sensor, and determines a calibration state of the sensor according to a comparison result, including:
determining that the first sensor calibration is abnormal under the condition that the average position offset distance of the corresponding point coordinates in the second point cloud data and the third point cloud data exceeds a first set threshold value;
and under the condition that the average position offset distance of the corresponding point coordinates in the second point cloud data and the third point cloud data does not exceed a first set threshold value, determining that the first sensor is calibrated normally.
In combination with any one of the real-time modes provided by the disclosure, the calibration detection module determines a calibration state of the sensor according to the second point cloud data and the merging point cloud data, including:
registering the first point cloud data of the first sensor and the merging point cloud data to obtain a second conversion relation;
determining that the first sensor calibration is abnormal under the condition that the parameter deviation in the first conversion relation and the second conversion relation exceeds a second set threshold value;
and under the condition that the parameter deviation in the first conversion relation and the second conversion relation does not exceed a second set threshold value, determining that the first sensor is calibrated normally.
In combination with any one of the embodiments provided in the present disclosure, the merging point cloud generating module merges the second point cloud data of at least two sensors in the plurality of sensors to obtain merging point cloud data, which is specifically configured to:
and merging the second point cloud data of the other sensors except the first sensor in the plurality of sensors to obtain merged point cloud data.
In combination with any one of the embodiments provided in the present disclosure, the apparatus further includes a first angle comparison module configured to:
Carrying out multi-frame point cloud data combination on the combination point cloud data to obtain multi-frame combination point cloud data;
based on the multi-frame merging point cloud data, acquiring relative displacement of the vehicle to obtain the moving direction of the vehicle;
and comparing the positive direction orientation of the coordinate system of the merging point cloud with the moving direction of the vehicle, and determining the calibration state of the sensor according to the comparison result.
In combination with any one of the embodiments provided in the present disclosure, the first angle comparing module compares the coordinate system positive direction of the merging point cloud with the vehicle moving direction, and determines, according to a comparison result, a calibration state of the sensor, where the first angle comparing module is specifically configured to:
determining that at least one sensor of the plurality of sensors is abnormal in calibration under the condition that the angle of the included angle between the two directions exceeds a first angle threshold;
and under the condition that the angle of the included angle between the two directions does not exceed a first angle threshold, determining that the calibration of the plurality of sensors is normal.
In combination with any one of the embodiments provided in the present disclosure, the apparatus further includes a second angle comparison module configured to:
performing ground segmentation processing on the merging point cloud data to obtain ground segmentation point cloud data;
and comparing the merging point cloud data with the ground division point cloud data, and determining the calibration state of the sensor according to a comparison result.
In combination with any one of the embodiments provided in the present disclosure, the second angle comparison module compares the merging point cloud data with the ground division point cloud data, and determines a calibration state of the sensor according to a comparison result, where the second angle comparison module is specifically configured to:
determining that at least one sensor of the plurality of sensors is abnormal in calibration under the condition that the angle between the positive direction of the coordinate system of the merging point cloud data and the ground included angle acquired by the ground division point cloud exceeds a second angle threshold;
and under the condition that the angle between the positive direction of the coordinate system of the merging point cloud data and the ground included angle acquired by the ground division point cloud does not exceed a second angle threshold value, determining that the plurality of sensors are calibrated normally.
In combination with any one of the embodiments provided in the present disclosure, the apparatus further includes an anomaly reporting module configured to:
and under the condition that the sensor calibration abnormality exists, generating a sensor calibration abnormality report, and sending the abnormality report to an operation and maintenance terminal.
In combination with any one of the embodiments provided in the present disclosure, the apparatus further includes a detection report module configured to:
and generating a sensor calibration detection report according to the calibration state of the sensor at set time intervals, and sending the detection report to an operation and maintenance terminal. For the device embodiments, reference is made to the description of the method embodiments for the relevant points, since they essentially correspond to the method embodiments. The apparatus embodiments described above are merely illustrative, wherein the elements described above as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the objectives of the disclosed solution. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
Fig. 5 illustrates a block diagram of an electronic device according to an exemplary embodiment of the present disclosure.
As shown in fig. 5, the apparatus may include: a processor, a memory, a network interface, and an internal bus. Wherein the processor, the memory, and the network interface are communicatively coupled to each other within the device via a bus.
The processor may be implemented as a general-purpose CPU (Central Processing Unit ), microprocessor, application specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, etc. for executing the relevant programs to implement the technical solutions provided herein.
The Memory may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory ), static storage devices, dynamic storage devices, or the like. The memory may store an operating system and other application programs, and when the technical solutions provided in the present application are implemented in software or firmware, relevant program codes are stored in the memory and invoked by the processor for execution.
The network interface is used to connect a communication module (not shown) to enable communication interaction between the device and other devices. The communication module may implement communication through a wired manner (such as USB, network cable, etc.), or may implement communication through a wireless manner (such as mobile network, WIFI, bluetooth, etc.).
A bus includes a path to transfer information between components of a device (e.g., processor, memory, network interface).
It should be noted that although the above device only shows a processor, a memory, a network interface, and a bus, the device may include other components necessary to achieve normal operation in a specific implementation. Furthermore, it will be understood by those skilled in the art that the above-described apparatus may include only the components necessary to implement the present application, and not all the components shown in the drawings.
In an exemplary embodiment, the present disclosure also provides a non-transitory computer-readable storage medium including instructions, such as a memory including instructions, executable by a processor of an electronic device to implement the steps of the wireless headset connection method described above. For example, the non-transitory computer readable storage medium may be ROM, random Access Memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.
In an exemplary embodiment, the disclosure further provides an intelligent vehicle, including the electronic device.
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any adaptations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.
It is to be understood that the present disclosure is not limited to the precise arrangements and instrumentalities shown in the drawings, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
Claims (16)
1. A method of sensor detection, the method comprising:
acquiring first point cloud data of a plurality of sensors arranged on a vehicle in an initial coordinate system ；
Based on a first conversion relation, respectively converting first point cloud data of the sensor into a vehicle coordinate system to obtain second point cloud data of the sensor, wherein the first conversion relation is obtained according to an initial setting pose of the sensor on the vehicle;
combining second point cloud data of at least two sensors in the plurality of sensors to obtain combined point cloud data;
and comparing the second point cloud data with the merging point cloud data or other data processed by the merging point cloud data, determining whether a coordinate system of the second point cloud data is the vehicle coordinate system, and determining the calibration state of the sensor.
2. The method of claim 1, wherein the converting the first point cloud data of the sensor to the vehicle coordinate system based on the first conversion relation, respectively, comprises:
The first point cloud data of the sensor is respectively converted to a vehicle coordinate system based on a six-degree-of-freedom conversion matrix converted from an initial coordinate system of the sensor to the vehicle coordinate system, wherein the six-degree-of-freedom conversion matrix comprises a rotation matrix and a translation matrix.
3. The method of claim 1, wherein the determining the calibration state of the sensor comprises:
registering the first point cloud data of the first sensor with the merging point cloud data to obtain a second conversion relation, wherein the first sensor is any one of the plurality of sensors;
converting the first point cloud data of the first sensor into a coordinate system of a merging point cloud based on the second conversion relation to obtain third point cloud data of the first sensor, wherein the coordinate system of the merging point cloud is obtained by fitting the merging point cloud data;
and comparing the second point cloud data with the third point cloud data of the first sensor, and determining the calibration state of the first sensor according to the comparison result.
4. A method according to claim 3, wherein comparing the second point cloud data of the first sensor with the third point cloud data and determining the calibration state of the sensor according to the comparison result comprises:
Determining that the first sensor calibration is abnormal under the condition that the average position offset distance of the corresponding point coordinates in the second point cloud data and the third point cloud data exceeds a first set threshold value;
and under the condition that the average position offset distance of the corresponding point coordinates in the second point cloud data and the third point cloud data does not exceed a first set threshold value, determining that the first sensor is calibrated normally.
5. The method of claim 1, wherein the determining the calibration state of the sensor comprises:
registering the first point cloud data of the first sensor with the merging point cloud data to obtain a second conversion relation, wherein the first sensor is any one of the plurality of sensors;
determining that the first sensor calibration is abnormal under the condition that the parameter deviation in the first conversion relation and the second conversion relation exceeds a second set threshold value;
and under the condition that the parameter deviation in the first conversion relation and the second conversion relation does not exceed a second set threshold value, determining that the first sensor is calibrated normally.
6. The method of claim 3, wherein the merging the second point cloud data of at least two of the plurality of sensors to obtain merged point cloud data comprises:
And merging the second point cloud data of the other sensors except the first sensor in the plurality of sensors to obtain merged point cloud data.
7. The method according to claim 1, wherein the method further comprises:
carrying out multi-frame point cloud data combination on the combination point cloud data to obtain multi-frame combination point cloud data;
based on the multi-frame merging point cloud data, acquiring relative displacement of the vehicle to obtain the moving direction of the vehicle;
and comparing the positive direction orientation of the coordinate system of the merging point cloud with the moving direction of the vehicle, and determining the calibration state of the sensor according to the comparison result.
8. The method of claim 7, wherein comparing the coordinate system positive direction orientation of the merging point cloud with the vehicle movement direction, and determining the calibration state of the sensor according to the comparison result, comprises:
determining that at least one sensor of the plurality of sensors is abnormal in calibration under the condition that the angle of the included angle between the two directions exceeds a first angle threshold;
and under the condition that the angle of the included angle between the two directions does not exceed a first angle threshold, determining that the calibration of the plurality of sensors is normal.
9. The method according to claim 1, wherein the method further comprises:
Performing ground segmentation processing on the merging point cloud data to obtain ground segmentation point cloud data;
and comparing the merging point cloud data with the ground division point cloud data, and determining the calibration state of the sensor according to a comparison result.
10. The method of claim 9, wherein comparing the combined point cloud data with the ground split point cloud data, and determining the calibration state of the sensor based on the comparison result, comprises:
determining that at least one sensor of the plurality of sensors is abnormal in calibration under the condition that the angle between the positive direction of the coordinate system of the merging point cloud data and the ground included angle acquired by the ground division point cloud exceeds a second angle threshold;
and under the condition that the angle between the positive direction of the coordinate system of the merging point cloud data and the ground included angle acquired by the ground division point cloud does not exceed a second angle threshold value, determining that the plurality of sensors are calibrated normally.
11. The method according to any one of claims 1-10, further comprising:
and under the condition that the sensor calibration abnormality is determined, generating a sensor calibration abnormality report, and sending the abnormality report to an operation and maintenance terminal.
12. The method according to any one of claims 1-10, further comprising:
and generating a sensor calibration detection report according to the calibration state of the sensor at set time intervals, and sending the calibration detection report to an operation and maintenance terminal.
13. A sensor detection device, the device comprising:
the first point cloud acquisition module is used for respectively acquiring first point cloud data of a plurality of sensors arranged on the vehicle under an initial coordinate system;
the second point cloud generation module is used for respectively converting the first point cloud data of the sensor into a vehicle coordinate system based on a first conversion relation to obtain second point cloud data of the sensor, wherein the first conversion relation is obtained according to the initial setting pose of the sensor on the vehicle;
the merging point cloud generation module is used for merging second point cloud data of at least two sensors in the plurality of sensors to obtain merging point cloud data;
and the calibration detection module is used for comparing the second point cloud data with the merging point cloud data or other data processed by the merging point cloud data, determining whether the coordinate system of the second point cloud data is the vehicle coordinate system, and determining the calibration state of the sensor.
14. An electronic device, the electronic device comprising:
a memory for storing processor-executable instructions;
a processor configured to execute executable instructions in the memory to implement the steps of the method of any one of claims 1 to 12.
15. A computer readable storage medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the steps of the method according to any one of claims 1-12.
16. An intelligent vehicle comprising the electronic device of claim 14.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111486395.7A CN115235525B (en) | 2021-12-07 | 2021-12-07 | Sensor detection method, sensor detection device, electronic equipment and readable storage medium |
| PCT/CN2022/071109 WO2023103143A1 (en) | 2021-12-07 | 2022-01-10 | Sensor inspection method and apparatus, electronic device, and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111486395.7A CN115235525B (en) | 2021-12-07 | 2021-12-07 | Sensor detection method, sensor detection device, electronic equipment and readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115235525A CN115235525A (en) | 2022-10-25 |
| CN115235525B true CN115235525B (en) | 2023-05-23 |
Family
ID=83666049
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111486395.7A Active CN115235525B (en) | 2021-12-07 | 2021-12-07 | Sensor detection method, sensor detection device, electronic equipment and readable storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN115235525B (en) |
| WO (1) | WO2023103143A1 (en) |
Family Cites Families (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP6649191B2 (en) * | 2016-06-29 | 2020-02-19 | クラリオン株式会社 | In-vehicle processing unit |
| CN107167090A (en) * | 2017-03-13 | 2017-09-15 | 深圳市速腾聚创科技有限公司 | Vehicle overall dimension measuring method and system |
| US11346950B2 (en) * | 2018-11-19 | 2022-05-31 | Huawei Technologies Co., Ltd. | System, device and method of generating a high resolution and high accuracy point cloud |
| DE102019117821A1 (en) * | 2019-07-02 | 2021-01-07 | Valeo Schalter Und Sensoren Gmbh | Calibration of an active optical sensor system using a calibration target |
| CN110741282B (en) * | 2019-08-21 | 2023-05-26 | 深圳市速腾聚创科技有限公司 | External parameter calibration method, device, computing equipment and computer storage medium |
| CN112819896B (en) * | 2019-11-18 | 2024-03-08 | 商汤集团有限公司 | Sensor calibration method and device, storage medium and calibration system |
| CN111007530B (en) * | 2019-12-16 | 2022-08-12 | 武汉汉宁轨道交通技术有限公司 | Laser point cloud data processing method, device and system |
| WO2021189479A1 (en) * | 2020-03-27 | 2021-09-30 | 深圳市速腾聚创科技有限公司 | Pose correction method and device for roadbed sensor, and roadbed sensor |
| CN112241007A (en) * | 2020-07-01 | 2021-01-19 | 北京新能源汽车技术创新中心有限公司 | Calibration method and arrangement structure of automatic driving environment perception sensor and vehicle |
| US11703577B2 (en) * | 2020-08-14 | 2023-07-18 | Baidu Usa Llc | Recalibration determination system for autonomous driving vehicles with multiple LiDAR sensors |
| CN112146682B (en) * | 2020-09-22 | 2022-07-19 | 福建牧月科技有限公司 | Sensor calibration method and device for intelligent automobile, electronic equipment and medium |
| CN112577517A (en) * | 2020-11-13 | 2021-03-30 | 上汽大众汽车有限公司 | Multi-element positioning sensor combined calibration method and system |
| CN113610745A (en) * | 2021-01-20 | 2021-11-05 | 腾讯科技（深圳）有限公司 | Calibration evaluation parameter acquisition method and device, storage medium and electronic equipment |
| CN113487479B (en) * | 2021-06-30 | 2024-08-02 | 北京易控智驾科技有限公司 | Method and system for detecting and identifying high-precision map boundary at vehicle end in real time |
| CN113658256B (en) * | 2021-08-16 | 2024-07-16 | 智道网联科技(北京)有限公司 | Target detection method and device based on laser radar and electronic equipment |
-
2021
- 2021-12-07 CN CN202111486395.7A patent/CN115235525B/en active Active
-
2022
- 2022-01-10 WO PCT/CN2022/071109 patent/WO2023103143A1/en unknown
Also Published As
| Publication number | Publication date |
|---|---|
| CN115235525A (en) | 2022-10-25 |
| WO2023103143A1 (en) | 2023-06-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10423164B2 (en) | Object position measurement with automotive camera using vehicle motion data | |
| JP6442193B2 (en) | Point cloud position data processing device, point cloud position data processing system, point cloud position data processing method and program | |
| CN110119698B (en) | Method, apparatus, device and storage medium for determining object state | |
| CN107710094B (en) | Online calibration check during autonomous vehicle operation | |
| US20140168423A1 (en) | Monitoring device, monitoring system and monitoring method | |
| CN110470333B (en) | Calibration method and device of sensor parameters, storage medium and electronic device | |
| US10974730B2 (en) | Vehicle perception system on-line diangostics and prognostics | |
| CN110597249B (en) | Robot and recharging positioning method and device thereof | |
| CN110827361B (en) | Camera group calibration method and device based on global calibration frame | |
| CN107782304B (en) | Mobile robot positioning method and device, mobile robot and storage medium | |
| CN105307115A (en) | Distributed vision positioning system and method based on action robot | |
| CN112605987A (en) | Robot navigation working method and device and robot | |
| CN115565058A (en) | Robot, obstacle avoidance method, device and storage medium | |
| WO2024104418A1 (en) | Calibration method for lidar and calibration apparatus, storage medium, and terminal device | |
| CN115235525B (en) | Sensor detection method, sensor detection device, electronic equipment and readable storage medium | |
| CN115100287B (en) | External parameter calibration method and robot | |
| CN113739819B (en) | Verification method, verification device, electronic equipment, storage medium and chip | |
| CN114227699B (en) | Robot motion adjustment method, apparatus, and storage medium | |
| CN116105790A (en) | Detection method, system and equipment for monitoring abnormal state of instrument box | |
| CN111736137B (en) | LiDAR external parameter calibration method, system, computer equipment and readable storage medium | |
| JP2022175768A (en) | Position measurement system | |
| CN111352090B (en) | Laser radar integrated with attitude system and control method | |
| CN113379850A (en) | Mobile robot control method, mobile robot control device, mobile robot, and storage medium | |
| CN114217600A (en) | Robot-based intelligent inspection method and system for substation indoor protection screen cabinet | |
| CN113837385B (en) | Data processing method, device, equipment, medium and product |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |