CN114754769A - Data synchronization time service system and method for laser radar and inertial sensor - Google Patents

Data synchronization time service system and method for laser radar and inertial sensor Download PDF

Info

Publication number
CN114754769A
CN114754769A CN202210671990.6A CN202210671990A CN114754769A CN 114754769 A CN114754769 A CN 114754769A CN 202210671990 A CN202210671990 A CN 202210671990A CN 114754769 A CN114754769 A CN 114754769A
Authority
CN
China
Prior art keywords
time
imu
laser radar
data
processing unit
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202210671990.6A
Other languages
Chinese (zh)
Other versions
CN114754769B (en
Inventor
黄战华
李晓伟
潘成
王康年
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sichuan Innovation Research Institute Of Tianjin University
Original Assignee
Sichuan Innovation Research Institute Of Tianjin University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sichuan Innovation Research Institute Of Tianjin University filed Critical Sichuan Innovation Research Institute Of Tianjin University
Priority to CN202210671990.6A priority Critical patent/CN114754769B/en
Publication of CN114754769A publication Critical patent/CN114754769A/en
Application granted granted Critical
Publication of CN114754769B publication Critical patent/CN114754769B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/10—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration
    • G01C21/12—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning
    • G01C21/16—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation
    • G01C21/165—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation combined with non-inertial navigation instruments
    • G01C21/1652—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation combined with non-inertial navigation instruments with ranging devices, e.g. LIDAR or RADAR
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/86—Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S19/00—Satellite radio beacon positioning systems; Determining position, velocity or attitude using signals transmitted by such systems
    • G01S19/01—Satellite radio beacon positioning systems transmitting time-stamped messages, e.g. GPS [Global Positioning System], GLONASS [Global Orbiting Navigation Satellite System] or GALILEO
    • G01S19/13—Receivers
    • G01S19/24—Acquisition or tracking or demodulation of signals transmitted by the system
    • G01S19/25—Acquisition or tracking or demodulation of signals transmitted by the system involving aiding data received from a cooperating element, e.g. assisted GPS
    • G01S19/256—Acquisition or tracking or demodulation of signals transmitted by the system involving aiding data received from a cooperating element, e.g. assisted GPS relating to timing, e.g. time of week, code phase, timing offset
    • G—PHYSICS
    • G04—HOROLOGY
    • G04R—RADIO-CONTROLLED TIME-PIECES
    • G04R20/00—Setting the time according to the time information carried or implied by the radio signal
    • G04R20/02—Setting the time according to the time information carried or implied by the radio signal the radio signal being sent by a satellite, e.g. GPS

Landscapes

  • Engineering & Computer Science (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Electromagnetism (AREA)
  • Automation & Control Theory (AREA)
  • Optical Radar Systems And Details Thereof (AREA)

Abstract

The invention provides a data synchronization time service system and a data synchronization time service method for a laser radar and an inertial sensor, wherein the system comprises a satellite time service module, an FPGA processing unit, an IMU, a laser radar and a control unit; the satellite time service module generates high-precision PPS (pulse per second) pulse and UTC (universal time coordinated) time, the FPGA (field programmable gate array) processing unit calibrates the PPS pulse and generates an IMU (inertial measurement unit) trigger instruction, the calibrated PPS pulse and the IMU trigger instruction control synchronous time service of the IMU and the laser radar, and a corresponding timestamp is acquired; and the industrial personal computer analyzes the calibrated laser radar data packet and the calibrated IMU data packet to complete time system alignment of the IMU data and the laser radar data. The system and the method effectively solve the problems that the IMU and the laser radar are difficult to time with the GNSS in the current multi-sensing fusion process, and the multi-sensing data are low in time synchronization precision.

Description

Data synchronization time service system and method for laser radar and inertial sensor
Technical Field
The invention relates to the technical field of time synchronization of radar data and inertial sensor data, in particular to a data synchronization time service system of a laser radar and an inertial sensor and a data synchronization time service method of the laser radar and the inertial sensor.
Background
In recent years, collapse accidents of buildings such as bridges and dams often occur in our country, and even huge casualties and property loss are caused by the collapse accidents sometimes, so that regular inspection, maintenance and repair of the buildings such as bridges and dams are very important. The detection methods for cracks and flaws on buildings such as bridges and the like mainly comprise two methods: firstly, a crane is erected on a bridge to manually enter a bridge bottom or a bridge floor for human eye close-range observation and detection; the second is to take a crack image with a high resolution camera at a distance and perform subsequent analysis. The first method has extremely high risk through human visual observation, and the observation precision is low; in the second method of taking pictures by using a camera, if the distance is long, a high error exists, and the requirement of detection precision cannot be met.
With the continuous progress of scientific technology, the unmanned aerial vehicle technology is mature more and more, the application of the unmanned aerial vehicle is wide more and more, the application of the unmanned aerial vehicle to crack detection of buildings such as bridges and the like also receives wide attention and research of people, and the method for detecting the cracks of the bridges in a short distance by using the unmanned aerial vehicle is proved to be feasible. However, when an Inertial sensor (Inertial Measurement Unit, abbreviated as IMU) and a lidar are carried on an unmanned aerial vehicle for data acquisition, data such as high-precision pose and angle of the unmanned aerial vehicle need to be obtained, and the IMU and the lidar also need to meet the condition of accurate triggering and data time alignment, so that the data obtained under the condition can be applied to subsequent algorithms for processing.
However, in the existing equipment scheme, the time synchronization precision of the pose data acquired by the IMU and the point cloud data acquired by the laser radar is insufficient, which may cause the point cloud data to fail in distortion correction, and affect the algorithm reconstruction effect, thereby affecting the detection result. Therefore, the key to obtain an ideal detection result is to ensure synchronous time service of the IMU and the laser radar and time alignment of data.
Disclosure of Invention
The present invention aims to address at least one of the above-mentioned deficiencies of the prior art. For example, an object of the present invention is to provide a method and a system for synchronizing time service between a laser radar and inertial sensing data, so as to solve the problems of poor bridge detection effect and insufficient time synchronization precision between an IMU and a laser radar in the existing unmanned aerial vehicle detection method.
In order to achieve the above object, the present invention provides a data synchronization time service method for a laser radar and an inertial sensor, where the data synchronization time service method includes the following steps: generating a first PPS (pulse per second) pulse and a GPRMC (general packet radio network controller) message at preset time intervals by using a satellite time service module; analyzing the GPRMC message to obtain UTC time; the method comprises the steps that a second PPS pulse is generated after a first PPS pulse is calibrated through an FPGA processing unit and is sent to a laser radar, so that the laser radar is triggered to start to collect point cloud data, and first counting time in a time system of the FPGA processing unit and built-in counting time in the time system of the laser radar are recorded, wherein the first counting time is the time for generating the second PPS pulse, and the built-in counting time is the time for the laser radar to start to collect the point cloud data; generating an IMU (inertial measurement unit) triggering instruction according to the preset time offset parameter through the FPGA processing unit and transmitting the IMU triggering instruction to the inertial sensor so as to trigger the inertial sensor to start to acquire IMU data and recording second counting time under the time system of the FPGA processing unit, wherein the second counting time is the time for outputting a response signal by the inertial sensor; and determining the second counting time as the real IMU acquisition time, solving the real Lidar acquisition time corresponding to each frame of point cloud data according to the first counting time and the built-in counting time, and carrying out one-to-one correspondence on the real Lidar acquisition time and the point cloud data.
In an exemplary embodiment of the data synchronization time service method of the laser radar and the inertial sensor, the frequency for generating the IMU trigger instruction may be determined according to the data acquisition frequency of the inertial sensor.
In an exemplary embodiment of the data synchronization time service method for the lidar and the inertial sensor, each time the IMU trigger instruction is generated and sent, the inertial sensor is triggered to acquire and send IMU data once, and the IMU trigger instruction may be a string of multi-byte instructions based on a serial protocol.
In an exemplary embodiment of the data synchronization time service method of the laser radar and the inertial sensor, the first count time may be determined by: setting a base counter, and taking the time count value of the generated second PPS time base counter as a laser radar time stamp; determining a first counting time according to the laser radar timestamp and the UTC time; the second count time may be determined in a manner of: taking the time count value of a time-base counter of the response signal output by the inertial sensor as an IMU timestamp; a second count time is determined based on the IMU timestamp and the UTC time.
In an exemplary embodiment of the data synchronization time service method for the laser radar and the inertial sensor, a calculation formula of the real Lidar acquisition time may be as follows:t f =(tpc-T f pc_)+T f ，t 1=(t pc -T pc1_)+T 1，T 0=t f -t 1+dt，t real1_=t 1+dT+T 0in the formula (I), wherein,T 0for the initial moment of the FPGA processing unit,dTfor the calibration delay of the FPGA processing unit for the first PPS pulse,T f when the control unit receives the laser radar time stamp, the corresponding FPGA processing unit is at the time under the time system,T f_pc when the control unit receives the laser radar time stamp, the corresponding control unit time stamp，T 1When the control unit receives the collected data of the laser radar, the corresponding time of the laser radar is under the time system,T pc1_when the control unit receives the collected data of the laser radar, the corresponding control unit time stamp,dtfor the time elapsed since the control unit started receiving the lidar acquisition data,t pc to experiencedtControl unit for rear-end connectionThe time stamp is a time stamp of the time,t f to experiencedtThe time of the post-FPGA processing unit time system,t 1to experiencedtThe time of the later laser radar is the time of the next laser radar,t real1_is composed oft 1And acquiring real Lidar time under a corresponding FPGA processing unit time system.
The invention also provides a data synchronization time service system of the laser radar and the inertial sensor, which comprises a satellite time service module, an FPGA processing unit, the laser radar, the inertial sensor and a control unit, wherein the satellite time service module is used for outputting a first PPS pulse and a GPRMC message; the FPGA processing unit is connected with the satellite time service module and used for generating a second PPS pulse and an IMU trigger instruction, sending the second PPS pulse to the laser radar, determining first counting time under the time system of the FPGA processing unit, sending the IMU trigger instruction to the inertial sensor, and determining second counting time under the time system of the FPGA processing unit, wherein the first counting time is the time for the FPGA processing unit to generate the second PPS pulse, and the second counting time is the time for the inertial sensor to output a response signal; the laser radar is connected with the FPGA processing unit and used for collecting point cloud data after receiving the second PPS pulse and outputting the point cloud data and the built-in counting time under the laser radar time system; the inertial sensor is connected with the FPGA processing unit and used for receiving IMU triggering instructions and then starting to acquire IMU data containing the pose of the unmanned aerial vehicle and outputting response signals; the control unit is respectively connected with the FPGA processing unit and the laser radar and used for solving the real Lidar acquisition time corresponding to each frame of point cloud data according to the first counting time and the built-in counting time and enabling the real Lidar acquisition time to correspond to the point cloud data one by one.
In an exemplary embodiment of the data synchronization time service system of the lidar and the inertial sensor, the FPGA processing unit may include a base clock module, a UTC analysis module, a trigger calibration module, a trigger instruction generation module, an IMU data analysis module, and a splicing module, wherein the base clock module is configured to generate a base clock counter in combination with a first PPS pulse, the base counter is configured to generate a lidar timestamp and an IMU timestamp in a time system of the FPGA processing unit, the lidar timestamp corresponds to a time count value when a second PPS pulse is generated, and the IMU timestamp corresponds to a time count value when an inertial sensor outputs a response signal; the UTC analysis module is configured to analyze the GPRMC message so as to extract UTC time and perform labeling operation on the UTC time; the trigger calibration module is connected with the base clock module and is configured to calibrate the first PPS pulse based on the base clock to generate a second PPS pulse and generate an IMU trigger signal according to the offset parameter at the preset moment; the trigger instruction generation module is connected with the trigger calibration module and is configured to generate an IMU trigger instruction based on an IMU trigger signal; the IMU data analysis module is connected with the inertial sensor and is configured to analyze a response signal output by the inertial sensor to obtain IMU data; the splicing module is connected with the IMU data analysis module and is configured to splice the analyzed IMU data and the IMU timestamp and perform labeling operation to obtain an IMU data packet.
In an exemplary embodiment of the data synchronization time service system of the lidar and the inertial sensor, the FPGA processing unit may further include a first buffer area, a second buffer area, a third buffer area, and a priority management module, wherein the first buffer area is connected to the splicing module and is configured to buffer and store a lidar timestamp; the second buffer area is connected with the splicing module and is used for buffering and storing IMU data packets; the third buffer area is connected with the UTC analysis module and used for buffering and storing UTC time; the priority management module is respectively connected with the first buffer area, the second buffer area and the third buffer area and used for generating access signals according to a preset priority sequence to access the three buffer areas in sequence and generating serial data packets.
In an exemplary embodiment of the data synchronization time service system of the laser radar and the inertial sensor, the control unit may include a time synchronization module and a time coordinate alignment module, wherein the time synchronization module is configured to be capable of solving a real Lidar acquisition time corresponding to each frame of point cloud data according to the first counting time and the built-in counting time; the time coordinate alignment module is configured to enable one-to-one correspondence of real Lidar acquisition times to point cloud data.
Compared with the prior art, the beneficial effects of the invention comprise at least one of the following:
(1) the application provides a data synchronization time service system and method of a laser radar and an inertial sensor, wherein an unstable PPS second time pulse generated by a GNSS time service module is calibrated by a trigger calibration unit in an FPGA, and the corrected accurate and stable PPS second time pulse is generated;
(2) the application provides a data synchronization time service system and method of a laser radar and an inertial sensor, a high-precision IMU trigger signal and a corrected accurate and stable PPS second time pulse are generated by the aid of the PPS second time pulse generated by a GNSS time service module and by combination of a base clock unit in an FPGA and calibration of a trigger calibration unit, so that the influence of instability of the PPS second time pulse generated by the GNSS time service module is avoided, and meanwhile, the trigger time of the IMU inertial measurement unit and the laser radar can be accurately controlled;
(3) the industrial personal computer analyzes the calibrated laser radar data packet and the IMU data packet, so that the IMU data and the laser radar data analyzed by the industrial personal computer are ensured to be in the same time coordinate system, and data time alignment is completed. The method and the device effectively solve the problems that the time synchronization precision of pose data acquired by an IMU and point cloud data acquired by a laser radar is insufficient, so that the point cloud data is not corrected successfully, the algorithm reconstruction effect is influenced, and the detection result is influenced.
Drawings
The above and other objects and/or features of the present invention will become more apparent from the following description taken in conjunction with the accompanying drawings, in which:
FIG. 1 is a schematic structural flow diagram of a synchronous time service system according to an exemplary embodiment of the data synchronous time service system of the laser radar and the inertial sensor.
FIG. 2 is a schematic flow chart of a synchronous time service method according to an exemplary embodiment of the data synchronous time service method of the laser radar and the inertial sensor.
Fig. 3 is a schematic diagram illustrating time asynchronization between the lidar and the inertial sensor according to an exemplary embodiment of the data synchronization and time service system for the lidar and the inertial sensor of the present invention.
FIG. 4 is a schematic diagram illustrating a relationship between an FPGA processing unit time system and a lidar time system of an exemplary embodiment of the data synchronization time service system of the lidar and the inertial sensor according to the present invention.
Fig. 5 shows a schematic diagram of a relationship among an FPGA processing unit time system, a lidar time system, and an industrial personal computer time check timer according to an exemplary embodiment of the data synchronization time service system of the lidar and the inertial sensor of the present invention.
FIG. 6 is a schematic flow chart illustrating a lidar time synchronization method according to an exemplary embodiment of the lidar and inertial sensor data synchronization time service system of the present invention.
Description of the reference numerals:
the method comprises the following steps of 100-GNSS high-precision time service module, 200-FPGA processing unit, 201-base clock module, 202-UTC analysis module, 203-trigger calibration module, 204-trigger instruction generation module, 205-IMU data analysis module, 206-splicing module, 207-first buffer area, 208-second buffer area, 209-third buffer area, 210-priority management module, 300-laser radar, 400-IMU inertial sensor and 500-industrial personal computer.
Detailed Description
Hereinafter, the data synchronization timing system and method of the lidar and the inertial sensor of the invention will be described in detail with reference to exemplary embodiments.
It should be noted that "first," "second," and the like are merely for convenience of description and for convenience of distinction, and are not to be construed as indicating or implying relative importance.
In the description of the present invention, it should also be noted that, unless otherwise explicitly specified or limited, the terms "mounted" and "connected" are to be interpreted broadly, e.g., as either a fixed connection, a removable connection, or an integral connection; can be directly connected or indirectly connected; either a wired or wireless connection. The specific meaning of the above terms in the present invention can be understood as appropriate to those of ordinary skill in the art.
In the process of detecting cracks on the surface of a bridge by using an unmanned aerial vehicle, the problem that the synchronous precision of an Inertial sensor (Inertial Measurement Unit, hereinafter abbreviated as IMU) carried on the unmanned aerial vehicle and the acquired data of a laser radar is insufficient in time often occurs, so that the point cloud data distortion correction fails, the algorithm reconstruction result is influenced, and the detection result is influenced.
In order to solve the above problems, the present invention provides a System and a method for synchronizing a laser radar and inertial sensing data based on a Global Navigation Satellite System (GNSS), where the System uses a high-precision PPS second Pulse (PPS), a Universal Time Coordinated (UTC) Time, and a high-precision control signal generated by an FPGA (Field Programmable Gate Array), which are generated by the GNSS, to accurately control the respective triggered precise times of the IMU and the laser radar, and to accurately implement the synchronization of the data of the two on Time. In addition, the industrial personal computer can perform time synchronization on the point cloud data set and the IMU data set according to the UTC time of the point cloud acquired by the laser radar and the timestamp of the IMU acquired data, so that the problems that the IMU and the laser radar cannot perform synchronous time service and the data time synchronization precision is low in the traditional method, the point cloud data set acquired by the laser radar fails to be distorted and corrected, and the reconstruction effect and even the detection result are affected are effectively solved.
Specifically, the invention provides a data synchronization time service method for a laser radar and an inertial sensor.
In an exemplary embodiment of the invention, a data synchronization time service method of a laser radar and an inertial sensor is realized through the following steps.
Step one, a satellite time service module is used for generating a first PPS pulse and a GPRMC message at intervals of preset time and sending the first PPS pulse and the GPRMC message to an FPGA processing unit.
And step two, analyzing a GPRMC message (Recommended Minimum Specific GPS/TRANSIT Data (RMC) and recommending positioning information) and acquiring UTC time.
And step three, calibrating the first PPS pulse through the FPGA processing unit to obtain an accurate second PPS pulse, sending the second PPS pulse to the laser radar to trigger the laser radar to start to collect point cloud data, and recording first counting time under the time system of the FPGA processing unit and built-in counting time under the time system of the laser radar. The first counting time is the time for generating the second PPS pulse, and the built-in counting time is the time for starting the laser radar to collect point cloud data.
For example, the trigger frequency of the laser radar is 1Hz, and the trigger high level duration is 20ms to 200ms, that is, 10 times of point cloud data are collected each time the laser radar receives the rising edge of the second PPS pulse.
The first PPS pulse refers to an original PPS signal from a satellite time service module, and in the actual operation process of equipment, the GNSS signal loss condition is likely to be met, so that the PPS signal is interrupted, and therefore the first PPS pulse needs to be calibrated into a stable second PPS pulse through an FPGA processing unit.
And fourthly, after the first PPS pulse is calibrated through the FPGA processing unit, generating an IMU trigger instruction according to the offset parameter at the preset moment and transmitting the IMU trigger instruction to the inertial sensor so as to trigger the inertial sensor to start to acquire IMU data, and recording second counting time under the time system of the FPGA processing unit, wherein the second counting time is the time for the inertial sensor to output a response signal.
The response signal includes acquired IMU data, for example, data information such as corresponding pose of the unmanned aerial vehicle.
And step five, determining the second counting time as the real IMU acquisition time, solving the real Lidar acquisition time corresponding to each frame of point cloud data according to the first counting time and the built-in counting time, and carrying out one-to-one correspondence on the real Lidar acquisition time and the point cloud data.
The synchronous time service of the laser radar and the inertial sensor is realized by converting the acquisition time of the laser radar into a time system which is the same as the acquisition time of the inertial sensor. The synchronous time service actually means that the two sensors are both in a time system, namely, the two sensors are timed by the same base clock and the same base counter, and actually trigger time, the two sensors (namely, the laser radar and the inertial sensor) can be adjusted according to actual conditions, but are in a unified time system.
In this embodiment, the frequency of the IMU trigger command generated by the FPGA processing unit is determined by the data acquisition frequency of the inertial sensor. The frequency of the IMU trigger instruction generated by the FPGA processing unit may be equal to the data acquisition frequency of the inertial sensor, for example, the operating frequency of the inertial sensor is 100Hz, that is, the inertial sensor is triggered to send a request signal every 10ms, that is, the frequency of the IMU trigger instruction generated by the FPGA processing unit is 100 Hz.
And each time the IMU trigger instruction is generated and sent, the inertial sensor is triggered to acquire and send IMU data once. The IMU trigger instruction is a series of multi-byte instructions based on a serial protocol, such as an 8-byte trigger instruction under the UART protocol.
In the third step, a base counter is set, and the time count value of the base counter for generating the second PPS pulse is used as the laser radar timestamp; a first count time is determined based on the lidar timestamp and the UTC time.
In step four, the second counting time may be determined by: taking the time count value of a time-base counter of the response signal output by the inertial sensor as an IMU timestamp; a second count time is determined based on the IMU timestamp and the UTC time.
In the present embodiment, in step five, the real Lidar acquisition time is calculated by the following formula.
t f =(tpc-T f pc_)+T f （1）
t 1=(t pc -T pc1_)+T 1（2）
T 0=t f -t 1+dt（3）
t real1_=t 1+dT+T 0（4）
Wherein,T 0for the initial moment of the processing unit of the FPGA,dTfor the calibration delay of the FPGA processing unit for the first PPS pulse,T f when the control unit receives the laser radar time stamp, the corresponding FPGA processing unit processes the time at the moment under the time system,T f_pc when the control unit receives the laser radar time stamp, the corresponding control unit time stamp，T 1When the control unit receives the collected data of the laser radar, the corresponding time of the laser radar is under the time system,T pc1_when the control unit receives the collected data of the laser radar, the corresponding control unit time stamp,dtfor the time elapsed since the control unit started receiving the lidar acquisition data,t pc to experiencedtThe time stamp of the corresponding control unit in the back,t f to experiencedtThe time of the post-FPGA processing unit time system,t 1to experiencedtThe time of the later laser radar under the time system,t real1_is composed oft 1And acquiring real Lidar acquisition time under a corresponding FPGA processing unit time system.
The invention further provides a data synchronization time service system of the laser radar and the inertial sensor.
In another exemplary embodiment of the invention, the data synchronization time service system of the laser radar and the inertial sensor comprises a satellite time service module, an FPGA processing unit, the laser radar, the inertial sensor and a control unit.
The satellite time service module is used for outputting a first PPS pulse and a GPRMC message every predetermined time (for example, every 1 s). For example, the satellite time service module is configured to generate a first PPS pulse with high precision every second, output a GPRMC message including universal coordinated time (UTC) information at the current time while generating the first PPS pulse, and send the first PPS pulse and the GPRMC message to the FPGA processing unit.
The FPGA processing unit is connected with the satellite time service module and is used for generating a second PPS pulse and an IMU trigger instruction, sending the second PPS pulse to the laser radar and determining first counting time under the time system of the FPGA processing unit; and sending an IMU trigger instruction to the inertial sensor, and determining second counting time under the FPGA processing unit time system. The first counting time is the time for generating the second PPS pulse by the FPGA processing unit, and the second counting time is the time for outputting a response signal by the inertial sensor.
Specifically, when receiving a first PPS pulse, the FPGA processing unit calibrates the first PPS pulse to obtain a second PPS pulse, and sends the second PPS pulse to the laser radar to trigger the laser radar to start to acquire point cloud data; and on the other hand, an IMU triggering instruction is generated after the first PPS pulse is calibrated, and is transmitted to the inertial sensor so as to trigger the inertial sensor to start to acquire IMU data. And the FPGA processing unit analyzes the GPRMC message when receiving the message, extracts UTC time and transmits the UTC time to the control unit.
And the laser radar is connected with the FPGA processing unit. And the laser radar is used for receiving the second PPS pulse and starting to acquire point cloud data under the trigger of the second PPS pulse.
The inertial sensor is connected with the FPGA processing unit. The inertial sensor is used for receiving IMU triggering instructions and then starting to acquire IMU data containing the pose of the unmanned aerial vehicle and outputting response signals. Wherein, the response signal contains the collected IMU data.
The control unit is respectively connected with the FPGA processing unit and the laser radar. The control unit is used for solving the real Lidar acquisition time corresponding to each frame of point cloud data according to the first counting time and the built-in counting time, and enabling the real Lidar acquisition time to be in one-to-one correspondence with the point cloud data.
In this embodiment, the FPGA processing unit may include a base clock module, a UTC parsing module, a trigger calibration module, a trigger instruction generating module, an IMU data parsing module, and a splicing module.
Wherein the base clock module is configured to generate a base clock and a base counter in conjunction with the first PPS pulse.
It should be noted that, the specific implementation of the base clock module may be: a second counter is provided, and if the second counter is full or the first PPS pulse is received, the second counter is reset, the second counter is used as a base counter of the whole data synchronization time service system, and a clock pulse signal generated by the base counter is used as a base clock signal.
The base counter is active all the time after the system is powered up and is reset on the first PPS pulse rising edge or when the internal second counter is full. The base counter is used for generating a laser radar time stamp and an IMU time stamp under the FPGA processing unit time system, the laser radar time stamp corresponds to a time count value when the second PPS pulse is generated, and the IMU time stamp corresponds to a time count value when the inertial sensor outputs a response signal. The UTC analysis module is configured to analyze the GPRMC message to extract UTC time and perform a labeling operation on the UTC time.
The trigger calibration module is connected with the base clock module and configured to calibrate the first PPS pulse based on the base clock to generate a second PPS pulse, and generate an IMU trigger signal according to the preset time offset parameter. The specific implementation mode of the trigger calibration unit is that a corresponding trigger calibration condition is set to generate a corresponding trigger signal according to the actual trigger signal requirement through a base clock and a base counter. The triggering condition is adjusted through preset parameters, and the parameters can be adjusted through calibrating actual triggering feedback, so that the final triggering precision is ensured to be met. The adjusted PPS pulse is the second PPS pulse. The method has the characteristics that the GNSS offline is kept stable, the GNSS online is preferentially adopted, and the GNSS online is manually adjustable.
The trigger instruction generation module is connected with the trigger calibration module and is configured to generate an IMU trigger instruction based on the IMU trigger signal.
The IMU data analysis module is connected with the inertial sensor and is configured to analyze the response signal output by the inertial sensor to obtain IMU data.
The splicing module is connected with the IMU data analysis module and is configured to splice the analyzed IMU data and the IMU timestamp and perform labeling operation to obtain an IMU data packet.
Furthermore, the FPGA processing unit can also comprise a first buffer area, a second buffer area, a third buffer area and a priority management module.
The first buffer area is connected with the splicing module and used for buffering and storing the laser radar time stamp.
The second buffer area is connected with the splicing module and used for buffering and storing the IMU data packets.
And the third buffer area is connected with the UTC analysis module and is used for buffering and storing UTC time.
The priority management module is respectively connected with the first buffer area, the second buffer area and the third buffer area and used for generating access signals according to a preset priority sequence to access the three buffer areas in sequence and generating serial data packets.
In this embodiment, the control unit may include a time synchronization module and a time coordinate alignment module. And the time synchronization module is configured to solve the real Lidar acquisition time corresponding to each frame of point cloud data according to the first counting time and the built-in counting time. The time coordinate alignment module is configured to enable real Lidar acquisition time to be in one-to-one correspondence with the point cloud data so as to ensure that the IMU data and the point cloud data are in the same time coordinate system.
For a better understanding of the above-described exemplary embodiments of the present invention, reference is made to the following detailed description and accompanying drawings.
Example 1
As shown in fig. 1, the schematic structural diagram of a data synchronization time service system of a lidar and an inertial sensor based on an FPGA includes a GNSS high-precision time service module 100, an FPGA processing unit 200, a lidar 300, an IMU inertial sensor 400, and an industrial personal computer 500. The GNSS high-precision time service module 100 is connected with the FPGA processing unit 200, the FPGA processing unit 200 is respectively connected with the laser radar 300 and the IMU inertial sensor 400, and the industrial personal computer 500 is respectively connected with the laser radar 300 and the FPGA processing unit 200.
The GNSS high-precision time service module 100 is configured to generate a first PPS pulse and a GNSS-NMEA standard message with high precision every second, and transmit the first PPS pulse and the first PPS pulse to the FPGA processing unit 200. The FPGA processing unit 200 is configured to generate a lidar trigger signal (i.e., a second PPS pulse) and an IMU trigger signal while receiving the first PPS pulse transmitted by the GNSS high-precision time service module 100, and transmit the lidar trigger signal and the IMU trigger signal to the lidar and the inertial sensor, respectively.
The FPGA processing unit 200 may specifically include a base clock module 201, a UTC parsing module 202, a trigger calibration module 203, a trigger instruction generating module 204, an IMU data parsing module 205, a splicing module 206, a first buffer 207, a second buffer 208, a third buffer 209, and a priority management module 210.
The base clock module 201 is configured to combine the first PPS pulse generated by the GNSS high-precision timing module 100 to generate an accurate and stable base clock and a base counter, and transmit the base clock and the base counter to the trigger calibration module 203. The base counter is used to generate a lidar timestamp and an IMU timestamp. Recording the count value of the moment base counter at the second PPS pulse as a laser radar time stamp; the count value of the moment base counter is recorded as the IMU timestamp when the inertial sensor outputs the reply signal.
The UTC parsing module 202 is configured to receive the GNSS-NMEA standard message generated by the GNSS high-precision time service module 100, parse the standard message, extract UTC time, perform tagging operation on the extracted UTC time, and buffer and store the UTC time in a third buffer 209 preset by the FPGA processing unit.
The trigger calibration module 203 is configured to calibrate the first PPS pulse based on the base clock, and transmit a second PPS pulse obtained after calibration to the laser radar 300, so as to trigger the laser radar to acquire data. The trigger frequency of the laser radar 300 is 1Hz, and the trigger high level duration is 20-200 ms, namely, the laser radar is triggered to acquire point cloud data for 10 times every time the rising edge of the second PPS pulse is received. The laser radar 300 transmits the collected point cloud data packet to the industrial personal computer 500 through the internet access. Meanwhile, a laser radar time stamp is determined according to the base counter, and the laser radar time stamp corresponds to the time when the calibration module 203 is triggered to generate the calibrated second PPS pulse, and the count value corresponding to the base counter. The splicing module 206 is configured to label the lidar timestamp, and buffer and store the labeled lidar timestamp in a first buffer 207 preset by the FPGA processing unit.
In addition, the trigger calibration module 203 is further configured to generate an IMU trigger signal according to the preset time offset parameter, and send the IMU trigger signal to the trigger instruction generating module 204. The trigger instruction generating module 204 is configured to generate an IMU trigger instruction, and send the IMU trigger instruction to the IMU inertial sensor 400, so as to trigger the IMU inertial sensor to acquire data. The trigger calibration module calibrates the laser radar and the IMU trigger signal based on the base clock, and generates a second PPS pulse and an IMU trigger signal which are accurate after calibration, so that the influence of instability of the first PPS pulse generated by the GNSS high-precision time service module is avoided, and meanwhile, the trigger time of the IMU inertial sensor and the laser radar can be accurately controlled.
The frequency of the IMU trigger instructions generated by the FPGA processing unit 200 is determined by the data acquisition frequency of the IMU inertial sensor 400. For example, each time an IMU trigger command is generated and sent, which is a series of multi-byte commands based on a serial protocol, such as 8-byte trigger commands under the UART protocol, the inertial sensor is triggered to perform acquisition and sending of IMU data.
The IMU inertial sensor 400 starts to collect data according to the received IMU trigger instruction and returns the collected IMU data packets to the IMU data parsing module 205. When receiving the IMU trigger instruction, the IMU inertial sensor 400 outputs a response signal, where the response signal includes data information such as a corresponding pose.
The IMU data parsing module 205 is configured to parse the data packet returned by the IMU inertial sensor 400, and transmit the parsed IMU data to the stitching module 206. The splicing module 206 is configured to splice the parsed IMU data and IMU timestamp together, perform a labeling operation on the spliced data packet, and buffer and store the IMU data and IMU timestamp spliced and labeled IMU data packet in the second buffer 208 preset by the FPGA processing unit.
The IMU timestamp corresponds to a count value corresponding to the IMU inertial sensor starting to return data time base counter. After receiving the IMU trigger instruction generated by the trigger instruction generation module 204, the IMU inertial sensor 400 starts to acquire IMU data including the pose of the drone; when the IMU inertial sensor 400 generates a response signal, the count value of the base counter at this time is saved as the time stamp of the current IMU acquisition data frame. When the rising edge of the PPS pulse (i.e., the first PPS pulse) before calibration is detected or the base counter counts to a count value corresponding to 1s, i.e., 50M, the base counter may be automatically cleared and counting may be restarted.
The actions of buffering and storing the laser radar timestamp to the first buffer area, buffering and storing the IMU data and the IMU data packet spliced and marked by the IMU timestamp to the second buffer area, and buffering and storing the UTC time to the third buffer area are parallel (simultaneous), namely the UTC time, the IMU data packet spliced and marked by the IMU data and the IMU timestamp, and the laser radar timestamp are buffered and stored in the respective buffer areas in parallel (simultaneous). The priority management module 210 generates access signals according to a preset priority order to sequentially access the three buffers, generates serial data packets, and transmits the serial data packets to the industrial personal computer 500 through a serial protocol. The serial data packets may include UTC time, lidar time stamps, IMU time stamps, and IMU data packets. For example, the priority order may be: UTC data > lidar timestamp data > IMU related data. The UTC data is 1Hz, the laser radar timestamp is 1Hz, and the IMU data is 1Hz, so that the UTC data can be uploaded at the highest speed; compared with IMU (inertial measurement Unit) data, the laser radar time stamp data has lower generation frequency and shorter data length, so that the problem of mutual preemption from parallel to serial can be reduced as much as possible.
The reason for setting the cache area and the priority management module is as follows: the three data sources are not necessarily generated at the same time, and are likely to be generated at the same time or at similar times. And the FPGA processing unit has only one data channel for uploading to the industrial personal computer, so that various data sources cannot be uploaded at the same time. Through setting up buffer and priority management module, can be with the data cache of parallel production in the buffer to read out according to priority order in proper order through priority management module, thereby make parallel data become serial data, thereby can upload to the industrial computer through a passageway.
The industrial personal computer 500 calibrates the timestamp of the point cloud data packet fed back by the laser radar 300 through the network port according to the laser radar timestamp of the laser radar collection point cloud uploaded from the FPGA processing unit 200 to form a calibrated laser radar data timestamp. Because the laser radar time stamp and the IMU time stamp uploaded by the FPGA processing unit are generated under the calibrated base clock, after the industrial personal computer analyzes the calibrated laser radar data packet and the calibrated IMU data packet, the IMU data and the laser radar data analyzed by the industrial personal computer are ensured to be under the same time coordinate system, and therefore data time alignment is completed. The method effectively solves the problems that the time synchronization precision of the pose data acquired by the IMU and the point cloud data acquired by the laser radar is insufficient, so that the point cloud data is not corrected successfully, the algorithm reconstruction effect is influenced, and the detection result is influenced.
In addition, the industrial personal computer can also complete motion distortion compensation of the Lidar point cloud by combining the Lidar data and the IMU data. Due to the movement of the laser radar, the problem that the radar coordinate system of each measurement of the sampling point is inconsistent is caused, so that the industrial personal computer is required to project all the frame point clouds to the common laser radar coordinate system. Due to the high frequency of the IMU, there is a lot of IMU data within a frame of point cloud. The body state quantity recursion, namely the body position and posture, can be carried out by utilizing the IMU data and the IMU motion model. Meanwhile, due to the high sampling rate of the laser radar, a plurality of point cloud data exist between two frames of IMU data. And carrying out interpolation by utilizing the timestamps of the laser sampling points to obtain the relative pose of each laser sampling point relative to the last point of the frame point cloud, and projecting the relative pose to a laser radar coordinate system where the last point is located, thereby completing the motion distortion correction of the point cloud.
As shown in fig. 2, the method is a schematic flow chart of a data synchronization time service method for a lidar and an inertial sensor based on an FPGA, and includes the following steps.
S201: the GNSS high-precision time service module can generate high-precision PPS second time pulse (namely first PPS pulse) and GNSS-NMEA standard message every second and sends the high-precision PPS second time pulse and the GNSS-NMEA standard message to the FPGA processing unit, the FPGA processing unit is calibrated with the trigger calibration module through the internal base clock module, the calibrated PPS second time pulse (namely second PPS pulse) is sent to the laser radar, the laser radar is triggered to start to collect point cloud data, and the collected point cloud data packet is sent to the industrial personal computer through the network port.
When the FPGA processing unit generates a calibrated PPS second time pulse (i.e., a second PPS pulse) via the internal trigger calibration module, the count value of the base counter corresponds to the lidar timestamp.
S202: and the FPGA processing unit analyzes the GNSS-NMEA standard message through an internal UTC analysis module, extracts UTC time and performs labeling operation on the extracted UTC time.
S203: the FPGA processing unit generates IMU trigger signals according to the preset moment offset parameters through the internal trigger calibration module, sends the IMU trigger signals to the trigger instruction generation module, generates IMU trigger instructions and sends the IMU trigger instructions to the IMU inertial sensor. And the IMU inertial sensor starts to acquire data after receiving the IMU trigger instruction and returns an acquired IMU data packet to an IMU data analysis module in the FPGA processing unit.
When the IMU inertial sensor begins to return data, the count value of the base counter at this time corresponds to the IMU timestamp.
S204: the IMU data analysis module analyzes an IMU data packet returned by the IMU inertial sensor and transmits the analyzed IMU data to the splicing module; and the splicing module splices the parsed IMU data and the IMU timestamp together, and simultaneously performs labeling operation on the spliced IMU data packet.
S205: buffering and storing the laser radar timestamp in a first buffer area preset for the FPGA processing unit by the FPGA processing unit; buffering the spliced IMU data packet in a second buffer area preset for the FPGA processing unit; and the UTC time is buffered and stored in a third buffer area preset for the FPGA processing unit. The priority management module generates access signals according to a preset priority sequence to sequentially access the three buffer areas, generates serial data packets and sends the serial data packets to the industrial personal computer through a serial protocol.
And the UTC time, the IMU data packet after the IMU data and IMU timestamp splicing mark and the laser radar timestamp are buffered and stored in respective buffer areas in parallel (simultaneously).
S206: the industrial personal computer calibrates the laser radar time stamp in the point cloud data packet fed back by the laser radar through the network port according to the laser radar time stamp of the laser radar collection point cloud uploaded from the FPGA processing unit to form a calibrated laser radar data time stamp (comprising the point cloud data packet and the calibrated time stamp); and the industrial personal computer analyzes the calibrated laser radar data packet and the calibrated IMU data packet, so that the IMU data and the laser radar data are ensured to be in the same time coordinate system, and the data time alignment is completed.
According to the specific implementation mode described above, the system and the method for data synchronization time service of the laser radar and the inertial sensor based on the FPGA provided by the application generate the high-precision IMU trigger signal and the PPS second time pulse which is accurate and stable after calibration by combining the PPS second time pulse generated by the GNSS time service module and the calibration of the base clock module and the trigger calibration module inside the FPGA, so that the trigger time of the IMU inertial sensor and the laser radar can be accurately controlled, and the condition of synchronization time service of the laser radar and the inertial sensor is ensured; in addition, the industrial personal computer analyzes the calibrated laser radar data packet and the calibrated IMU data packet, so that the IMU data and the laser radar data are ensured to be under the same time coordinate system, the data time alignment is completed, and the problems that the IMU and the laser radar cannot synchronously time service and the time synchronization precision is low in the traditional method, so that the point cloud data set acquired by the laser radar fails to be distorted and corrected, and the reconstruction effect and even the detection result are influenced are effectively solved.
The UTC time is directly extracted from the GPRMC message output by the satellite time service module through the Verilog program, wherein the UTC time comprises sixteen bytes and is year, month, day, hour and minute, and the extracted UTC time is transmitted to the industrial personal computer through the FPGA and is used for forming a complete UTC timestamp with the timestamp in seconds uploaded by the IMU and the laser radar. The FPGA processing unit generates a high-precision control signal by adopting a method of two counters, wherein one counter is a fixed-frequency counter, the other counter is a counter representing offset, certain offset is carried out on the fixed-frequency signal, so that the time generated by the control signal is uniquely determined by common constraint, and the precision error is less than 1 ns. The fixed frequency counter refers to a base counter, and the offset counter refers to a counter that calibrates the trigger signal according to the base counter according to a preset trigger signal offset parameter. The offset counter and the base counter together generate a calibrated trigger signal, and the corresponding base counter count value at that time serves as the final timestamp.
The system needs to upload a plurality of sensor data packets, and a parallel FIFO storage unit (namely three buffers) and a serial uploading unit based on priority (namely a priority management module) are designed. The system is characterized in that the FPGA adds unique marks to the data of different sensors, the marked data are stored in FIFO created for the sensors of the FPGA in parallel, and the data are uploaded in series according to the preset priority.
In order to reduce the instability of the first PPS pulse generated by the GNSS high-precision time service module, a manually set micro delay exists between the second PPS pulse actually triggering the laser radar and the first PPS pulse generated by the time service module
Figure 617916DEST_PATH_IMAGE001
. The built-in timestamp of the laser radar is not UTC time, but is the internal timestamp of the laser radar from 0, and is different from the time system of the FPGA processing unit, and the timestamp of the laser radar needs to be calibrated.
In particular, the PPS second pulse transmitted to the FPGA processing unit by the GNSS high-precision time service module may disappear due to signal problemsTo solve the problem, the FPGA processing unit of the present application calibrates the signal using the internal clock to generate the internal clock, and the PPS signal for the lidar has a certain delay compared to the base clock, and the delay time is recorded asdT. On one hand, the FPGA processing unit analyzes the message from the GNSS and uploads the UTC time corresponding to the current second to the industrial personal computer at the position of 10-20 ms per second, and the frequency is 1 Hz; and on the other hand, the laser radar generates 10 frames of point cloud data after receiving the second PPS pulse, and uploads the point cloud data and the internal timestamp of the laser radar from 0 to the industrial personal computer. Therefore, the core of realizing the time synchronization of the laser radar and the inertial sensor is as follows: and solving the real time stamp of each frame of laser radar point cloud data by utilizing the UTC time of 1Hz of an internal time system of the FPGA processing unit and the laser radar internal time stamp of 10Hz of the laser radar time system.
For the laser radar synchronization, the data flow is as shown in fig. 3, and the industrial personal computer 500 receives two kinds of data from the FPGA processing unit 200, which are UTC time (1 Hz, returning the data 10ms to 20ms per second) and calibration delay time, respectivelydT(1 Hz, upload immediately after calibration PPS generation per second); the industrial personal computer 500 also receives data packets from the laser radar 300 (10 Hz at most per second, including the data packet and the time corresponding to the internal timestamp of the laser radar). Due to the fact that a certain time delay exists in the PPS second pulse triggering the laser radar in the calibration process, a certain delay exists between a timestamp in a data packet returned to the industrial personal computer by the laser radar and the FPGA processing unit. And the laser radar does not receive the UTC time sent by the industrial personal computer, the time of the laser radar starts to be 0, and the time starting point of the FPGA processing unit is the UTC time received by the starting time of the FPGA processing unit. Therefore, the time stamp in the data packet returned to the industrial personal computer by the laser radar needs to be calibrated into a time system synchronous with the inertial sensor.
Based on this, the following scheme is proposed to solve the time synchronization problem of the laser radar and the inertial sensor. FIG. 4 is a schematic diagram showing the relationship between the FPGA processing unit time system and the lidar time systemAnd (5) explaining the steps. Line a in fig. 4 represents the FPGA processing unit time frame and line B represents the lidar time frame. As shown in fig. 4, the start point of the base clock of the FPGA processing unit, i.e., the UTC time corresponding to the boot time of the FPGA processing unit, is recorded as T0The time is an integer. Due to the existence of the second PPS pulse by the FPGA processing unitdTIs thus subject todTAnd the laser radar starts to acquire point cloud data only after the point cloud data is acquired, wherein the frequency is 10Hz, and the time corresponds to 0 moment of a laser radar time system. Therefore, the time synchronization between the laser radar and the inertial sensor is completed only by calculating the time of each frame point cloud in the time system of the laser radar corresponding to the time system of the FPGA processing unit according to the time of each frame point cloud in the time system of the laser radar.
As shown in fig. 5, a schematic diagram of solving the FPGA processing unit time corresponding to the lidar is shown. Line segment a in fig. 5 represents the FPGA processing unit time system, line segment B represents the laser radar time system, and line segment C represents the industrial personal computer time check timer. As shown in FIG. 5, the elapsed time since the system was started is set todtExperienced throughdtThe time of the time system of the post FPGA processing unit ist f Experienced throughdtThe time of the post-lidar time system ist l . Then there ist f =T 0+dt，t l=dt-dT. Therefore, only the initial time of the FPGA processing unit needs to be calibratedT 0And solving for each momentdtI.e. accurately solving the correspondence of the lidar data framest f 。
When the industrial personal computer receives the time stamp of the FPGA processing unit, the time of the corresponding FPGA processing unit time system is respectively recordedT f And operating system timestamp corresponding to the timeT f_pc The refresh frequency at this time is 1 Hz. Similarly, when the point cloud data of the laser radar is received, the time of the corresponding laser radar time system is respectively recordedT lAndT pcl_the refresh frequency at this time is 10 Hz. Therefore, when any frame of point cloud data is received, the time stamp of the operating system is recorded ast pc Then the time of the corresponding FPGA processing unit time systemt f =(t pc -T f_pc )+T f =T 0+dtAt the moment of the corresponding lidar time systemt l=(t pc -T pcl_)+T l=dt-dT。
Receiving at another threadt f Andt land processed to calculateT estimate0_=t f -t l+dt. To pairT estimate0_By rounding to obtainT 0The exact value of (c). Calibrating FPGA processing unit initial timeT 0Then, the time corresponding to the FPGA processing unit time system at each moment of the laser radar can be expressed ast f =T 0+t l+dT. In addition, the calibration can be performed again every certain time in the data acquisition processT 0If the value is always a stable value, the acquisition process is normally carried out, otherwise, the system is abnormal. And thus, data synchronization between the laser radar and the inertial sensor is completed.
As shown in fig. 6, the time synchronization process of the lidar and the inertial sensor is as follows:
i) refreshing UTC time per second from FPGA processing unitsT f Refreshing the corresponding time stamp of each data frame at 10Hz from the lidarT 1。
ii) recording the timestamp of the industrial personal computer corresponding to the refreshing time of the FPGA processing unitT f_pc And, record the industrial personal computer time stamp corresponding to the laser radar refreshing timeT pcl_。
iii) starting the time of the industrial personal computer, checking the low-speed timer task (for example, 25Hz per second), confirming whether the timer is triggered, and if so, calculating the time system time of the FPGA processing unit at the momentt f =(t pc -T f_pc )+T f And calculating the time of the laser radar time system at the momentt l=(t pc -T pcl_)+T l。
iiii) receiving calibration delays from FPGA processing unitsdTAnd calculating the starting UTC time of the FPGA processing unit time systemT 0=t f -t 1+dt。
iiiii) calculate the exact time of the lidar data framet 1Corresponding FPGA processing unit time systemt real1_=t 1+dT+T 0。
In conclusion, the application has the beneficial effects that:
(1) according to the data synchronization time service system and method of the laser radar and the inertial sensor based on the FPGA, the PPS second time pulse generated by the GNSS time service module is combined with the calibration of the base clock module and the trigger calibration module in the FPGA processing unit to generate a high-precision IMU trigger signal and a corrected accurate and stable PPS second time pulse, so that the influence of instability of the PPS second time pulse generated by the GNSS time service module is avoided, meanwhile, the trigger time of the IMU inertial sensor and the laser radar can be accurately controlled, and the synchronous time service of the laser radar and the IMU and the high-precision time synchronization condition of a point cloud data set acquired by the laser radar and a data set acquired by the IMU are guaranteed;
(2) the industrial personal computer arranged in the system analyzes the calibrated laser radar data packet and the IMU data packet, so that the IMU data and the laser radar data analyzed by the industrial personal computer are ensured to complete data time alignment under the same time coordinate system, and the time synchronization precision is very high (for example, the synchronization precision error is less than 1 ns);
(3) the method and the device effectively solve the problems of poor reconstruction effect and even detection result caused by insufficient time synchronization precision of point cloud data and pose data in the traditional method.
Although the present invention has been described above in connection with the exemplary embodiments and the accompanying drawings, it will be apparent to those of ordinary skill in the art that various modifications may be made to the above-described embodiments without departing from the spirit and scope of the claims.

Claims (9)

1. A data synchronization time service method of a laser radar and an inertial sensor is characterized by comprising the following steps:
generating a first PPS (pulse per pulse) pulse and a GPRMC (general packet radio network) message at preset time intervals by using a satellite time service module;
analyzing the GPRMC message to obtain UTC time;
the method comprises the steps that a second PPS pulse is generated after a first PPS pulse is calibrated through an FPGA processing unit and is sent to a laser radar, so that the laser radar is triggered to start to collect point cloud data, and first counting time in a time system of the FPGA processing unit and built-in counting time in the time system of the laser radar are recorded, wherein the first counting time is the time for generating the second PPS pulse, and the built-in counting time is the time for the laser radar to start to collect the point cloud data;
generating an IMU (inertial measurement unit) triggering instruction according to the preset time offset parameter through the FPGA processing unit and transmitting the IMU triggering instruction to the inertial sensor so as to trigger the inertial sensor to start to acquire IMU data and recording second counting time under the time system of the FPGA processing unit, wherein the second counting time is the time for outputting a response signal by the inertial sensor;
and determining the second counting time as the real IMU acquisition time, solving the real Lidar acquisition time corresponding to each frame of point cloud data according to the first counting time and the built-in counting time, and carrying out one-to-one correspondence on the real Lidar acquisition time and the point cloud data.
2. The data synchronization time service method of the lidar and the inertial sensor, as recited in claim 1, wherein a frequency of generating the IMU trigger command is determined according to a data acquisition frequency of the inertial sensor.
3. The data synchronization timing method of the lidar and the inertial sensor according to claim 2, wherein the inertial sensor is triggered to acquire and transmit IMU data once every time the IMU trigger command is generated and transmitted, the IMU trigger command is a string of multi-byte commands based on a serial protocol.
4. The data synchronization time service method of the laser radar and the inertial sensor according to claim 1, wherein the first count time is determined by: setting a base counter, and taking the time count value of the generated second PPS time base counter as a laser radar time stamp; determining a first counting time according to the laser radar timestamp and the UTC time;
the second counting time is determined in the following manner: taking the time count value of a time-base counter of the response signal output by the inertial sensor as an IMU timestamp; a second count time is determined based on the IMU timestamp and the UTC time.
5. The data synchronization time service method of the laser radar and the inertial sensor as claimed in claim 4, wherein the calculation formula of the real Lidar acquisition time is as follows:
t f =(tpc-T f pc_)+T f ，
t 1=(t pc -T pc1_)+T 1，
T 0=t f -t 1+dt，
t real1_=t 1+dT+T 0，
in the formula,T 0for the initial moment of the FPGA processing unit,dTfor the calibration delay of the FPGA processing unit for the first PPS pulse,T f when the control unit receives the laser radar time stamp, the corresponding FPGA processing unit is at the time under the time system,T f_pc is a control unitWhen the laser radar time stamp is received, the corresponding control unit time stamp，T 1When the control unit receives the collected data of the laser radar and the corresponding time of the laser radar under the time system,T pc1_when the control unit receives the collected data of the laser radar, the corresponding control unit time stamp,dtfor the time elapsed since the control unit started receiving the lidar acquisition data,t pc to experiencedtThe time stamp of the later corresponding control unit,t f to experiencedtThe time of the post-FPGA processing unit time system,t 1to experiencedtThe time of the later laser radar is the time of the next laser radar,t real1_is composed oft 1And acquiring real Lidar time under a corresponding FPGA processing unit time system.
6. A data synchronization time service system of a laser radar and an inertial sensor is characterized by comprising a satellite time service module, an FPGA processing unit, the laser radar, the inertial sensor and a control unit,
the satellite time service module is used for outputting a first PPS pulse and a GPRMC message;
the FPGA processing unit is connected with the satellite time service module and used for generating a second PPS pulse and an IMU trigger instruction, sending the second PPS pulse to the laser radar, determining first counting time under the time system of the FPGA processing unit, sending the IMU trigger instruction to the inertial sensor, and determining second counting time under the time system of the FPGA processing unit, wherein the first counting time is the time for the FPGA processing unit to generate the second PPS pulse, and the second counting time is the time for the inertial sensor to output a response signal;
the laser radar is connected with the FPGA processing unit and used for collecting point cloud data after receiving the second PPS pulse and outputting the point cloud data and the built-in counting time under the laser radar time system;
the inertial sensor is connected with the FPGA processing unit and used for receiving IMU triggering instructions and then starting to acquire IMU data containing the pose of the unmanned aerial vehicle and outputting response signals;
the control unit is respectively connected with the FPGA processing unit and the laser radar and is used for solving the real Lidar acquisition time corresponding to each frame of point cloud data according to the first counting time and the built-in counting time and enabling the real Lidar acquisition time to be in one-to-one correspondence with the point cloud data.
7. The data synchronization time service system of the laser radar and the inertial sensor as claimed in claim 6, wherein the FPGA processing unit comprises a base clock module, a UTC analysis module, a trigger calibration module, a trigger instruction generation module, an IMU data analysis module and a splicing module, wherein,
the base clock module is configured to generate, in conjunction with the first PPS pulse, a base clock and a base counter for generating a lidar timestamp and an IMU timestamp in the FPGA processing unit time frame, the lidar timestamp corresponding to a count of time when the second PPS pulse was generated, the IMU timestamp corresponding to a count of time when the inertial sensor outputs the reply signal;
the UTC analysis module is configured to analyze the GPRMC message so as to extract UTC time and perform labeling operation on the UTC time;
the trigger calibration module is connected with the base clock module and is configured to calibrate the first PPS pulse based on the base clock to generate a second PPS pulse and generate an IMU trigger signal according to the offset parameter at the preset moment;
the trigger instruction generation module is connected with the trigger calibration module and is configured to generate an IMU trigger instruction based on an IMU trigger signal;
the IMU data analysis module is connected with the inertial sensor and is configured to analyze a response signal output by the inertial sensor to obtain IMU data;
the splicing module is connected with the IMU data analysis module and is configured to splice the analyzed IMU data and the IMU timestamp and perform labeling operation to obtain an IMU data packet.
8. The lidar and inertial sensor data synchronization timing system according to claim 7, wherein the FPGA processing unit further comprises a first buffer, a second buffer, a third buffer and a priority management module, wherein,
the first buffer area is connected with the splicing module and used for buffering and storing the laser radar time stamp;
the second buffer area is connected with the splicing module and is used for buffering and storing IMU data packets;
the third buffer area is connected with the UTC analysis module and used for buffering and storing UTC time;
the priority management module is respectively connected with the first buffer area, the second buffer area and the third buffer area and used for generating access signals according to a preset priority sequence to access the three buffer areas in sequence and generating serial data packets.
9. The lidar and inertial sensor data synchronization timing system of claim 6, wherein the control unit comprises a time synchronization module and a time coordinate alignment module, wherein,
the time synchronization module is configured to be capable of solving a real Lidar acquisition time corresponding to each frame of point cloud data according to the first counting time and the built-in counting time;
the time coordinate alignment module is configured to enable one-to-one correspondence of real Lidar acquisition times to point cloud data.
CN202210671990.6A 2022-06-15 2022-06-15 Data synchronization time service system and method for laser radar and inertial sensor Active CN114754769B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210671990.6A CN114754769B (en) 2022-06-15 2022-06-15 Data synchronization time service system and method for laser radar and inertial sensor

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210671990.6A CN114754769B (en) 2022-06-15 2022-06-15 Data synchronization time service system and method for laser radar and inertial sensor

Publications (2)

Publication Number Publication Date
CN114754769A true CN114754769A (en) 2022-07-15
CN114754769B CN114754769B (en) 2022-11-18

Family

ID=82336321

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210671990.6A Active CN114754769B (en) 2022-06-15 2022-06-15 Data synchronization time service system and method for laser radar and inertial sensor

Country Status (1)

Country Link
CN (1) CN114754769B (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115359131A (en) * 2022-10-20 2022-11-18 北京格镭信息科技有限公司 Calibration verification method, device, system, electronic equipment and storage medium
CN115840234A (en) * 2022-10-28 2023-03-24 苏州知至科技有限公司 Radar data acquisition method and device and storage medium
CN118363038A (en) * 2024-06-19 2024-07-19 山东科技大学 Time synchronization method for airborne laser radar and positioning and attitude determination system
CN118473584A (en) * 2024-07-15 2024-08-09 山东科技大学 Time synchronization method based on time delay design of airborne laser radar system

Citations (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107797129A (en) * 2017-10-13 2018-03-13 重庆市勘测院 Without the cloud data acquisition method and device under GNSS signal
CN108919238A (en) * 2018-07-18 2018-11-30 浙江大学 A kind of bearing calibration of rotary laser radar data and system based on Inertial Measurement Unit
WO2019018315A1 (en) * 2017-07-17 2019-01-24 Kaarta, Inc. Aligning measured signal data with slam localization data and uses thereof
CN110133999A (en) * 2019-04-23 2019-08-16 武汉星源云意科技有限公司 A kind of method for synchronizing time and clock synchronization system for adopting platform based on nebula laser point cloud number
CN110329273A (en) * 2019-06-18 2019-10-15 浙江大学 A kind of method and device synchronous for unmanned acquisition data
CN110456788A (en) * 2019-07-22 2019-11-15 中兴智能汽车有限公司 A kind of automatic Pilot bus controlling system and automatic Pilot car
CN110703586A (en) * 2019-09-09 2020-01-17 广州市中海达测绘仪器有限公司 Time synchronization method, data synchronization method, device, system, equipment and medium
CN111474532A (en) * 2020-04-10 2020-07-31 北京建筑大学 Time synchronization method and device for vehicle-mounted mobile laser radar measurement system
CN112230240A (en) * 2020-09-30 2021-01-15 深兰人工智能(深圳)有限公司 Space-time synchronization system, device and readable medium for laser radar and camera data
CN112861660A (en) * 2021-01-22 2021-05-28 上海西井信息科技有限公司 Laser radar array and camera synchronization device, method, equipment and storage medium
CN113267186A (en) * 2021-07-16 2021-08-17 成都纵横大鹏无人机科技有限公司 Data synchronous acquisition system and data synchronous acquisition method
WO2021204144A1 (en) * 2020-04-08 2021-10-14 阿里巴巴集团控股有限公司 Data processing system and method, sensor, mobile acquisition backpack, and device
US20210341631A1 (en) * 2020-04-30 2021-11-04 Baidu Usa Llc Dual inertial measurement units for inertial navigation system
US20210356915A1 (en) * 2019-03-29 2021-11-18 Beijing Voyager Technology Co., Ltd. Systems and methods for time synchronization
CN114499733A (en) * 2022-02-16 2022-05-13 东南大学 Four-legged robot-mounted SLAM device and sensor time synchronization method
CN114614934A (en) * 2022-03-28 2022-06-10 智己汽车科技有限公司 Time synchronization triggering device and method

Patent Citations (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019018315A1 (en) * 2017-07-17 2019-01-24 Kaarta, Inc. Aligning measured signal data with slam localization data and uses thereof
CN107797129A (en) * 2017-10-13 2018-03-13 重庆市勘测院 Without the cloud data acquisition method and device under GNSS signal
CN108919238A (en) * 2018-07-18 2018-11-30 浙江大学 A kind of bearing calibration of rotary laser radar data and system based on Inertial Measurement Unit
CN113708877A (en) * 2019-03-29 2021-11-26 北京航迹科技有限公司 Time synchronization system and method for vehicle
US20210356915A1 (en) * 2019-03-29 2021-11-18 Beijing Voyager Technology Co., Ltd. Systems and methods for time synchronization
CN110133999A (en) * 2019-04-23 2019-08-16 武汉星源云意科技有限公司 A kind of method for synchronizing time and clock synchronization system for adopting platform based on nebula laser point cloud number
CN110329273A (en) * 2019-06-18 2019-10-15 浙江大学 A kind of method and device synchronous for unmanned acquisition data
CN110456788A (en) * 2019-07-22 2019-11-15 中兴智能汽车有限公司 A kind of automatic Pilot bus controlling system and automatic Pilot car
CN110703586A (en) * 2019-09-09 2020-01-17 广州市中海达测绘仪器有限公司 Time synchronization method, data synchronization method, device, system, equipment and medium
WO2021204144A1 (en) * 2020-04-08 2021-10-14 阿里巴巴集团控股有限公司 Data processing system and method, sensor, mobile acquisition backpack, and device
CN111474532A (en) * 2020-04-10 2020-07-31 北京建筑大学 Time synchronization method and device for vehicle-mounted mobile laser radar measurement system
US20210341631A1 (en) * 2020-04-30 2021-11-04 Baidu Usa Llc Dual inertial measurement units for inertial navigation system
CN112230240A (en) * 2020-09-30 2021-01-15 深兰人工智能(深圳)有限公司 Space-time synchronization system, device and readable medium for laser radar and camera data
CN112861660A (en) * 2021-01-22 2021-05-28 上海西井信息科技有限公司 Laser radar array and camera synchronization device, method, equipment and storage medium
CN113267186A (en) * 2021-07-16 2021-08-17 成都纵横大鹏无人机科技有限公司 Data synchronous acquisition system and data synchronous acquisition method
CN114499733A (en) * 2022-02-16 2022-05-13 东南大学 Four-legged robot-mounted SLAM device and sensor time synchronization method
CN114614934A (en) * 2022-03-28 2022-06-10 智己汽车科技有限公司 Time synchronization triggering device and method

Non-Patent Citations (7)

* Cited by examiner, † Cited by third party
Title
JINHONG XU等: "Real-time LiDAR data assocation aided by IMU in high dynamic environment", 《2018 IEEE INTERNATIONAL CONFERENCE ON REAL-TIME COMPUTING AND ROBOTICS (RCAR)》 *
SOOJEONG LEE等: "Analysis and Compensation of Time Synchronization Error on SAR Image", 《ISSN》 *
WEI LI等: "KFS-LIO: Key-Feature Selection for Lightweight Lidar Inertial Odometry", 《2021 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION (ICRA)》 *
姚利红: "基于FPGA的电力系统GPS/北斗时间同步装置", 《中国优秀博硕士学位论文全文数据库(硕士) 基础科学辑》 *
宋俊男等: "地外探测设备多传感器硬件时间同步方法研究", 《空间控制技术与应用》 *
郭明等: "北斗导航授时的移动激光雷达测量系统时间同步装置设计", 《红外与激光工程》 *
黄战华等: "基于TOF的海潮高度在线测量系统设计", 《遥感技术学报》 *

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115359131A (en) * 2022-10-20 2022-11-18 北京格镭信息科技有限公司 Calibration verification method, device, system, electronic equipment and storage medium
CN115359131B (en) * 2022-10-20 2023-02-03 北京格镭信息科技有限公司 Calibration verification method, device, system, electronic equipment and storage medium
CN115840234A (en) * 2022-10-28 2023-03-24 苏州知至科技有限公司 Radar data acquisition method and device and storage medium
CN115840234B (en) * 2022-10-28 2024-04-19 苏州知至科技有限公司 Radar data acquisition method, device and storage medium
CN118363038A (en) * 2024-06-19 2024-07-19 山东科技大学 Time synchronization method for airborne laser radar and positioning and attitude determination system
CN118473584A (en) * 2024-07-15 2024-08-09 山东科技大学 Time synchronization method based on time delay design of airborne laser radar system

Also Published As

Publication number Publication date
CN114754769B (en) 2022-11-18

Similar Documents

Publication Publication Date Title
CN114754769B (en) Data synchronization time service system and method for laser radar and inertial sensor
US11385359B2 (en) Point cloud data acquisition method and device under situation of no GNSS signal
CN109104259B (en) Multi-sensor time synchronization system and method
CN110133999B (en) Time synchronization method and system based on satellite cloud laser point cloud data acquisition platform
US10789771B2 (en) Method and apparatus for fusing point cloud data
CN103744372B (en) The multisensor method for synchronizing time of unmanned plane electric inspection process and system
WO2021204144A1 (en) Data processing system and method, sensor, mobile acquisition backpack, and device
CN107807365A (en) Small-sized digital photography there-dimensional laser scanning device for the unmanned airborne vehicle in low latitude
CN114755693B (en) Infrastructure facility measuring system and method based on multi-rotor unmanned aerial vehicle
CN111007554A (en) Data acquisition time synchronization system and method
CN110865406A (en) Multi-sensor data synchronous processing system and method based on vehicle-mounted GPS time service system
CN114964175B (en) Multi-sensor data synchronous acquisition device and acquisition method
CN108282242B (en) Space-time synchronization device and system, map acquisition vehicle and map intelligent production system
CN111860604A (en) Data fusion method, system and computer storage medium
CN111343379A (en) FPGA-based high-speed data acquisition device and method
CN105319567B (en) A kind of laser gyro position attitude system method for synchronizing time
CN212012915U (en) Binocular image and IMU data high-speed acquisition device
CN116577802A (en) Unmanned aerial vehicle platform multisensor data synchronous acquisition system based on ZYNQ
CN111193568A (en) Time synchronization method, device, system, storage medium and vehicle
CN112398559A (en) Clock synchronization method, device, system and storage medium
CN116032412B (en) Multi-camera cross-platform time synchronization method, device and system and electronic equipment
CN102901517B (en) Measurement system based on time synchronization for rocket sled test
CN115389246B (en) Speed precision measuring method, system and device of motion capture system
WO2020113370A1 (en) Air vehicle, data processing system, and data processing method for air vehicle
CN111457917A (en) Multi-sensor time synchronization measuring method and system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
CB03 Change of inventor or designer information
CB03 Change of inventor or designer information

Inventor after: Wang Kangnian

Inventor after: Huang Zhanhua

Inventor after: Pan Cheng

Inventor after: Li Xiaowei

Inventor before: Huang Zhanhua

Inventor before: Li Xiaowei

Inventor before: Pan Cheng

Inventor before: Wang Kangnian

GR01 Patent grant
GR01 Patent grant