CN113922910A - Sensor time synchronization processing method, device and system - Google Patents
Sensor time synchronization processing method, device and system Download PDFInfo
- Publication number
- CN113922910A CN113922910A CN202111175336.8A CN202111175336A CN113922910A CN 113922910 A CN113922910 A CN 113922910A CN 202111175336 A CN202111175336 A CN 202111175336A CN 113922910 A CN113922910 A CN 113922910A
- Authority
- CN
- China
- Prior art keywords
- time
- sensor
- service module
- calibration
- analysis
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04J—MULTIPLEX COMMUNICATION
- H04J3/00—Time-division multiplex systems
- H04J3/02—Details
- H04J3/06—Synchronising arrangements
- H04J3/0635—Clock or time synchronisation in a network
- H04J3/0638—Clock or time synchronisation among nodes; Internode synchronisation
- H04J3/0644—External master-clock
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Position Fixing By Use Of Radio Waves (AREA)
Abstract
The application relates to a sensor time synchronization processing method, device and system. The sensor time synchronization processing method comprises the following steps: acquiring time information data sent by a satellite navigation system time service module; analyzing the time information data to obtain analysis time; and respectively calibrating the time of different sensors according to the analysis time to achieve time synchronization. The scheme provided by the application can realize the time synchronization of different sensors, and can also reduce the cost and improve the precision of the time synchronization.
Description
Technical Field
The present application relates to the field of data processing technologies, and in particular, to a method, an apparatus, and a system for processing sensor time synchronization.
Background
At present, a flying automobile can acquire environmental data through sensors such as a laser radar and a camera, and assists in flying through analysis of the environmental data. The laser radar can collect point cloud data, and the camera can collect image data.
In order to fuse the point cloud data of the laser radar and the image data of the camera, the time of the laser radar and the camera needs to be synchronized. In the related art, an external FPGA (Field Programmable Gate Array) chip may be used as a unified clock source of the laser radar and the camera to time the radar and the camera, so as to achieve time synchronization of the radar and the camera.
However, the method needs to add an additional FPGA chip, and the cost is high.
Disclosure of Invention
In order to solve or partially solve the problems in the related art, the application provides a sensor time synchronization processing method, device and system, which can realize time synchronization of different sensors, reduce cost and improve time synchronization precision.
The application provides a sensor time synchronization processing method in a first aspect, including:
acquiring time information data sent by a satellite navigation system time service module;
analyzing the time information data to obtain analysis time;
and respectively calibrating the time of different sensors according to the analysis time to achieve time synchronization.
In one embodiment, the calibrating the time of different sensors respectively according to the analytic time to achieve time synchronization includes:
sending the analysis time to a first sensor, so that the first sensor takes the analysis time as calibration time after receiving a hardware pulse sent by the satellite navigation system time service module, and sends first sensing data according to the calibration time;
and sending the analysis time to a second sensor, so that the second sensor takes the analysis time as calibration time after receiving the hardware pulse sent by the satellite navigation system time service module, and sends second sensing data according to the calibration time.
In one embodiment, the calibrating the time of different sensors respectively according to the analytic time to achieve time synchronization includes:
sending the analysis time to a first sensor, so that the first sensor takes the analysis time as calibration time after receiving a hardware pulse sent by the satellite navigation system time service module, and sends first sensing data according to the calibration time;
after receiving the hardware pulse sent by the satellite navigation system time service module, adding a set time to the analysis time to serve as self calibration time, marking a local timestamp by referring to the self calibration time of the second sensor sent after the second sensor receives the hardware pulse sent by the satellite navigation system time service module, and determining the calibration time of the second sensor according to the local timestamp and the time parameter transmitted by the second sensor data.
In one embodiment, the determining the calibration time of the second sensor according to the local timestamp and the time parameter of the second sensing data transmission includes:
determining the added value of the exposure time, the reading time and the transmission time of the second sensing data;
determining the exposure time of the second sensing data according to the difference value between the local timestamp and the added value;
and searching corresponding analysis time according to the exposure moment, and taking the searched analysis time as the calibration time of the second sensor.
In one embodiment, the first sensor is a radar and the second sensor is a camera.
In one embodiment, the satellite navigation system time service module is a GPS time service module or a beidou time service module.
The second aspect of the present application provides a sensor time synchronization processing apparatus, including:
the acquisition module is used for acquiring time information data sent by the satellite navigation system time service module;
the analysis module is used for analyzing the time information data acquired by the acquisition module to obtain analysis time;
and the calibration module is used for respectively calibrating the time of different sensors according to the analysis time obtained by the analysis module so as to achieve time synchronization.
A third aspect of the present application provides a sensor time synchronization processing system, including:
the satellite navigation system time service module is used for sending time information data to the main control unit;
the main control unit is used for acquiring time information data sent by the satellite navigation system time service module; analyzing the time information data to obtain analysis time; respectively calibrating the time of different sensors according to the analysis time to achieve time synchronization;
the first sensor is used for being matched with the main control unit to calibrate the time of the first sensor;
and the second sensor is used for matching with the main control unit to calibrate the time of the second sensor.
In one embodiment, after receiving a hardware pulse sent by the satellite navigation system time service module, the first sensor takes the analysis time sent by the main control unit as calibration time, and sends first sensing data according to the calibration time;
and after receiving the hardware pulse sent by the satellite navigation system time service module, the second sensor takes the analysis time sent by the main control unit as calibration time, and sends second sensing data according to the calibration time.
In one embodiment, after receiving a hardware pulse sent by the satellite navigation system time service module, the first sensor takes the analysis time sent by the main control unit as calibration time, and sends first sensing data according to the calibration time;
the main control unit increases the analysis time by a set time after receiving the hardware pulse sent by the satellite navigation system time service module to be used as self calibration time, marks a local timestamp with reference to the self calibration time after receiving second sensing data sent by a second sensor after receiving the hardware pulse sent by the satellite navigation system time service module, and determines the calibration time of the second sensor according to the local timestamp and the time parameter transmitted by the second sensing data.
In one embodiment, the first sensor is a radar and the second sensor is a camera.
A fourth aspect of the present application provides a computer-readable storage medium having stored thereon executable code, which, when executed by a processor of an electronic device, causes the processor to perform the method as described above.
The technical scheme provided by the application can comprise the following beneficial effects:
according to the scheme, a satellite navigation system time service module is uniformly used as a time service clock source, and after time information data sent by the satellite navigation system time service module is obtained, the time information data is analyzed to obtain analysis time; and respectively calibrating the time of different sensors according to the analysis time to achieve time synchronization, so that the different sensors can realize time synchronization according to the same time service clock source, and the method is lower in cost and higher in synchronization precision.
According to the technical scheme, the first sensor can be a radar, the second sensor can be a camera, the analysis time can be sent to the first sensor, so that after the first sensor receives a hardware pulse sent by the satellite navigation system time service module, the analysis time is used as calibration time, and first sensing data are sent according to the calibration time; and sending the analysis time to a second sensor so that the second sensor takes the analysis time as calibration time after receiving a hardware pulse sent by the satellite navigation system time service module, and sending second sensing data according to the calibration time, so that different sensors can be subjected to time synchronization by combining a soft synchronization mode and a hard synchronization mode, and the synchronization precision is higher.
According to the technical scheme, the analysis time can be sent to a first sensor, so that the first sensor takes the analysis time as calibration time after receiving a hardware pulse sent by the satellite navigation system time service module, and sends first sensing data according to the calibration time; after receiving the hardware pulse sent by the satellite navigation system time service module, adding a set time to the analysis time to serve as self calibration time, marking a local timestamp by referring to the self calibration time of the received second sensor after receiving the hardware pulse sent by the satellite navigation system time service module, and determining the calibration time of the second sensor according to the local timestamp and the time parameter transmitted by the second sensor data.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.
Drawings
The foregoing and other objects, features and advantages of the application will be apparent from the following more particular descriptions of exemplary embodiments of the application as illustrated in the accompanying drawings wherein like reference numbers generally represent like parts throughout the exemplary embodiments of the application.
FIG. 1 is a schematic flow chart diagram illustrating a sensor time synchronization processing method according to an embodiment of the present disclosure;
FIG. 2 is another schematic flow chart diagram illustrating a sensor time synchronization processing method according to an embodiment of the present disclosure;
FIG. 3 is another schematic flow chart diagram illustrating a sensor time synchronization processing method according to an embodiment of the present disclosure;
FIG. 4 is a schematic diagram of a system application architecture of a sensor time synchronization processing method according to an embodiment of the present application;
fig. 5 is a schematic view of an application architecture of a GPS time service module and a radar for implementing time synchronization according to an embodiment of the present application;
fig. 6 is a schematic view of an application architecture for implementing time synchronization between a GPS time service module and a main control unit according to an embodiment of the present application;
FIG. 7 is a schematic diagram of an application architecture of a GPS time service module and a camera for implementing time synchronization according to an embodiment of the present application;
FIG. 8 is a schematic diagram of another application architecture of the GPS time service module and the camera for implementing time synchronization according to the embodiment of the present application;
fig. 9 is a schematic structural diagram of a sensor time synchronization processing apparatus according to an embodiment of the present application;
FIG. 10 is a schematic structural diagram of a sensor time synchronization processing system according to an embodiment of the present disclosure;
fig. 11 is a schematic structural diagram of an electronic device shown in an embodiment of the present application.
Detailed Description
Embodiments of the present application will be described in more detail below with reference to the accompanying drawings. While embodiments of the present application are illustrated in the accompanying drawings, it should be understood that the present application may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in this application and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.
It should be understood that although the terms "first," "second," "third," etc. may be used herein to describe various information, these information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present application. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of the present application, "a plurality" means two or more unless specifically limited otherwise.
In the related technology, the time synchronization of the radar and the camera is realized, an additional FPGA chip is needed, and the cost is high. In view of the above problems, the present application provides a sensor time synchronization processing method, which can implement time synchronization of different sensors, and can also reduce cost and improve accuracy of time synchronization.
The technical solutions of the embodiments of the present application are described in detail below with reference to the accompanying drawings.
Fig. 1 is a schematic flowchart of a sensor time synchronization processing method according to an embodiment of the present application.
Referring to fig. 1, the method includes:
in S101, time information data transmitted by the satellite navigation system time service module is acquired.
The satellite navigation system time service module can be a GPS time service module or a Beidou time service module. The main control unit may obtain time information data sent by a time service module of the satellite navigation system, where the time information data may be, for example, NEMA (National Marine Electronics Association) data, and the NEMA data may have time information of a whole second accurate to ns level.
In S102, the time information data is analyzed to obtain an analysis time.
After receiving the NEMA data, the main control unit analyzes the GPS time as analysis time and stores the analysis time in a file system.
In S103, the times of the different sensors are respectively calibrated according to the analysis time to achieve time synchronization.
Wherein the different sensors may comprise, for example, a first sensor and a second sensor. Wherein the first sensor may be a radar and the second sensor may be a camera.
In this step, the main control unit may send the analysis time to the first sensor, so that the first sensor takes the analysis time as the calibration time after receiving the hardware pulse sent by the satellite navigation system time service module, and sends the first sensing data according to the calibration time; and sending the analysis time to a second sensor so that the second sensor takes the analysis time as calibration time after receiving the hardware pulse sent by the satellite navigation system time service module, and sending second sensing data according to the calibration time.
In the step, the analysis time can also be sent to the first sensor, so that the first sensor takes the analysis time as calibration time after receiving a hardware pulse sent by a satellite navigation system time service module, and sends first sensing data according to the calibration time; after receiving the hardware pulse sent by the satellite navigation system time service module, adding the analysis time by a set time to be used as self calibration time, marking a local timestamp by referring to the self calibration time of the second sensor which is received and sent after receiving the hardware pulse sent by the satellite navigation system time service module, and determining the calibration time of the second sensor according to the local timestamp and the time parameter of the second sensor data transmission.
Wherein determining the calibration time of the second sensor based on the local timestamp and the time parameter of the second sensor data transmission comprises: determining the added value of the exposure time, the reading time and the transmission time of the second sensing data; determining the exposure time of the second sensing data according to the difference value between the local timestamp and the added value; and searching corresponding analysis time according to the exposure time, and taking the searched analysis time as the calibration time of the second sensor.
According to the embodiment, the satellite navigation system time service module is uniformly used as a time service clock source, and after time information data sent by the satellite navigation system time service module is obtained, the time information data is analyzed to obtain analysis time; and respectively calibrating the time of different sensors according to the analysis time to achieve time synchronization, so that the different sensors can realize time synchronization according to the same time service clock source, and the method is lower in cost and higher in synchronization precision.
In the following, the present application is described in further detail, in which the first sensor is a radar, the second sensor is a camera, the first sensing data is point cloud data, the second sensing data is image data, and the satellite navigation system time service module is a GPS time service module, for example, but not limited thereto, the satellite navigation system time service module may also be a beidou time service module, the sensor may also be another sensor other than a radar or a camera, and the number of each sensor may be 1 or more.
Fig. 2 is another schematic flow chart of a sensor time synchronization processing method according to an embodiment of the present application. In the flow shown in fig. 2, time synchronization of the radar and the camera can be achieved.
Referring to fig. 4, fig. 4 is a schematic diagram of a system application architecture of a sensor time synchronization processing method according to an embodiment of the present application. The system application architecture mainly comprises a GPS time service module, a laser radar module, a binocular camera module, a main control unit, a pulse frequency multiplier and the like. In fig. 4, the radar is a laser radar, and the camera is a binocular camera for illustration but not limitation. Wherein, laser radar and binocular camera all support hardware to trigger. The GPS time service module is used as a clock source of the whole system and is responsible for triggering the exposure of the binocular camera and calibrating the time of the laser radar and the main control unit. The main control unit is used as a core working module and is responsible for data acquisition of each sensor, receiving and analyzing NEMA data, completing self time calibration, completing time calibration of the laser radar and the binocular camera, and finally completing synchronization of the laser radar and the binocular camera. The laser radar sends point cloud data to the main control unit, and the binocular camera sends image data to the main control unit.
Referring to fig. 2, the method includes:
in S201, the GPS time service module sends hardware pulses to the radar and the camera, and sends the hardware pulses and time information data to the main control unit.
Referring also to fig. 5, the GPS time service module sends out hardware pulses and time information data, wherein the sent out hardware pulses may be, for example, PPS (Pulse Per Second) sending out 1HZ, the sent out time information data may be, for example, NEMA (National Marine Electronics Association) data, and the NEMA data may have time information of whole Second accurate to ns level. Wherein the PPS signal of 1HZ can be sent to the radar and the main control unit, respectively, and the PPS signal of the camera can reach 30HZ through the frequency multiplier.
The PPS port of the GPS timing module sends a hardware pulse (PPS signal) once per second, and then the data port sends NEMA data corresponding to the rising edge of this pulse.
In S202, the main control unit obtains time information data sent by the GPS time service module.
The main control unit can create a plurality of independent threads on an application layer, wherein the independent threads are respectively used for receiving data of a radar, data of a camera, NEMA data of a GPS time service module and the like.
In S203, the main control unit analyzes the time information data to obtain an analysis time.
After receiving NEMA data, the main control unit parses out correct GPS Time and stores the GPS Time as parsing Time in a file system, where the parsing Time may also be referred to as Universal Time Coordinated (UTC).
In S204, the main control unit transmits the resolution time to the radar.
The main control unit simultaneously sends the analyzed GPS time (UTC time) to the radar, namely sends the analyzed time to the radar.
In S205, after receiving the hardware pulse sent by the GPS time service module, the radar uses the analysis time as the calibration time, and sends the point cloud data according to the calibration time.
After the radar receives the PPS signal and the GPS time, the time of point cloud data is updated to be the GPS time, namely the GPS time is used as calibration time, and the point cloud data is continuously accumulated and output according to the time reference, so that the time synchronization of the GPS and the radar is completed.
More specifically, after the radar receives the rising edge of the PPS signal and the GPS time sent by the main control unit, the time of the point cloud data is set to be the GPS time, that is, the time of the output point cloud data is the time after calibration, and the time reference is kept to be continuously accumulated, so that the time synchronization of the GPS device and the radar is realized.
In S206, after receiving the hardware pulse sent by the GPS time service module, the main control unit increases the analysis time by a set time to obtain a self-calibration time.
Referring to fig. 6, the main control unit may be driven by modifying a GPIO (General Purpose Input/Output) for receiving a PPS signal Output by the GPS time service module. After receiving the PPS signal, the GPIO of the main control unit enters interruption, the recorded time t (second) of the last PPS signal is obtained from the file system, and the system time of the main control unit is updated by using the time t +1 (second) as calibration time, so that the time synchronization of the GPS time service module and the main control unit is completed.
For example, after receiving NEMA data, the USB driver of the core layer of the main control unit analyzes the GPS time at the current time, for example, N time, through the application layer and stores the GPS time in the file system, the GPIO driver of the core layer of the main control unit detects the rising edge of the PPS signal at N +1 time, and uses the GPS time +1 second at N time as the calibration time and updates the system clock.
Specifically, a PPS port of the GPS time service module sends a hardware pulse (PPS signal) once per second, then the data port sends NEMA data corresponding to a rising edge of the pulse once, and the main control unit analyzes correct GPS time after receiving the NEMA data and stores the GPS time in a file system. And after the main control unit detects that the GPIO continuously receives two PPS signals, adding 1 second to the previous PPS signal time as calibration time, and updating the system time of the main control unit to realize time synchronization of the time service module and the main control unit.
It should be noted that, in this embodiment, synchronization between the radar and the GPS time service module does not depend on time synchronization of the main control unit, but the GPS time service module and the camera need to be calibrated first.
In S207, the main control unit refers to the local calibration time of the received image data sent by the camera after receiving the hardware pulse sent by the GPS time service module, and determines the calibration time of the camera according to the local time stamp and the time parameter of image data transmission.
Wherein the sum of the exposure time, the readout time and the transmission time of the image data can be determined; determining the exposure time of the image data according to the difference value of the local timestamp and the added value; and searching corresponding analysis time according to the exposure time, and taking the searched analysis time as the calibration time of the camera.
Referring to fig. 7, after the camera data receiving thread in the main control unit acquires the image data sent by the camera, a local timestamp T4 is marked, the exposure time T4- (T1+ T2+ T3) of the current image frame is estimated according to the known or calculated exposure time T1, reading time T2 and transmission time T3, the whole second time of the GPS, that is, the corresponding resolution time, is searched for by using the exposure time T, and if the matching is completed, the GPS time is marked to the current image data, and the searched resolution time is used as the calibration time of the camera. And the next image frame within 1 second can be increased by 1/fps based on the whole second time of the GPS, so that the time synchronization of the GPS time service module and the camera is completed.
The PPS signal of 1HZ of GPS can obtain a pulse of 30HZ through a frequency multiplier to be used as an exposure trigger signal of the camera. The camera receives the pulse signal of 30HZ, carries out image exposure according to the frequency and outputs image data to the main control unit. The GPS time service module needs to complete the local clock calibration of the main control unit.
It should be noted that, in the above steps, the time synchronization steps of the radar and the camera may be performed simultaneously.
In summary, in the embodiment, the time of the radar, the system time of the main control unit, and the time of the camera are respectively unified under the GPS time frame through the above steps, so as to complete the time hard synchronization of the radar and the camera. In the scheme of the embodiment, the GPS time service module is used, the radar and the camera are subjected to time synchronization in a mode of calibrating a system clock of the main control unit and hard synchronization, the accuracy of us level can be achieved, the accuracy is higher compared with a soft synchronization mode, and the cost is lower compared with an FPGA mode under the same accuracy.
Fig. 3 is another schematic flow chart of a sensor time synchronization processing method according to an embodiment of the present application. The difference between the flow of fig. 3 and the flow of fig. 2 is mainly the time synchronization processing manner of the camera.
Referring to fig. 3, the method includes:
in S301, the GPS time service module sends hardware pulses to the radar and the camera, and sends the hardware pulses and time information data to the main control unit.
This step S301 may refer to the description in S201, and is not described herein again.
In S302, the main control unit obtains time information data sent by the GPS time service module.
This step S302 can refer to the description in S202, and is not described herein again.
In S303, the main control unit analyzes the time information data to obtain an analysis time.
This step S303 can refer to the description in S203, and is not described herein again.
In S304, the master control unit transmits the resolution time to the radar.
The step S304 can refer to the description in S204, and is not described herein again.
In S305, after receiving the hardware pulse transmitted by the GPS time service module, the radar uses the analysis time as the calibration time, and transmits the point cloud data according to the calibration time.
This step S305 can refer to the description in S205, and is not described herein again.
In S306, the main control unit transmits the resolution time to the camera.
Referring to fig. 8, according to different camera models, if the camera is internally provided with a high-precision clock and has a synchronous interface to support capturing PPS signals and receiving UTC time, the time stamp of the image of the camera can be corrected every second, and the master control unit can directly send the analysis time to the camera. That is, if the camera side can perform synchronization, the main control unit is not required to realize time synchronization with the GPS time service module first. It should be noted that, generally, when the requirement of us-level time synchronization is to be achieved, the camera is required to have a high-precision clock, and if the requirement of us-level time synchronization is not to be achieved, the camera may not have a high-precision clock.
In S307, the camera receives the hardware pulse transmitted by the GPS time service module, then uses the analysis time as the calibration time, and transmits the image data according to the calibration time.
After the camera receives the PPS signal and the GPS time, the time of the image data is updated to be the GPS time, namely the GPS time is taken as the calibration time, the image data is continuously accumulated and output according to the time reference, and the time synchronization of the GPS and the camera is completed.
More specifically, after the camera receives the rising edge of the PPS signal and the GPS time sent by the main control unit, the camera sets the time of the image data as the GPS time, that is, the time of the output image data is the time after calibration, and keeps the time reference for continuous accumulation, thereby realizing time synchronization of the GPS device and the camera.
It should be noted that, in the above steps, the time synchronization steps of the radar and the camera may be performed simultaneously.
In summary, in the embodiment, the time of the radar and the time of the camera are respectively unified under the GPS time frame through the above steps, so that the time hard synchronization of the radar and the camera is completed. In the scheme of the embodiment, the GPS time service module is used, and the radar and the camera are subjected to time synchronization in a hard synchronization mode, so that the us-level precision can be achieved, and the cost can be reduced.
Corresponding to the embodiment of the application function implementation method, the application also provides a sensor time synchronization processing device, a system, an electronic device and a corresponding embodiment.
Fig. 9 is a schematic structural diagram of a sensor time synchronization processing apparatus according to an embodiment of the present application. The sensor time synchronization processing means may be located at the master control unit.
Referring to fig. 9, a sensor time synchronization processing device 80 includes: an acquisition module 81, an analysis module 82, and a calibration module 83.
And the obtaining module 81 is configured to obtain time information data sent by the satellite navigation system time service module. The satellite navigation system time service module can be a GPS time service module or a Beidou time service module.
And an analyzing module 82, configured to analyze the time information data acquired by the acquiring module 81 to obtain an analysis time.
And the calibration module 83 is configured to calibrate the times of the different sensors respectively according to the analysis time obtained by the analysis module 82 to achieve time synchronization. Wherein the different sensors may comprise, for example, a first sensor and a second sensor. Wherein the first sensor may be a radar and the second sensor may be a camera.
The calibration module 83 may send the analysis time to the first sensor, so that the first sensor takes the analysis time as the calibration time after receiving the hardware pulse sent by the satellite navigation system time service module, and sends the first sensing data according to the calibration time;
the calibration module 83 may increase the analysis time by a set time after receiving the hardware pulse sent by the satellite navigation system time service module, and then use the analysis time as a self calibration time, mark a local timestamp with reference to the self calibration time of the second sensor received the second sensing data sent by the second sensor after receiving the hardware pulse sent by the satellite navigation system time service module, and determine the calibration time of the second sensor according to the local timestamp and a time parameter transmitted by the second sensing data.
According to the embodiment, the satellite navigation system time service module is uniformly used as a time service clock source, and after time information data sent by the satellite navigation system time service module is obtained, the time information data is analyzed to obtain analysis time; and respectively calibrating the time of different sensors according to the analysis time to achieve time synchronization, so that the different sensors can realize time synchronization according to the same time service clock source, and the method is lower in cost and higher in synchronization precision.
Fig. 10 is a schematic structural diagram of a sensor time synchronization processing system according to an embodiment of the present application.
Referring to fig. 10, a sensor time synchronization processing system 90 includes: the system comprises a satellite navigation system time service module 91, a main control unit 92, a first sensor 93 and a second sensor 94.
The satellite navigation system time service module 91 is configured to send time information data to the main control unit 92.
The main control unit 92 is configured to obtain time information data sent by the satellite navigation system time service module 91; analyzing the time information data to obtain analysis time; and respectively calibrating the time of different sensors according to the analysis time to achieve time synchronization.
And a first sensor 93 for cooperating with the main control unit 92 to calibrate the timing of the first sensor.
A second sensor 94 for cooperating with the master control unit 92 to time-calibrate the second sensor.
The satellite navigation system time service module 91 may be a GPS time service module or a beidou time service module, the first sensor 93 may be a radar, and the second sensor 94 may be a camera.
In one embodiment, after receiving the hardware pulse sent by the satellite navigation system time service module 91, the first sensor 93 takes the analysis time sent by the main control unit 92 as the calibration time, and sends the first sensing data according to the calibration time; after receiving the hardware pulse sent by the satellite navigation system time service module 91, the second sensor 94 takes the analysis time sent by the main control unit 92 as the calibration time, and sends the second sensing data according to the calibration time. The first sensing data may be point cloud data and the second sensing data may be image data.
In one embodiment, after receiving the hardware pulse sent by the satellite navigation system time service module 91, the first sensor 93 takes the analysis time sent by the main control unit 92 as the calibration time, and sends the first sensing data according to the calibration time; the main control unit 92 increases the analysis time by a set time after receiving the hardware pulse sent by the satellite navigation system time service module 91 to be used as self calibration time, marks a local timestamp with reference to the self calibration time of the received second sensing data sent by the second sensor 94 after receiving the hardware pulse sent by the satellite navigation system time service module 91, and determines the calibration time of the second sensor 94 according to the local timestamp and the time parameter transmitted by the second sensing data.
Wherein determining the calibration time of the second sensor based on the local timestamp and the time parameter of the second sensor data transmission comprises: determining the added value of the exposure time, the reading time and the transmission time of the second sensing data; determining the exposure time of the second sensing data according to the difference value between the local timestamp and the added value; and searching corresponding analysis time according to the exposure time, and taking the searched analysis time as the calibration time of the second sensor.
It can be seen from this embodiment that, the system provided in the present application unifies the time of the radar, the system time of the main control unit, and the time of the camera into the GPS time frame, respectively, and completes the time hard synchronization of the radar and the camera. In the scheme of the embodiment, the GPS time service module is used, the radar and the camera are subjected to time synchronization in a mode of calibrating a system clock of the main control unit and hard synchronization, the accuracy of us level can be achieved, the accuracy is higher compared with a soft synchronization mode, and the cost is lower compared with an FPGA mode under the same accuracy.
With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.
Fig. 11 is a schematic structural diagram of an electronic device shown in an embodiment of the present application.
Referring to fig. 11, the electronic device 1000 includes a memory 1010 and a processor 1020.
The Processor 1020 may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory 1010 may include various types of storage units, such as system memory, Read Only Memory (ROM), and permanent storage. Wherein the ROM may store static data or instructions that are needed by the processor 1020 or other modules of the computer. The persistent storage device may be a read-write storage device. The persistent storage may be a non-volatile storage device that does not lose stored instructions and data even after the computer is powered off. In some embodiments, the persistent storage device employs a mass storage device (e.g., magnetic or optical disk, flash memory) as the persistent storage device. In other embodiments, the permanent storage may be a removable storage device (e.g., floppy disk, optical drive). The system memory may be a read-write memory device or a volatile read-write memory device, such as a dynamic random access memory. The system memory may store instructions and data that some or all of the processors require at runtime. Further, the memory 1010 may comprise any combination of computer-readable storage media, including various types of semiconductor memory chips (e.g., DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), magnetic and/or optical disks, among others. In some embodiments, memory 1010 may include a removable storage device that is readable and/or writable, such as a Compact Disc (CD), a digital versatile disc read only (e.g., DVD-ROM, dual layer DVD-ROM), a Blu-ray disc read only, an ultra-dense disc, a flash memory card (e.g., SD card, min SD card, Micro-SD card, etc.), a magnetic floppy disk, or the like. Computer-readable storage media do not contain carrier waves or transitory electronic signals transmitted by wireless or wired means.
The memory 1010 has stored thereon executable code that, when processed by the processor 1020, may cause the processor 1020 to perform some or all of the methods described above.
Furthermore, the method according to the present application may also be implemented as a computer program or computer program product comprising computer program code instructions for performing some or all of the steps of the above-described method of the present application.
Alternatively, the present application may also be embodied as a computer-readable storage medium (or non-transitory machine-readable storage medium or machine-readable storage medium) having executable code (or a computer program or computer instruction code) stored thereon, which, when executed by a processor of an electronic device (or server, etc.), causes the processor to perform part or all of the various steps of the above-described method according to the present application.
Having described embodiments of the present application, the foregoing description is intended to be exemplary, not exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen in order to best explain the principles of the embodiments, the practical application, or improvements made to the technology in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.
Claims (12)
1. A sensor time synchronization processing method is characterized by comprising the following steps:
acquiring time information data sent by a satellite navigation system time service module;
analyzing the time information data to obtain analysis time;
and respectively calibrating the time of different sensors according to the analysis time to achieve time synchronization.
2. The method of claim 1, wherein separately calibrating the time of different sensors to achieve time synchronization based on the resolved time comprises:
sending the analysis time to a first sensor, so that the first sensor takes the analysis time as calibration time after receiving a hardware pulse sent by the satellite navigation system time service module, and sends first sensing data according to the calibration time;
and sending the analysis time to a second sensor, so that the second sensor takes the analysis time as calibration time after receiving the hardware pulse sent by the satellite navigation system time service module, and sends second sensing data according to the calibration time.
3. The method of claim 1, wherein separately calibrating the time of different sensors to achieve time synchronization based on the resolved time comprises:
sending the analysis time to a first sensor, so that the first sensor takes the analysis time as calibration time after receiving a hardware pulse sent by the satellite navigation system time service module, and sends first sensing data according to the calibration time;
after receiving the hardware pulse sent by the satellite navigation system time service module, adding a set time to the analysis time to serve as self calibration time, marking a local timestamp by referring to the self calibration time of the second sensor sent after the second sensor receives the hardware pulse sent by the satellite navigation system time service module, and determining the calibration time of the second sensor according to the local timestamp and the time parameter transmitted by the second sensor data.
4. The method of claim 3, wherein determining the calibration time of the second sensor based on the local timestamp and the time parameter of the second sensory data transmission comprises:
determining the added value of the exposure time, the reading time and the transmission time of the second sensing data;
determining the exposure time of the second sensing data according to the difference value between the local timestamp and the added value;
and searching corresponding analysis time according to the exposure moment, and taking the searched analysis time as the calibration time of the second sensor.
5. The method according to any one of claims 2 to 4, wherein:
the first sensor is a radar and the second sensor is a camera.
6. The method according to any one of claims 1 to 4, characterized in that:
the satellite navigation system time service module is a GPS time service module or a Beidou time service module.
7. A sensor time synchronization processing apparatus, comprising:
the acquisition module is used for acquiring time information data sent by the satellite navigation system time service module;
the analysis module is used for analyzing the time information data acquired by the acquisition module to obtain analysis time;
and the calibration module is used for respectively calibrating the time of different sensors according to the analysis time obtained by the analysis module so as to achieve time synchronization.
8. A sensor time synchronization processing system, comprising:
the satellite navigation system time service module is used for sending time information data to the main control unit;
the main control unit is used for acquiring time information data sent by the satellite navigation system time service module; analyzing the time information data to obtain analysis time; respectively calibrating the time of different sensors according to the analysis time to achieve time synchronization;
the first sensor is used for being matched with the main control unit to calibrate the time of the first sensor;
and the second sensor is used for matching with the main control unit to calibrate the time of the second sensor.
9. The system of claim 8, wherein:
after the first sensor receives the hardware pulse sent by the satellite navigation system time service module, the first sensor takes the analysis time sent by the main control unit as calibration time and sends first sensing data according to the calibration time;
and after receiving the hardware pulse sent by the satellite navigation system time service module, the second sensor takes the analysis time sent by the main control unit as calibration time, and sends second sensing data according to the calibration time.
10. The system of claim 8, wherein:
after the first sensor receives the hardware pulse sent by the satellite navigation system time service module, the first sensor takes the analysis time sent by the main control unit as calibration time and sends first sensing data according to the calibration time;
the main control unit increases the analysis time by a set time after receiving the hardware pulse sent by the satellite navigation system time service module to be used as self calibration time, marks a local timestamp with reference to the self calibration time after receiving second sensing data sent by a second sensor after receiving the hardware pulse sent by the satellite navigation system time service module, and determines the calibration time of the second sensor according to the local timestamp and the time parameter transmitted by the second sensing data.
11. The system according to any one of claims 8 to 10, wherein:
the first sensor is a radar and the second sensor is a camera.
12. A computer-readable storage medium having stored thereon executable code, which when executed by a processor of an electronic device, causes the processor to perform the method of any of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111175336.8A CN113922910B (en) | 2021-10-09 | 2021-10-09 | Sensor time synchronization processing method, device and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111175336.8A CN113922910B (en) | 2021-10-09 | 2021-10-09 | Sensor time synchronization processing method, device and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113922910A true CN113922910A (en) | 2022-01-11 |
| CN113922910B CN113922910B (en) | 2023-09-19 |
Family
ID=79238551
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111175336.8A Active CN113922910B (en) | 2021-10-09 | 2021-10-09 | Sensor time synchronization processing method, device and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113922910B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114499733A (en) * | 2022-02-16 | 2022-05-13 | 东南大学 | Four-legged robot-mounted SLAM device and sensor time synchronization method |
Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103763056A (en) * | 2013-12-31 | 2014-04-30 | 电信科学技术第五研究所 | Method for time synchronization high-precision remote monitoring |
| CN204650151U (en) * | 2015-02-12 | 2015-09-16 | 北京七维航测科技股份有限公司 | Multipath high-speed pulse entry time synchronizer |
| CN107743054A (en) * | 2017-08-25 | 2018-02-27 | 杭州德泽机器人科技有限公司 | System during a kind of synchronous pair of multisensor |
| CN111279282A (en) * | 2018-12-27 | 2020-06-12 | 深圳市大疆创新科技有限公司 | Time service method, switching method, device, control system and unmanned aerial vehicle |
| CN111756463A (en) * | 2019-03-29 | 2020-10-09 | 北京航迹科技有限公司 | Time synchronization system and method for vehicle |
| US20200331480A1 (en) * | 2019-04-16 | 2020-10-22 | Baidu Usa Llc | Time synchronization scheme between different computation nodes in autonomous driving system |
| CN112398563A (en) * | 2020-10-15 | 2021-02-23 | 北京汽车研究总院有限公司 | Time synchronization method and time synchronization system of vehicle and vehicle |
| CN112485806A (en) * | 2020-09-27 | 2021-03-12 | 浙江众合科技股份有限公司 | Laser radar and camera time synchronization system and method |
| CN112672415A (en) * | 2020-12-25 | 2021-04-16 | 之江实验室 | Multi-sensor time synchronization method, device, system, electronic device and medium |
| CN112711043A (en) * | 2020-12-30 | 2021-04-27 | 上海歌诺助航灯光技术有限公司 | GPS-based accurate time service method and electronic equipment |
| CN112787740A (en) * | 2020-12-26 | 2021-05-11 | 武汉光庭信息技术股份有限公司 | Multi-sensor time synchronization device and method |
| CN112865902A (en) * | 2020-12-24 | 2021-05-28 | 深兰人工智能(深圳)有限公司 | Data acquisition and time synchronization method and device, electronic equipment and storage medium |
| CN112911700A (en) * | 2021-01-15 | 2021-06-04 | 新石器慧义知行智驰(北京)科技有限公司 | Vehicle time synchronization method, device and system |
| CN113093781A (en) * | 2021-04-12 | 2021-07-09 | 广东汇天航空航天科技有限公司 | Manned aircraft, data interface module and transmission method of flight control data |
| CN113353090A (en) * | 2021-06-16 | 2021-09-07 | 深圳市道通智能汽车有限公司 | Data synchronization system, data synchronization method, positioning system and unmanned equipment |
-
2021
- 2021-10-09 CN CN202111175336.8A patent/CN113922910B/en active Active
Patent Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103763056A (en) * | 2013-12-31 | 2014-04-30 | 电信科学技术第五研究所 | Method for time synchronization high-precision remote monitoring |
| CN204650151U (en) * | 2015-02-12 | 2015-09-16 | 北京七维航测科技股份有限公司 | Multipath high-speed pulse entry time synchronizer |
| CN107743054A (en) * | 2017-08-25 | 2018-02-27 | 杭州德泽机器人科技有限公司 | System during a kind of synchronous pair of multisensor |
| CN111279282A (en) * | 2018-12-27 | 2020-06-12 | 深圳市大疆创新科技有限公司 | Time service method, switching method, device, control system and unmanned aerial vehicle |
| CN111756463A (en) * | 2019-03-29 | 2020-10-09 | 北京航迹科技有限公司 | Time synchronization system and method for vehicle |
| US20200331480A1 (en) * | 2019-04-16 | 2020-10-22 | Baidu Usa Llc | Time synchronization scheme between different computation nodes in autonomous driving system |
| CN112485806A (en) * | 2020-09-27 | 2021-03-12 | 浙江众合科技股份有限公司 | Laser radar and camera time synchronization system and method |
| CN112398563A (en) * | 2020-10-15 | 2021-02-23 | 北京汽车研究总院有限公司 | Time synchronization method and time synchronization system of vehicle and vehicle |
| CN112865902A (en) * | 2020-12-24 | 2021-05-28 | 深兰人工智能(深圳)有限公司 | Data acquisition and time synchronization method and device, electronic equipment and storage medium |
| CN112672415A (en) * | 2020-12-25 | 2021-04-16 | 之江实验室 | Multi-sensor time synchronization method, device, system, electronic device and medium |
| CN112787740A (en) * | 2020-12-26 | 2021-05-11 | 武汉光庭信息技术股份有限公司 | Multi-sensor time synchronization device and method |
| CN112711043A (en) * | 2020-12-30 | 2021-04-27 | 上海歌诺助航灯光技术有限公司 | GPS-based accurate time service method and electronic equipment |
| CN112911700A (en) * | 2021-01-15 | 2021-06-04 | 新石器慧义知行智驰(北京)科技有限公司 | Vehicle time synchronization method, device and system |
| CN113093781A (en) * | 2021-04-12 | 2021-07-09 | 广东汇天航空航天科技有限公司 | Manned aircraft, data interface module and transmission method of flight control data |
| CN113353090A (en) * | 2021-06-16 | 2021-09-07 | 深圳市道通智能汽车有限公司 | Data synchronization system, data synchronization method, positioning system and unmanned equipment |
Non-Patent Citations (3)
| Title |
|---|
| PHILIPP KOSCHORREK: "A Multi-sensor Traffic Scene Dataset with Omnidirectional Video" * |
| 刘钊: "多传感器时空一致及其信息融合技术研究" * |
| 隗寒冰: "基于多源异构信息融合的智能汽车目标检测算法" * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114499733A (en) * | 2022-02-16 | 2022-05-13 | 东南大学 | Four-legged robot-mounted SLAM device and sensor time synchronization method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113922910B (en) | 2023-09-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111381487B (en) | Multi-sensor synchronous time service system, method and device and electronic equipment | |
| CN111007554A (en) | Data acquisition time synchronization system and method | |
| US10720168B2 (en) | PPS tagging of acoustic sample data | |
| US10197680B2 (en) | GNSS receiver | |
| CN112865902B (en) | Data acquisition and time synchronization method and device, electronic equipment and storage medium | |
| CN111385866B (en) | Method and device for synchronization between devices | |
| CN113922910B (en) | Sensor time synchronization processing method, device and system | |
| CN111556226A (en) | Camera system | |
| CN114019782B (en) | Method and device for taming navigation satellite clock of automatic driving domain controller | |
| US10728453B2 (en) | Motion stabilized image sensor, camera module and apparatus comprising same | |
| CN113498625A (en) | Clock synchronization method and device, chip system, unmanned aerial vehicle and terminal | |
| JP6075377B2 (en) | COMMUNICATION DEVICE, COMMUNICATION SYSTEM, COMMUNICATION METHOD, AND PROGRAM | |
| CN116614194A (en) | Vehicle time synchronization method, device, vehicle, storage medium and program product | |
| CN108710141B (en) | Baseband chip sensitivity test method and device | |
| CN113890668A (en) | Multi-sensor time synchronization method and device, electronic equipment and storage medium | |
| CN114545762A (en) | Time data synchronization method and system based on inertial measurement unit | |
| CN112286039A (en) | Clock calibration method and device and readable storage medium | |
| TWI664393B (en) | Mobile mapping system and positioning terminal device | |
| US10009872B2 (en) | Mobile device, operating method thereof, and non-transitory computer readable storage medium for controlling an external device | |
| CN112526574A (en) | Satellite positioning method and device | |
| US20230418325A1 (en) | Clock synchronisation | |
| JP2015200593A (en) | Count value acquisition apparatus and count value acquisition program | |
| US11874384B2 (en) | Locator, accuracy evaluation system therefor, and positioning method | |
| CN110971327A (en) | Time synchronization method and device for environment target | |
| CN117518770A (en) | Time management system and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |