CN112541527A - Multi-sensor synchronization method and device, electronic equipment and storage medium - Google Patents
Multi-sensor synchronization method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN112541527A CN112541527A CN202011350318.4A CN202011350318A CN112541527A CN 112541527 A CN112541527 A CN 112541527A CN 202011350318 A CN202011350318 A CN 202011350318A CN 112541527 A CN112541527 A CN 112541527A
- Authority
- CN
- China
- Prior art keywords
- sensor
- data
- sensors
- data processing
- result
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 60
- 238000012545 processing Methods 0.000 claims abstract description 189
- 230000001360 synchronised effect Effects 0.000 claims abstract description 88
- 230000004927 fusion Effects 0.000 claims abstract description 21
- 238000004590 computer program Methods 0.000 claims description 11
- 238000010586 diagram Methods 0.000 description 8
- 230000009286 beneficial effect Effects 0.000 description 4
- 238000004891 communication Methods 0.000 description 4
- 238000007499 fusion processing Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000013480 data collection Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/251—Fusion techniques of input or preprocessed data
Landscapes
- Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Arrangements For Transmission Of Measured Signals (AREA)
Abstract
The application provides a multi-sensor synchronization method, a multi-sensor synchronization device, electronic equipment and a storage medium, wherein the method comprises the following steps: acquiring sensor data synchronously acquired by a plurality of sensors, and determining the synchronous time of the sensor data; respectively processing sensor data synchronously acquired by a plurality of sensors, and storing data processing results of the plurality of sensors into result queues of corresponding sensors; acquiring the latest data processing result in the result queue of the reference sensor as a reference data processing result; and respectively acquiring a data processing result with the synchronous acquisition time closest to the synchronous acquisition time corresponding to the reference data processing result from the result queues of other sensors, and forming synchronous data to be fused together with the reference data processing result. The synchronization of the data acquisition end and the synchronization of the data fusion end are completed, and the problem of inaccurate data fusion caused by the asynchronous data processing result of the multiple sensors is solved.
Description
Technical Field
The present application relates to the field of data synchronization technologies, and in particular, to a multi-sensor synchronization method and apparatus, an electronic device, and a storage medium.
Background
In the field of automatic driving of vehicles, the safety of automatic driving is mainly guaranteed by the accuracy of data collected by a sensor. The accuracy of the data collected by the sensors depends on the consistency of the sensor data in time, namely, the synchronization among the sensor data. However, the consistency of the sensor data in time is difficult to ensure because each sensor operates independently, and the time of the MCU in the sensor is not completely synchronized, so the timestamp carried by the sensor data represents the time of the sensor itself, and cannot accurately indicate the chronological order between the sensor data and other sensor data. In addition, the autopilot system needs to perform fusion processing on data collected by the sensors for reducing overlapping targets, mutual complementation of different sensor data, and the like. However, in actual processing, the processing rates of data of a plurality of sensors are inconsistent due to the difference of algorithms of various sensor suppliers and the difference of sensor hardware performance, and when data fusion is performed at a post-fusion end, the phenomenon that the data processing results are asynchronous still exists.
However, the current multi-sensor synchronization methods only focus on the synchronization of the data acquisition end. Therefore, a method capable of solving the data synchronization at the post-fusion end is needed to avoid the problem of inaccurate data fusion caused by asynchronous data processing results of multiple sensors.
Disclosure of Invention
The application provides a multi-sensor synchronization method, a multi-sensor synchronization device, electronic equipment and a storage medium, so that data synchronization of a collection end and data synchronization of a post-fusion end are achieved.
The application provides a multi-sensor synchronization method, which comprises the following steps:
acquiring sensor data synchronously acquired by a plurality of sensors;
respectively processing the sensor data synchronously acquired by the sensors, and respectively storing the data processing results of the sensors into corresponding result queues;
acquiring the latest data processing result in the result queue corresponding to the reference sensor as a reference data processing result;
and acquiring a data processing result with the synchronous acquisition time closest to the synchronous acquisition time corresponding to the reference data processing result from the result queues corresponding to other sensors, and forming synchronous data to be fused together with the reference data processing result.
According to the multi-sensor synchronization method provided by the application, the synchronous acquisition time corresponding to the data processing result is added by the system clock after the system clock acquires the sensor data synchronously acquired by the plurality of sensors.
According to the multi-sensor synchronization method provided by the application, the sensor data synchronously acquired by the plurality of sensors are respectively processed to obtain data processing results corresponding to the plurality of sensors, and the method specifically comprises the following steps:
respectively processing the sensor data synchronously acquired by the plurality of sensors to obtain data processing results corresponding to the plurality of sensors;
if the result queue corresponding to any sensor is full, deleting the data processing result which is stored firstly in the result queue corresponding to any sensor, and storing the data processing result which is currently corresponding to any sensor in the corresponding result queue;
otherwise, storing the data processing result currently corresponding to any sensor into a corresponding result queue.
According to the multi-sensor synchronization method provided by the application, the acquiring of the sensor data synchronously acquired by the plurality of sensors specifically comprises the following steps:
after the data of any sensor is acquired, adding an acquisition timestamp to the data of the sensor acquired by any sensor by a system clock;
and acquiring the sensor data synchronously acquired by the plurality of sensors based on the acquisition time stamps corresponding to the sensor data of the reference sensor and the acquisition time stamps corresponding to the sensor data of the other sensors.
According to the multi-sensor synchronization method provided by the application, the acquisition time stamp corresponding to the sensor data of the reference sensor and the acquisition time stamps corresponding to the sensor data of other sensors are used for acquiring the sensor data synchronously acquired by a plurality of sensors, and the method specifically comprises the following steps:
judging whether the difference value of the acquisition time stamp corresponding to the sensor data of each sensor except the reference sensor and the acquisition time stamp corresponding to the sensor data of the reference sensor is smaller than a preset synchronous threshold value or not;
if the difference value between the acquisition time stamp corresponding to the sensor data of any sensor and the acquisition time stamp corresponding to the sensor data of the reference sensor is smaller than the preset synchronous threshold value, acquiring the sensor data of any sensor, and adding one to the counter;
otherwise, discarding the sensor data of any sensor;
and if the value of the counter is equal to the number of the sensors, taking all the acquired sensor data as the synchronously acquired sensor data.
According to the multi-sensor synchronization method provided by the application, the reference sensor is the sensor with the slowest acquisition speed in the plurality of sensors.
The present application further provides a multi-sensor synchronization device, comprising:
the acquisition synchronization unit is used for acquiring sensor data synchronously acquired by a plurality of sensors;
the data processing unit is used for respectively processing the sensor data synchronously acquired by the sensors to obtain data processing results corresponding to the sensors;
and the fusion synchronization unit is used for acquiring a reference data processing result corresponding to a reference sensor in the plurality of sensors and a data processing result corresponding to other sensors and synchronized with the reference data processing result as data to be fused.
The present application further provides an electronic device, comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of the multi-sensor synchronization method as described in any of the above when executing the computer program.
The present application also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the multi-sensor synchronization method as any of the above.
According to the multi-sensor synchronization method, the multi-sensor synchronization device, the electronic equipment and the storage medium, sensor data acquired synchronously by the multiple sensors are acquired and processed respectively, then reference data processing results corresponding to reference sensors in the multiple sensors are acquired, and data processing results corresponding to other sensors and closest to the acquisition time of the reference data processing results are taken as synchronous data to be fused, so that synchronization of a data acquisition end and synchronization of a data fusion end are completed, and the problem of inaccurate data fusion caused by asynchronous data processing results of the multiple sensors is avoided.
Drawings
In order to more clearly illustrate the technical solutions in the present application or the prior art, the drawings needed for the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
FIG. 1 is a schematic flow chart of a multi-sensor synchronization method provided herein;
fig. 2 is a schematic flow chart of an acquisition synchronization method provided in the present application;
fig. 3 is a second schematic flow chart of the acquisition synchronization method provided in the present application;
FIG. 4 is a second schematic flow chart of a multi-sensor synchronization method provided in the present application;
FIG. 5 is a schematic diagram of a multi-sensor synchronization apparatus provided herein;
fig. 6 is a schematic structural diagram of an acquisition synchronization unit provided in the present application;
fig. 7 is a schematic structural diagram of a synchronous data acquisition unit provided in the present application;
fig. 8 is a schematic structural diagram of an electronic device provided in the present application.
Detailed Description
To make the purpose, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be clearly and completely described below with reference to the drawings in the present application, and it is obvious that the described embodiments are some, but not all embodiments of the present application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
In the field of automatic driving of vehicles, the safety of automatic driving is mainly guaranteed by the accuracy of data collected by a sensor. The accuracy of the data collected by the sensors depends on the consistency of the sensor data in time, namely, the synchronization among the sensor data. However, the consistency of the sensor data in time is difficult to ensure because each sensor operates independently, and the time of the MCU in the sensor is not completely synchronized, so the timestamp carried by the sensor data represents the time of the sensor itself, and cannot accurately indicate the chronological order between the sensor data and other sensor data. In addition, the autopilot system needs to perform fusion processing on data collected by the sensors for reducing overlapping targets, mutual complementation of different sensor data, and the like. However, in actual processing, the processing rates of data of a plurality of sensors are inconsistent due to the difference of algorithms of various sensor suppliers and the difference of sensor hardware performance, and when data fusion is performed at a post-fusion end, the phenomenon that the data processing results are asynchronous still exists.
However, the current multi-sensor synchronization methods only focus on the synchronization of the data acquisition end. Therefore, a method capable of solving the data synchronization at the post-fusion end is needed to avoid the problem of inaccurate data fusion caused by asynchronous data processing results of multiple sensors.
In view of the above, the embodiment of the present application provides a multi-sensor synchronization method. Fig. 1 is a schematic flowchart of a multi-sensor synchronization method provided in an embodiment of the present application, and as shown in fig. 1, the method includes:
and the synchronous acquisition time corresponding to the data processing result is the synchronous time corresponding to the sensor data.
Specifically, multiple sensors independently collect sensor data. The sensor may be a multi-line lidar, a single-line lidar, a multi-path camera, a combined inertial navigation system, and the like, which is not specifically limited in the embodiment of the present application. The method comprises the steps of receiving sensor data acquired by each sensor, screening the sensor data synchronously acquired by each sensor from the sensor data to ensure data synchronization of a data acquisition end, and adding a timestamp to each synchronously acquired sensor data by using a system clock to serve as the synchronization time of each sensor data. Here, the synchronization time of the sensor data synchronously acquired by each sensor is the same.
And respectively inputting the sensor data synchronously acquired by each sensor into the independent data processing module corresponding to each sensor for processing to obtain the data processing result corresponding to each sensor, and storing the data processing result corresponding to each sensor into the corresponding result queue. And one sensor corresponds to one result queue and is used for storing the data processing result corresponding to the sensor. Here, the data processing module performs data preprocessing, such as normalization processing, denoising, and the like, on the sensor data of each sensor before fusion. Because the processing rates of the data processing modules corresponding to the sensors are different, even if the sensor data synchronously acquired by the sensors are simultaneously input into the data processing modules, the data processing results output by the data processing modules are different. In order to synchronously input the data processing results corresponding to the sensor data synchronously acquired by the plurality of sensors into the data fusion module, the data processing results corresponding to the sensors need to be synchronously processed at the fusion end.
Among the plurality of sensors, one sensor may be selected as the reference sensor. And taking the reference data processing result corresponding to the reference sensor as a comparison reference for fusing data synchronization of the end, selecting a data processing result with the synchronous acquisition time closest to that of the reference data processing result from the result queues of other sensors, and taking the data processing result and the reference data processing result together as synchronous data to be fused. The reference data processing result is the latest data processing result in the result queue of the reference sensor, and corresponds to the data processing result obtained after the data processing is performed on the sensor data synchronously acquired by the reference sensor in step 110. And the synchronous acquisition time of the data processing result is the synchronous time of the sensor data corresponding to the data processing result.
Here, since the synchronization time of the sensor data is the same as the synchronization acquisition time of the data processing result corresponding thereto, the data processing result corresponding to the synchronously acquired sensor data can be identified from the synchronization time. When a data processing result synchronized with the reference data processing result is obtained, a data processing result whose synchronous acquisition time is closest to the synchronous acquisition time of the reference data processing result may be selected. Here, if the synchronous acquisition time is the same, it indicates that the sensor data corresponding to the data processing result is synchronous with the sensor data corresponding to the reference data processing result, and the data processing result is also synchronous with the reference data processing result; if the synchronous time stamps are close, it is indicated that the sensor data corresponding to the data processing result is approximately synchronous with the sensor data corresponding to the reference data processing result, and the data processing result and the reference data processing result can also be regarded as synchronous, so that data synchronization of the fusion end is realized.
According to the method provided by the embodiment of the application, the sensor data synchronously acquired by the plurality of sensors are acquired and are respectively processed, then the reference data processing result corresponding to the reference sensor in the plurality of sensors is acquired, and the data processing result corresponding to the other sensors and having the closest synchronous acquisition time to the reference data processing result is taken as the synchronous data to be fused, so that the synchronization of the data acquisition end and the synchronization of the data fusion end are completed, and the problem of inaccurate data fusion caused by the asynchronous data processing result of the plurality of sensors is avoided.
Based on the above embodiment, the reference sensor is the sensor with the slowest acquisition speed among the plurality of sensors.
Specifically, the sensor with the slowest acquisition speed among the plurality of sensors may be used as the reference sensor. At this moment, because the acquisition speed of the reference sensor is the slowest, after the acquisition of the reference sensor is finished, the acquisition of other sensors is finished, and at this moment, the synchronously acquired sensor data can be immediately acquired from the sensor data acquired by the plurality of sensors, and the subsequent steps of data processing and fusion end synchronization are executed.
Based on any of the above embodiments, in step 120, storing the data processing results of the multiple sensors into corresponding result queues respectively, specifically includes:
if the result queue corresponding to the sensor is full, deleting the data processing result which is stored firstly in the result queue corresponding to the sensor, and storing the data processing result which is currently corresponding to the sensor into the corresponding result queue;
otherwise, storing the current corresponding data processing result of the sensor into a corresponding result queue.
Specifically, after sensor data synchronously acquired by a plurality of sensors is acquired, the sensor data acquired by each sensor may be stored in a result queue corresponding to each sensor. The result queues corresponding to the sensors have length limitations, so before storing the corresponding data processing results into the queues, it is necessary to first determine whether the current result queues are full. If the result queue corresponding to any sensor is full, deleting the data processing result stored in the result queue first, and then storing the current data processing result into the result queue; if the current result queue is not full, the current data processing result can be directly stored in the result queue. It should be noted that, since the data processing result stored first is deleted when the result queue is full, if the data processing speed of the data processing module of the reference sensor is slow, when the sensor data synchronously acquired by the reference sensor is processed, the data processing modules of other sensor data may have already performed data processing for multiple times, and the data processing result of the sensor data synchronously acquired is deleted when the sensor data is stored in the result queue. Therefore, when a data processing result synchronized with the reference data processing result is acquired, there may be no data processing result identical to the synchronization time stamp of the reference data processing result, and therefore, a data processing result whose synchronization time stamp corresponding to another sensor is closest to the synchronization time stamp of the reference data processing result can be selected as the synchronized data processing result.
Based on any of the above embodiments, fig. 2 is a schematic flow chart of the acquisition synchronization method provided in the embodiments of the present application, and as shown in fig. 2, in step 110, acquiring sensor data synchronously acquired by a plurality of sensors specifically includes:
and step 112, acquiring the sensor data synchronously acquired by the plurality of sensors based on the acquisition time stamps corresponding to the sensor data of the reference sensor and the acquisition time stamps corresponding to the sensor data of the other sensors.
Specifically, a hardware device pulse trigger synchronization mode is generally adopted in the current data acquisition synchronization scheme, and the mode can additionally increase the cost of the hardware device, so that the embodiment of the invention utilizes a software algorithm to perform synchronization operation of an acquisition end when data acquisition is finished and a data processing module is to be accessed. Specifically, after data acquisition of any sensor is completed, the system clock adds an acquisition time stamp to the sensor data acquired by the sensor. At this time, the collection timestamp is the time when the sensor data collection is completed. After each sensor finishes collecting, corresponding collecting time stamps are added to the sensor data collected by each sensor, and the collecting time stamps of the sensor data are possibly the same and possibly different because each sensor independently collects the sensor data.
Based on the acquisition time stamp corresponding to the sensor data of the reference sensor and the acquisition time stamps corresponding to the sensor data of the other sensors, the sensor data which is closer to the acquisition time stamp corresponding to the sensor data of the reference sensor is acquired and is used as the sensor data synchronously acquired by the plurality of sensors. Here, since the acquisition timestamps of the sensor data are all close, that is, the acquisition time of each sensor data is close, the sensor data acquired in the above manner can be regarded as the synchronously acquired sensor data.
Based on any of the above embodiments, fig. 3 is a second schematic flow chart of the acquisition synchronization method provided in the embodiment of the present application, and as shown in fig. 3, step 112 specifically includes:
and 1123, if the value of the counter is equal to the number of the sensors, taking all the acquired sensor data as the synchronously acquired sensor data.
Specifically, the acquisition time stamp corresponding to the sensor data of the reference sensor is used as a comparison object, and whether the difference value between the acquisition time stamp corresponding to the sensor data of the other sensors and the comparison object is smaller than a preset synchronization threshold value is judged. Here, the preset synchronization threshold is preset according to the real-time requirement of the system, and may be set to 100ms to 500ms, for example. The smaller the preset synchronization threshold is, the closer the acquisition time corresponding to the finally acquired sensor data acquired synchronously is.
And if the difference value between the acquisition time stamp corresponding to the sensor data of any sensor and the acquisition time stamp corresponding to the sensor data of the reference sensor is smaller than the preset synchronization threshold value, acquiring the sensor data of the sensor as the sensor data synchronized with the sensor data of the reference sensor, and adding one to the counter. Wherein the value of the counter is used to indicate the amount of synchronously acquired sensor data that has been currently acquired. If the value of the counter is exactly equal to the total number of the sensors, the situation that the sensor data synchronously acquired by all the sensors are screened is indicated, so that all the sensor data acquired currently can be used as the sensor data synchronously acquired, and the data synchronization of the acquisition end is completed.
If the difference value between the acquisition time stamp corresponding to the sensor data of any sensor and the acquisition time stamp corresponding to the sensor data of the reference sensor is greater than or equal to the preset synchronization threshold value, the difference value between the acquisition time of the sensor data of the sensor and the acquisition time of the sensor data of the reference sensor is larger, so that the sensor data of the sensor is discarded, and the sensor is waited to acquire new sensor data for the next judgment.
Based on any of the above embodiments, fig. 4 is a second schematic flow chart of the multi-sensor synchronization method provided in the embodiments of the present application, as shown in fig. 4, the method includes:
the sensors independently acquire data, and comprise a multi-line laser radar, a single-line laser radar, n-path cameras and combined inertial navigation. After the sensors finish collecting, the system clock adds collecting time stamps to the data collected by each sensor, namely adding collecting time stamps to the multi-line laser radar point cloud data, the single-line laser radar point cloud data, the image data of the n-way camera and the positioning data of the combined inertial navigation.
The multi-line laser radar is used as a reference sensor, and the acquisition time stamp of the multi-line laser radar is used as a reference time stamp. And then, judging whether the absolute value of the difference value between the acquisition time stamp of the sensor data of each of the rest sensors and the reference time stamp is smaller than the synchronous time threshold value. If the current data is less than the preset value, the sensor counter is automatically added by 1, otherwise, the current data is discarded and new data is waited. And if the numerical value of the sensor counter is equal to n +3, namely the number of the sensors, completing the synchronization of the data acquisition end.
And adding synchronous time stamps to the sensor data synchronously acquired by each sensor by using a system clock. And then, inputting the data of each sensor into a corresponding independent processing module respectively for data processing. After the processing is finished, the corresponding data processing result is pressed into a result queue corresponding to the sensor.
And taking a data processing result corresponding to the latest multi-line laser radar, and taking a corresponding synchronization timestamp as a reference synchronization timestamp. And then, searching a data processing result with the synchronous time stamp closest to the reference synchronous time stamp in the result queue corresponding to each sensor, and taking the data processing result as synchronous data to be fused. And finally, fusing the acquired data processing results of the multiple sensors.
The multi-sensor synchronization device provided by the present application is described below, and the multi-sensor synchronization device described below and the multi-sensor synchronization method described above may be referred to in correspondence with each other.
Based on any of the above embodiments, fig. 5 is a schematic structural diagram of a multi-sensor synchronization apparatus provided in an embodiment of the present application, and as shown in fig. 5, the apparatus includes: an acquisition synchronization unit 510, a data processing unit 520, a reference determination unit 530, and a fusion synchronization unit 540.
The acquisition synchronization unit 510 is configured to acquire sensor data acquired by multiple sensors synchronously, and determine synchronization time of the sensor data;
the data processing unit 520 is configured to process sensor data synchronously acquired by the plurality of sensors, and store data processing results of the plurality of sensors in result queues of corresponding sensors;
the reference determination unit 530 is configured to acquire the latest data processing result in the result queue of the reference sensor as a reference data processing result;
the fusion synchronization unit 540 is configured to obtain, in the result queues of the other sensors, a data processing result whose synchronous acquisition time is closest to the synchronous acquisition time corresponding to the reference data processing result, and form synchronous data to be fused together with the reference data processing result; and the synchronous acquisition time corresponding to the data processing result is the synchronous time corresponding to the sensor data.
The device that this application embodiment provided, through the sensor data that acquires the synchronous collection of a plurality of sensors, and handle it respectively, then the benchmark data processing result that the benchmark sensor among the acquisition of a plurality of sensors corresponds, and the data processing result that the synchronous collection time that corresponds with benchmark data processing result is the closest of other sensors, as the synchronous data that awaits the fuse, the synchronization of data acquisition end and the synchronization of data fusion end have been accomplished, the inaccurate problem of data fusion that has avoided the data processing result of multisensor to lead to not synchronous.
Based on any one of the above embodiments, the reference sensor is the sensor with the slowest acquisition speed among the plurality of sensors.
Based on any of the above embodiments, the data processing unit 520 is specifically configured to:
if the result queue corresponding to the sensor is full, deleting the data processing result which is stored firstly in the result queue corresponding to the sensor, and storing the data processing result which is currently corresponding to the sensor into the corresponding result queue;
otherwise, storing the current corresponding data processing result of the sensor into a corresponding result queue.
Based on any of the above embodiments, fig. 6 is a schematic structural diagram of the acquisition synchronization unit provided in the embodiments of the present application, and as shown in fig. 6, the acquisition synchronization unit 510 specifically includes:
the acquisition time service unit 511 is used for adding an acquisition timestamp to the sensor data acquired by any sensor by using a system clock after the data acquisition of the sensor is finished;
the synchronous data acquiring unit 512 is configured to acquire sensor data synchronously acquired by the plurality of sensors based on the acquisition timestamps corresponding to the sensor data of the reference sensor and the acquisition timestamps corresponding to the sensor data of the other sensors.
Based on any of the above embodiments, fig. 7 is a schematic structural diagram of a synchronization data acquisition unit provided in an embodiment of the present application, and as shown in fig. 7, the synchronization data acquisition unit specifically includes:
a time determination unit 5121, configured to determine whether a difference between the acquisition timestamps corresponding to the sensor data of the other sensors and the acquisition timestamps corresponding to the sensor data of the reference sensor is smaller than a preset synchronization threshold;
a data acquiring unit 5122, configured to acquire sensor data of any sensor and increment a counter by one if a difference between a collection timestamp corresponding to the sensor data of the sensor and a collection timestamp corresponding to the sensor data of a reference sensor is smaller than a preset synchronization threshold; and if the value of the counter is equal to the number of the sensors, taking all the acquired sensor data as the synchronously acquired sensor data.
The multi-sensor synchronization device provided in the embodiment of the present application is used for executing the multi-sensor synchronization method, and the specific implementation manner thereof is consistent with the method implementation manner, and the same beneficial effects can be achieved, which is not described herein again.
Fig. 8 illustrates a physical structure diagram of an electronic device, and as shown in fig. 8, the electronic device may include: a processor (processor)810, a communication Interface 820, a memory 830 and a communication bus 840, wherein the processor 810, the communication Interface 820 and the memory 830 communicate with each other via the communication bus 840. The processor 810 may invoke logic instructions in the memory 830 to perform a multi-sensor synchronization method comprising: acquiring sensor data synchronously acquired by a plurality of sensors, and determining the synchronous time of the sensor data; respectively processing sensor data synchronously acquired by the sensors, and storing data processing results of the sensors into result queues of corresponding sensors; acquiring the latest data processing result in the result queue of the reference sensor as a reference data processing result; respectively acquiring data processing results with synchronous acquisition time closest to the synchronous acquisition time corresponding to the reference data processing results from result queues of other sensors, and forming synchronous data to be fused together with the reference data processing results; and the synchronous acquisition time corresponding to the data processing result is the synchronous time corresponding to the sensor data.
In addition, the logic instructions in the memory 830 may be implemented in software functional units and stored in a computer readable storage medium when the logic instructions are sold or used as independent products. Based on such understanding, the technical solution of the present application or portions thereof that substantially contribute to the prior art may be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The processor 810 in the electronic device provided in the embodiment of the present application may call a logic instruction in the memory 830 to implement the multi-sensor synchronization method, and a specific implementation manner of the method is consistent with that of the method, and the same beneficial effects may be achieved, which is not described herein again.
In another aspect, the present application also provides a computer program product comprising a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions which, when executed by a computer, enable the computer to perform the multi-sensor synchronization method provided by the above methods, the method comprising: acquiring sensor data synchronously acquired by a plurality of sensors, and determining the synchronous time of the sensor data; respectively processing sensor data synchronously acquired by the sensors, and storing data processing results of the sensors into result queues of corresponding sensors; acquiring the latest data processing result in the result queue of the reference sensor as a reference data processing result; respectively acquiring data processing results with synchronous acquisition time closest to the synchronous acquisition time corresponding to the reference data processing results from result queues of other sensors, and forming synchronous data to be fused together with the reference data processing results; and the synchronous acquisition time corresponding to the data processing result is the synchronous time corresponding to the sensor data.
When the computer program product provided in the embodiment of the present application is executed, the multi-sensor synchronization method is implemented, and the specific implementation manner is consistent with the method implementation manner, and the same beneficial effects can be achieved, which is not described herein again.
In yet another aspect, the present application also provides a non-transitory computer readable storage medium having stored thereon a computer program that, when executed by a processor, is implemented to perform the multi-sensor synchronization method provided above, the method comprising: acquiring sensor data synchronously acquired by a plurality of sensors, and determining the synchronous time of the sensor data; respectively processing sensor data synchronously acquired by the sensors, and storing data processing results of the sensors into result queues of corresponding sensors; acquiring the latest data processing result in the result queue of the reference sensor as a reference data processing result; respectively acquiring data processing results with synchronous acquisition time closest to the synchronous acquisition time corresponding to the reference data processing results from result queues of other sensors, and forming synchronous data to be fused together with the reference data processing results; and the synchronous acquisition time corresponding to the data processing result is the synchronous time corresponding to the sensor data.
When a computer program stored on a non-transitory computer-readable storage medium provided in the embodiments of the present application is executed, the multi-sensor synchronization method is implemented, and the specific implementation manner is consistent with the method implementation manner and can achieve the same beneficial effects, which is not described herein again.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions in the embodiments of the present application.
Claims (11)
1. A multi-sensor synchronization method, comprising:
acquiring sensor data synchronously acquired by a plurality of sensors, and determining the synchronous time of the sensor data;
respectively processing sensor data synchronously acquired by the sensors, and storing data processing results of the sensors into result queues of corresponding sensors;
acquiring the latest data processing result in the result queue of the reference sensor as a reference data processing result;
respectively acquiring data processing results with synchronous acquisition time closest to the synchronous acquisition time corresponding to the reference data processing results from result queues of other sensors, and forming synchronous data to be fused together with the reference data processing results; and the synchronous acquisition time corresponding to the data processing result is the synchronous time corresponding to the sensor data.
2. The multi-sensor synchronization method according to claim 1, wherein the storing the data processing results of the plurality of sensors into corresponding result queues respectively specifically comprises:
if the result queue corresponding to the sensor is full, deleting the data processing result which is stored firstly in the result queue corresponding to the sensor, and storing the data processing result which corresponds to the sensor currently in the corresponding result queue;
otherwise, storing the data processing result currently corresponding to the sensor into a corresponding result queue.
3. The multi-sensor synchronization method according to claim 1, wherein the acquiring sensor data synchronously acquired by a plurality of sensors specifically comprises:
after the data of any sensor is acquired, adding an acquisition timestamp to the data of the sensor acquired by any sensor by using a system clock;
and acquiring the sensor data synchronously acquired by the plurality of sensors based on the acquisition time stamps corresponding to the sensor data of the reference sensor and the acquisition time stamps corresponding to the sensor data of the other sensors.
4. The multi-sensor synchronization method according to claim 3, wherein the acquiring the sensor data synchronously acquired by the plurality of sensors based on the acquisition timestamps corresponding to the sensor data of the reference sensor and the acquisition timestamps corresponding to the sensor data of the other sensors comprises:
judging whether the difference value between the acquisition time stamps corresponding to the sensor data of other sensors and the acquisition time stamps corresponding to the sensor data of the reference sensor is smaller than a preset synchronous threshold value or not;
if the difference value between the acquisition time stamp corresponding to the sensor data of any sensor and the acquisition time stamp corresponding to the sensor data of the reference sensor is smaller than the preset synchronous threshold value, acquiring the sensor data of any sensor, and adding one to the counter;
and if the value of the counter is equal to the number of the sensors, taking all the acquired sensor data as the synchronously acquired sensor data.
5. The multi-sensor synchronization method according to any one of claims 1 to 4, wherein the reference sensor is a sensor having the slowest acquisition speed among the plurality of sensors.
6. A multi-sensor synchronization device, comprising:
the acquisition synchronization unit is used for acquiring sensor data synchronously acquired by a plurality of sensors and determining the synchronization time of the sensor data;
the data processing unit is used for respectively processing the sensor data synchronously acquired by the sensors and storing the data processing results of the sensors into result queues of the corresponding sensors;
a reference determination unit for acquiring the latest data processing result in the result queue of the reference sensor as a reference data processing result;
the fusion synchronization unit is used for respectively acquiring a data processing result with the synchronous acquisition time closest to the synchronous acquisition time corresponding to the reference data processing result from the result queues of other sensors, and the data processing result and the reference data processing result together form synchronous data to be fused; and the synchronous acquisition time corresponding to the data processing result is the synchronous time corresponding to the sensor data.
7. The multi-sensor synchronization device according to claim 6, wherein the data processing unit is specifically configured to:
if the result queue corresponding to the sensor is full, deleting the data processing result which is stored firstly in the result queue corresponding to the sensor, and storing the data processing result which corresponds to the sensor currently in the corresponding result queue;
otherwise, storing the data processing result currently corresponding to the sensor into a corresponding result queue.
8. The multi-sensor synchronization device according to claim 6, wherein the acquisition synchronization unit specifically comprises:
the acquisition time service unit is used for adding an acquisition timestamp to the sensor data acquired by any sensor by using a system clock after the data acquisition of any sensor is finished;
and the synchronous data acquisition unit is used for acquiring the sensor data synchronously acquired by the plurality of sensors based on the acquisition time stamps corresponding to the sensor data of the reference sensor and the acquisition time stamps corresponding to the sensor data of other sensors.
9. The multi-sensor synchronization device according to claim 8, wherein the synchronization data acquisition unit specifically comprises:
the time judging unit is used for judging whether the difference value between the acquisition time stamps corresponding to the sensor data of other sensors and the acquisition time stamps corresponding to the sensor data of the reference sensor is smaller than a preset synchronous threshold value or not;
the data acquisition unit is used for acquiring the sensor data of any sensor and adding one to the counter if the difference value between the acquisition time stamp corresponding to the sensor data of any sensor and the acquisition time stamp corresponding to the sensor data of the reference sensor is smaller than the preset synchronous threshold; and if the value of the counter is equal to the number of the sensors, taking all the acquired sensor data as the synchronously acquired sensor data.
10. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the multi-sensor synchronization method according to any of claims 1 to 5 when executing the program.
11. A non-transitory computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the multi-sensor synchronization method according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011350318.4A CN112541527A (en) | 2020-11-26 | 2020-11-26 | Multi-sensor synchronization method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011350318.4A CN112541527A (en) | 2020-11-26 | 2020-11-26 | Multi-sensor synchronization method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112541527A true CN112541527A (en) | 2021-03-23 |
Family
ID=75016811
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011350318.4A Pending CN112541527A (en) | 2020-11-26 | 2020-11-26 | Multi-sensor synchronization method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112541527A (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113610136A (en) * | 2021-07-30 | 2021-11-05 | 深圳元戎启行科技有限公司 | Sensor data synchronization method and device, computer equipment and storage medium |
| CN113743456A (en) * | 2021-07-27 | 2021-12-03 | 武汉光庭信息技术股份有限公司 | Scene positioning method and system based on unsupervised learning |
| CN113965289A (en) * | 2021-10-29 | 2022-01-21 | 际络科技(上海)有限公司 | Time synchronization method and device based on multi-sensor data |
| CN114006672A (en) * | 2021-09-17 | 2022-02-01 | 东风汽车集团股份有限公司 | Vehicle-mounted multi-sensor data synchronous acquisition method and system |
| CN114172915A (en) * | 2021-11-05 | 2022-03-11 | 中汽创智科技有限公司 | Message synchronization method, automatic driving system, storage medium and electronic equipment |
| CN114383667A (en) * | 2022-01-29 | 2022-04-22 | 重庆长安汽车股份有限公司 | Multi-sensor simulation data synchronous injection method and system |
| CN115840234A (en) * | 2022-10-28 | 2023-03-24 | 苏州知至科技有限公司 | Radar data acquisition method and device and storage medium |
| CN116707746A (en) * | 2023-08-07 | 2023-09-05 | 长春航盛艾思科电子有限公司 | Measuring method of array sensor |
| CN118337313A (en) * | 2024-06-12 | 2024-07-12 | 中国第一汽车股份有限公司 | Multi-sensor data synchronization method, device, equipment and medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109635870A (en) * | 2018-12-11 | 2019-04-16 | 百度在线网络技术（北京）有限公司 | Data processing method and device |
| CN110781949A (en) * | 2019-10-23 | 2020-02-11 | 福建汉特云智能科技有限公司 | Asynchronous serial multi-sensor-based flight path data fusion method and storage medium |
| CN111983948A (en) * | 2020-07-07 | 2020-11-24 | 宝能(广州)汽车研究院有限公司 | Multi-sensor data synchronization method and equipment thereof |
-
2020
- 2020-11-26 CN CN202011350318.4A patent/CN112541527A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109635870A (en) * | 2018-12-11 | 2019-04-16 | 百度在线网络技术（北京）有限公司 | Data processing method and device |
| CN110781949A (en) * | 2019-10-23 | 2020-02-11 | 福建汉特云智能科技有限公司 | Asynchronous serial multi-sensor-based flight path data fusion method and storage medium |
| CN111983948A (en) * | 2020-07-07 | 2020-11-24 | 宝能(广州)汽车研究院有限公司 | Multi-sensor data synchronization method and equipment thereof |
Non-Patent Citations (1)
| Title |
|---|
| 邵欣等: "《物联网技术及应用》", 北京航空航天大学出版社, pages: 167 * |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113743456A (en) * | 2021-07-27 | 2021-12-03 | 武汉光庭信息技术股份有限公司 | Scene positioning method and system based on unsupervised learning |
| CN113743456B (en) * | 2021-07-27 | 2024-05-10 | 武汉光庭信息技术股份有限公司 | Scene positioning method and system based on unsupervised learning |
| CN113610136A (en) * | 2021-07-30 | 2021-11-05 | 深圳元戎启行科技有限公司 | Sensor data synchronization method and device, computer equipment and storage medium |
| CN114006672A (en) * | 2021-09-17 | 2022-02-01 | 东风汽车集团股份有限公司 | Vehicle-mounted multi-sensor data synchronous acquisition method and system |
| CN114006672B (en) * | 2021-09-17 | 2024-04-02 | 东风汽车集团股份有限公司 | Vehicle-mounted multi-sensor data synchronous acquisition method and system |
| CN113965289B (en) * | 2021-10-29 | 2024-03-12 | 际络科技(上海)有限公司 | Time synchronization method and device based on multi-sensor data |
| CN113965289A (en) * | 2021-10-29 | 2022-01-21 | 际络科技(上海)有限公司 | Time synchronization method and device based on multi-sensor data |
| CN114172915B (en) * | 2021-11-05 | 2023-10-31 | 中汽创智科技有限公司 | Message synchronization method, automatic driving system, storage medium and electronic equipment |
| CN114172915A (en) * | 2021-11-05 | 2022-03-11 | 中汽创智科技有限公司 | Message synchronization method, automatic driving system, storage medium and electronic equipment |
| CN114383667A (en) * | 2022-01-29 | 2022-04-22 | 重庆长安汽车股份有限公司 | Multi-sensor simulation data synchronous injection method and system |
| CN115840234A (en) * | 2022-10-28 | 2023-03-24 | 苏州知至科技有限公司 | Radar data acquisition method and device and storage medium |
| CN115840234B (en) * | 2022-10-28 | 2024-04-19 | 苏州知至科技有限公司 | Radar data acquisition method, device and storage medium |
| CN116707746A (en) * | 2023-08-07 | 2023-09-05 | 长春航盛艾思科电子有限公司 | Measuring method of array sensor |
| CN116707746B (en) * | 2023-08-07 | 2024-03-22 | 长春航盛艾思科电子有限公司 | Measuring method of array sensor |
| CN118337313A (en) * | 2024-06-12 | 2024-07-12 | 中国第一汽车股份有限公司 | Multi-sensor data synchronization method, device, equipment and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112541527A (en) | Multi-sensor synchronization method and device, electronic equipment and storage medium | |
| CN109696172B (en) | Multi-sensor track fusion method and device and vehicle | |
| CN112787740A (en) | Multi-sensor time synchronization device and method | |
| CN107014381B (en) | PLD, DSP, integrated navigation system, data processing method and device | |
| US20220121641A1 (en) | Multi-sensor-based state estimation method and apparatus and terminal device | |
| WO2022247915A1 (en) | Fusion positioning method and apparatus, device, storage medium and program product | |
| CN109697247B (en) | Method and device for detecting data accuracy | |
| CN113572556A (en) | Time synchronization method, sensor and system | |
| CN111860551A (en) | Multi-sensor data fusion method and device and vehicle-mounted terminal | |
| CN113411499B (en) | Image fusion method and device, storage medium and electronic device | |
| CN118158141A (en) | Data receiving method, data sending method, system, equipment and medium | |
| CN114168767A (en) | Data labeling method, device, system, equipment and storage medium | |
| CN114006672B (en) | Vehicle-mounted multi-sensor data synchronous acquisition method and system | |
| CN115951344A (en) | Data fusion method and device for radar and camera, electronic equipment and storage medium | |
| CN114710228A (en) | Time synchronization method and device, storage medium and electronic device | |
| AU2021240231A1 (en) | Image synchronization method and apparatus, and device and computer storage medium | |
| CN112866431A (en) | IP address allocation method and device and computer equipment | |
| CN111563546A (en) | Sensor data matching method, device, equipment and storage medium | |
| CN116091725B (en) | 3D large-model web rendering lightweight rapid display method | |
| CN113965289B (en) | Time synchronization method and device based on multi-sensor data | |
| CN117808848B (en) | Identification tracking method and device, electronic equipment and storage medium | |
| CN117809395A (en) | Collected data association method, data recharging method and device of vehicle-mounted equipment | |
| CN117191075A (en) | Visual inertial odometer system for synchronous time synchronization of multiple sensors and time synchronization method thereof | |
| CN116358211A (en) | Intelligent refrigerator, food management method, equipment and medium | |
| CN118337313A (en) | Multi-sensor data synchronization method, device, equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210323 |