WO2020253260A1

WO2020253260A1 - Time synchronization processing method, electronic apparatus, and storage medium

Info

Publication number: WO2020253260A1
Application number: PCT/CN2020/076836
Authority: WO
Inventors: 王潇峰; 刘余钱; 章国锋
Original assignee: 上海商汤临港智能科技有限公司
Priority date: 2019-06-21
Filing date: 2020-02-26
Publication date: 2020-12-24
Also published as: JP2022510418A; CN112113582A; KR20210084622A

Abstract

A time synchronization processing method, comprising: respectively acquiring angular velocity information collected by two different sensors, wherein the angular velocity information is that of an electronic apparatus when performing a rotational motion, and the two different sensor are both disposed on the electronic apparatus and rigidly connected thereto (S101); performing alignment processing with respect to the two acquired pieces of angular velocity information, and determining information of a time delay between the two different sensors (S102); and performing time synchronization processing on respective measurement results of the two different sensors according to the information of the time delay (S103). Further disclosed are a time synchronization processing device, an electronic apparatus, and a computer-readable storage medium.

Description

Time synchronization processing method, electronic equipment and storage medium

Cross references to related applications

This disclosure is filed based on a Chinese patent application with an application number of 201910545218.8 and an application date of June 21, 2019, and claims the priority of the Chinese patent application. The entire content of the Chinese patent application is hereby incorporated into this disclosure by way of introduction.

Technical field

The embodiments of the present disclosure relate to the field of computer vision, and specifically relate to time synchronization processing methods, electronic devices, and storage media.

Background technique

Visual inertial odometry is currently a hot research topic in the field of computer vision. It is widely used in the navigation and entertainment of electronic devices. The main principle is to fuse vision and inertial sensors to estimate the position and posture of the camera itself during movement to obtain accuracy. The positioning information, which belongs to autonomous navigation.

Since different sensors have different trigger delays and transmission delays, the electronic device needs to determine the delay time information between different sensors when fusing the measurement results of different sensors. At present, the motion in the three-dimensional space is usually used to calibrate the delay time information between different sensors, or the time delay between different sensors is processed as a constant. However, during the movement of the electronic device, the delay time information between different sensors changes. Therefore, there is a problem of low time synchronization accuracy.

Summary of the invention

The embodiments of the present disclosure expect to provide a time synchronization processing method, electronic equipment and storage medium.

In a first aspect, an embodiment of the present disclosure provides a time synchronization processing method, the method includes: obtaining angular velocity information collected by two different sensors; the angular velocity information is the angular velocity information when the electronic device rotates, and the Two different sensors are installed on the electronic device and rigidly connected; the two angular velocity information obtained are aligned, and the delay time information between the two different sensors is determined; according to the delay time The information performs time synchronization processing on the respective measurement results of the two different sensors.

In a second aspect, an embodiment of the present disclosure provides a time synchronization processing device, the device includes: a first acquisition module, an alignment module, and a synchronization module, wherein the first acquisition module is configured to obtain two different sensors respectively The angular velocity information is collected; the angular velocity information is the angular velocity information when the electronic device rotates, and the two different sensors are both arranged on the electronic device and rigidly connected; the alignment module is used to compare the two obtained Aligning the two angular velocity information to determine the delay time information between the two different sensors; the synchronization module is configured to perform the respective measurement results of the two different sensors according to the delay time information Time synchronization processing.

In a third aspect, embodiments of the present disclosure provide an electronic device that includes at least a processor and a memory for storing a computer program that can run on the processor; when the processor is used to run the computer program, Perform the steps in the above-mentioned time synchronization processing method.

In a fourth aspect, embodiments of the present disclosure provide a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps in the above-mentioned time synchronization processing method are implemented.

The embodiments of the present disclosure provide a time synchronization processing method, an electronic device, and a storage medium. The angular velocity information collected by the two different sensors is obtained through two different sensors that are arranged on the electronic device and rigidly connected; The two angular velocity information is aligned to determine the delay time information between two different sensors, and then the respective measurement results of the two different sensors are time synchronized according to the delay time information, thereby making full use of the rigid connection angular velocity The principle of the same information realizes the time synchronization of different sensors.

Description of the drawings

FIG. 1 is a schematic diagram 1 of a system structure suitable for a time synchronization processing method according to an embodiment of the present disclosure;

2A is a schematic diagram 2 of a system architecture suitable for a time synchronization processing method according to an embodiment of the present disclosure;

2B is a third schematic diagram of a system architecture suitable for a time synchronization processing method according to an embodiment of the present disclosure;

3 is a schematic diagram 1 of the implementation process of a time synchronization processing method provided by an embodiment of the present disclosure;

4 is a schematic diagram of an exemplary time delay between the angular velocity of the visual sensor and the angular velocity of the inertial sensor in the embodiment of the disclosure;

FIG. 5 is a schematic diagram after the angular velocity of the visual sensor and the angular velocity of the inertial sensor are synchronized in an exemplary embodiment of the disclosure;

6 is a second schematic diagram of the implementation process of a time synchronization processing method provided by an embodiment of the disclosure;

FIG. 7 is a third schematic diagram of the implementation process of a time synchronization processing method provided by an embodiment of the disclosure;

FIG. 8 is a fourth schematic diagram of the implementation process of a time synchronization processing method provided by an embodiment of the present disclosure;

FIG. 9 is a first schematic diagram of measuring pose rotation matrix information by using a quaternion to represent a pose sensor in an embodiment of the disclosure;

FIG. 10 is a second schematic diagram of the embodiment of the disclosure using a quaternion to represent the pose sensor to measure the pose rotation matrix information;

FIG. 11 is a schematic diagram of the composition structure of a time synchronization processing device provided by an embodiment of the disclosure;

FIG. 12 is a schematic diagram of the composition structure of an electronic device provided by an embodiment of the disclosure.

Detailed ways

In order to make the objectives, technical solutions and advantages of the embodiments of the present disclosure clearer, the embodiments of the present disclosure will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the embodiments of the present disclosure. All other embodiments obtained by technicians without creative work fall within the protection scope of the embodiments of the present disclosure.

Fig. 1 is a schematic diagram 1 of a system structure suitable for a time synchronization processing method provided by an embodiment of the present disclosure. As shown in Fig. 1, the system may include a processor 11, two different sensors 12, and a memory 13, two different The sensors 12 respectively send the acquired information to the processor 11 for processing. One of the two different sensors 12 may be a sensor that directly measures angular velocity, and the other sensor may be a sensor that indirectly measures angular velocity. The sensor that indirectly measures angular velocity is capable of independently estimating the rotational movement of the sensor itself (that is, obtaining posture rotation). Matrix information) sensor. The two different sensors 12 may also be sensors of other structures, and the embodiment of the present disclosure does not limit the structure of the two different sensors.

Among them, the two sensors of the two different sensors 12 are rigidly fixed, that is, the two different sensors 12 of the electronic device are relatively fixed. During the movement, the angular velocity measured by different sensors at the same time is the same. Two different sensors 12 respectively input the acquired information into the processor 11, and the processor 11 performs time synchronization processing by executing the method provided in the embodiment of the present disclosure.

In some embodiments, the electronic device may include a time synchronization processing device and two different sensors 12; the time synchronization processing device may include the aforementioned processor 11 and memory 13. By arranging the above two different sensors 12 on the electronic device, the angular velocity information collected by the two different sensors is obtained respectively, and the two different sensors 12 input the obtained information into the processor 11, and the processor 11 executes this The method provided by the disclosed embodiment performs time synchronization processing.

In other embodiments, the system architecture of the time synchronization processing method may also be shown in FIGS. 2A and 2B. The system includes a time synchronization processing device 20, a first sensor 22a and a second sensor 22b, and a first sensor 22a and a second sensor 22a. The second sensor 22b is arranged on the electronic equipment of the vehicle, for example, the time synchronization processing device 20 may be an on-board equipment in the vehicle; the first sensor 22a and the second sensor 22b may send the acquired information to the time synchronization through an electrical connection or a wireless communication connection处理装置20。 Processing device 20. It can be understood that the time synchronization processing device 20 has a processor and a memory. The first sensor 22a and the second sensor 22b send the acquired information to the time synchronization processing device 20, which is processed by the processor to implement the information provided by the embodiments of the present disclosure. The method of time synchronization.

As shown in FIGS. 2A and 2B, the first sensor 22a and the second sensor 22b may be arranged at different positions of the vehicle, or may be arranged at the same position of the vehicle. For example, as shown in FIG. 2B, the first sensor 22a and the second sensor 22b may both be arranged on the tire of the vehicle; for another example, as shown in FIG. 2A, the first sensor 22a may also be arranged on the tire of the vehicle. The second sensor 22b is arranged in other parts of the vehicle, for example, in the vehicle-mounted device part. The embodiment of the present disclosure does not limit the deployment positions of the first sensor 22a and the second sensor 22b.

The processor processes the information sent by different sensors in different ways. For example, when the sensor is a sensor that directly measures angular velocity, the processor can directly process the obtained angular velocity information; when the sensor is a sensor that indirectly measures angular velocity, the processor first obtains the pose rotation matrix information, and then according to the pose rotation matrix Information acquires angular velocity information, and then processes the acquired angular velocity information.

Generally speaking, the time synchronization processing apparatus can be applied to various types of electronic devices with information processing capabilities during implementation. The time synchronization processing device can obtain the data collected by two different sensors in real time in an online manner, and use the technical solutions of the embodiments of the present disclosure to perform time synchronization processing. The time synchronization processing device may also obtain data collected by two different sensors in an offline manner, and use the technical solutions of the embodiments of the present disclosure to perform time synchronization processing. Wherein, obtaining the data collected by two different sensors in an offline manner may include: storing the data collected by two different sensors, and exporting the stored data when needed, so as to synchronize the respective measurement results of the different sensors; Obtaining the data collected by two different sensors in an online manner may include: obtaining data collected by two different sensors in real time, so as to synchronize the respective measurement results of the different sensors.

Among them, the time synchronization processing device can be arranged in the electronic equipment or outside the electronic equipment; two different sensors are arranged in the electronic equipment. Among them, the electronic device may be, for example, a mobile robot device, an unmanned device, or various types of mobile terminals and other devices. The unmanned device may include, but is not limited to, a vehicle, an airplane, or a ship, which is not limited in the embodiment of the present disclosure.

Based on the foregoing architecture, embodiments of the present disclosure propose a time synchronization processing method, which can solve the problem of low and complex time delay calibration between different sensors. The functions implemented by the time synchronization processing method can be implemented by the processor in the time synchronization processing device calling executable instructions. Of course, the executable instructions can be stored in the storage medium of the memory. It can be seen that the time synchronization processing device at least includes a processor And storage media.

FIG. 3 is a schematic diagram 1 of the implementation process of a time synchronization processing method provided in an embodiment of the present disclosure, which is applied to a time synchronization processing device. As shown in FIG. 3, the time synchronization processing method includes:

S101. Obtain angular velocity information collected by two different sensors; the angular velocity information is angular velocity information when the electronic device rotates, and the two different sensors are both arranged on the electronic device and rigidly connected.

In the embodiment of the present disclosure, two different sensors are rigidly connected, that is, the two different sensors are relatively fixed. During the rotational movement of the electronic device, the angular velocity represented by the angular velocity information acquired by two different sensors at the same time is the same.

It should be noted that the two different sensors in the embodiments of the present disclosure include sensors with different structures, or sensors with the same structure but different deployment positions.

For example, if two different sensors are sensors with different structures, the two different sensors may include any two of the following structures: vision sensors, inertial sensors, magnetic sensors, lidar sensors, and wheel sensors. There is no restriction here.

For another example, if two different sensors are sensors with the same structure but different positions, the two different sensors may be vision sensors or magnetic sensors deployed in different positions of the electronic device, and the embodiments of the present disclosure are not limited here.

In the embodiment of the present disclosure, the angular velocity information may be the angular velocity information when the electronic device rotates in a preset period of time, which is obtained by two different sensors. In other words, when the electronic device rotates, it rotates by a certain angle, so that the angular velocity information collected by different sensors can be obtained within a preset time period.

It should be noted that the foregoing preset time period may be set by the user according to actual conditions. For example, the preset time period may be set to half an hour or fifteen minutes, etc., which is not limited in the embodiments of the present disclosure.

It should be noted that the above two different sensors can be classified based on acquisition methods such as direct acquisition of angular velocity information or indirect acquisition of angular velocity information. Exemplarily, one type of sensor is a gyro sensor, which can directly measure angular velocity information; the other type of sensor is a pose sensor, which can indirectly measure angular velocity information. Alternatively, the above two different sensors can also be classified based on the structure of the sensor. Exemplarily, the gyro sensor and the attitude sensor are sensors with different structures.

In some optional embodiments of the embodiments of the present disclosure, at least one of the two different sensors is a pose sensor, and the obtaining angular velocity information collected by the two different sensors separately includes: obtaining a position The pose rotation matrix information collected by the pose sensor; the angular velocity information when the electronic device rotates is determined by the pose rotation matrix information.

In some optional embodiments of the embodiments of the present disclosure, at least one of the two different sensors is a gyro sensor, and obtaining angular velocity information collected by the two different sensors respectively includes: obtaining the information collected by the gyro sensor Angular velocity information.

In the embodiments of the present disclosure, acquiring angular velocity information when an electronic device undergoes a rotational movement through two different sensors includes the following application scenarios:

Scenario 1: Two different sensors are gyroscopic sensors, but the deployment positions of the two gyroscopic sensors are different, and the angular velocity information of the electronic device when the electronic device rotates is directly collected through the two gyroscopic sensors.

Scenario 2: The two different sensors are both pose sensors, but the deployment positions of the two pose sensors are different, and the pose rotation matrix information when the electronic device rotates is obtained through the two pose sensors. Then, the angular velocity information when the electronic device rotates is calculated through the information of the pose rotation matrix.

Scenario 3: Among the two different sensors, one is a pose sensor and the other is a gyro sensor; on the one hand, the pose rotation matrix information when the electronic device rotates can be obtained through the pose sensor, and then pass the position The posture rotation matrix information is calculated to obtain the angular velocity information when the electronic device rotates; on the other hand, the angular velocity information when the electronic device rotates can be directly collected through the gyro sensor.

Exemplarily, the gyro sensor includes any one of an inertial sensor, a magnetic sensor, and a wheel-type odometer sensor, and the attitude sensor includes any one of a vision sensor and a lidar sensor.

S102: Perform alignment processing on the two obtained angular velocity information, and determine delay time information between two different sensors.

In the embodiments of the present disclosure, since the data acquisition frequency of different sensors is different, after obtaining the two angular velocity information, the two angular velocity information need to be aligned to determine the delay time information between the two different sensors.

It should be noted that the alignment processing of the two angular velocity information in the embodiment of the present disclosure is to make the angular velocity information acquired by the two different sensors be consistent in frequency, so that the two angular velocity information can be determined based on the aligned two angular velocity information. Delay time information between two different sensors.

In the embodiment of the present disclosure, the two angular velocities can be interpolated to align the two angular velocity information.

Exemplarily, the interpolation processing may adopt any interpolation processing method among linear interpolation, cubic spline interpolation, and spherical interpolation, which is not limited in the embodiment of the present disclosure.

S103: Perform time synchronization processing on the respective measurement results of the two different sensors according to the delay time information.

In the embodiment of the present disclosure, after the delay time information between two different sensors is determined, the respective measurement results of the two different sensors can be time synchronized according to the delay time information. It should be noted that the time synchronization processing of the respective measurement results of two different sensors is to determine the respective measurement results of the two different sensors at the same time, so as to solve the trigger delay and transmission delay caused by the different sensors. The problem of low time synchronization accuracy.

In the embodiment of the present disclosure, the respective measurement results of the two different sensors are the measurement results obtained by the two different sensors respectively measuring the rotational movement of the electronic device when the electronic device undergoes rotational movement.

Exemplarily, the measurement results obtained by the two different sensors may include at least one of angular velocity information, azimuth angle information, and acceleration information. For example, the measurement result of the inertial sensor may include angular velocity information and acceleration information of the rotational movement of the electronic device; the measurement result of the magnetic sensor may include the azimuth angle information of the rotational movement of the electronic device.

In an embodiment, at least three sensors may also be provided on the electronic device, and the at least three sensors are rigidly connected; the angular velocity information collected by the at least three sensors is obtained respectively; and the three angular velocity information obtained is performed Alignment processing determines the delay time information between at least three sensors; according to the delay time information, time synchronization processing is performed on the respective measurement results of the at least three sensors.

It should be noted that when there are at least three sensors on the electronic device, at least three sensors can be combined in pairs to obtain the delay time information between any two sensors, and then the delay time information Time synchronization processing is performed on the respective measurement results of the two sensors corresponding to the same time, so as to realize the time synchronization processing of at least three sensors.

Exemplarily, in the case where the two different sensors provided on the electronic device are the visual sensor and the inertial sensor, the delay time information between the visual sensor and the inertial sensor can be acquired, and the visual effect can be adjusted based on the delay time information. The measurement result of the sensor and the measurement result of the inertial sensor are processed in time synchronization to realize the time synchronization of the measurement result of the visual sensor and the measurement result of the inertial sensor.

Exemplarily, in the case where the three sensors provided on the electronic device are respectively a vision sensor, an inertial sensor, and a lidar sensor, the delay time information between the vision sensor and the inertial sensor may be determined first, based on the delay time information The measurement result of the vision sensor and the measurement result of the inertial sensor are time-synchronized to realize the time synchronization of the measurement result of the vision sensor and the measurement result of the inertial sensor; then the delay time information between the vision sensor and the lidar sensor is determined, based on The delay time information performs time synchronization processing on the measurement result of the vision sensor and the measurement result of the lidar sensor, and realizes the time synchronization of the measurement result of the vision sensor and the measurement result of the lidar sensor. In this way, through two time synchronization processing, The time synchronization of the measurement results between the vision sensor, inertial sensor and lidar sensor.

In order to more clearly show the difference between the measurement results of two different sensors before and after synchronization, the following uses one of the two different sensors as a vision sensor, the other sensor as an inertial sensor, and the measurement result is angular velocity information as For example, the time synchronization processing in the embodiment of the present disclosure will be described.

FIG. 4 is a schematic diagram showing the time delay between the angular velocity information of the visual sensor and the angular velocity information of the inertial sensor in an exemplary embodiment of the disclosure. FIG. 5 is a schematic diagram after the angular velocity of the visual sensor and the angular velocity of the inertial sensor are synchronized in an exemplary embodiment of the disclosure. In Figures 4 and 5, the dotted line represents the angular velocity of the vision sensor, and the solid line represents the angular velocity of the inertial sensor.

It can be seen from Figure 4 that the angular velocity curve of the visual sensor is staggered from the angular velocity curve of the inertial sensor, and the angular velocity curve of the visual sensor is lagging. After synchronizing the angular velocity of the visual sensor through the delay time information between the visual sensor and the inertial sensor, it can be seen from Fig. 5 that the angular velocity curve of the visual sensor and the angular velocity curve of the inertial sensor are aligned and not staggered, thus According to the determined delay time information, the time synchronization processing of the angular velocity information between the visual sensor and the inertial sensor is realized.

In the embodiments of the present disclosure, after the delay time information between two different sensors is determined, the respective measurement results of the two different sensors can be directly synchronized in time according to the delay time information.

In an embodiment, after aligning the obtained two angular velocity information and determining the delay time information between two different sensors, the delay time information may also be stored; when the electronic device is in a non-moving state In this case, according to the stored delay time information, the respective measurement results of the two different sensors are time synchronized.

The technical solutions of the embodiments of the present disclosure can realize both online time synchronization processing and offline time synchronization processing, and the time synchronization processing is more flexible. At the same time, the time synchronization processing in the embodiments of the present disclosure does not need to use a calibration reference object, such as a checkerboard image, and the time synchronization is more convenient and simple, and has strong universality.

The embodiments of the present disclosure require the electronic device to be capable of rotating movement, that is, the electronic device is required to only need to rotate around the rotation axis at least to calibrate the delay time information between different sensors on the electronic device, without having to require movement around multiple axes , The complexity of time synchronization of different sensors is reduced, and it can be adapted to the needs of different scenarios; and the embodiments of the present disclosure can also calibrate the delay time based on the rotation motion generated by the electronic device rotating around multiple axes, so that more abundant rotation information can be obtained , In order to improve the accuracy of time synchronization; the embodiments of the present disclosure require sensors to independently obtain angular velocity information to determine the delay time information between different sensors, which can be widely used for time synchronization between multiple sensors. Universal adaptability; the embodiment of the present disclosure is a software-based method for realizing time synchronization, without additional deployment of dedicated hardware for time synchronization; the embodiment of the present disclosure performs time synchronization based on the delay time information obtained in real time, and can perform time synchronization online Processing: The embodiment of the present disclosure determines the delay time information through the angular velocity information obtained when the electronic device rotates, instead of treating the delay time information as a constant, and improves the accuracy of time synchronization.

FIG. 6 is a second schematic diagram of the implementation process of a time synchronization processing method provided by an embodiment of the present disclosure. As shown in FIG. 6, in the embodiment of the present disclosure, the angular velocity information acquired by two different sensors is aligned to determine The delay time information between two different sensors, namely S102, can include S102a and S102b, as follows:

S102a: Perform interpolation processing on at least one of the two angular velocity information to align the two angular velocity information.

In the embodiments of the present disclosure, the interpolation processing of sensors of different structures may be different from the processing of sensors of the same structure. Therefore, based on the different structures of the two sensors, interpolation processing can be performed on at least one of the two angular velocity information to align the two angular velocity information.

For example, when the two different sensors are both gyroscopic sensors, interpolation processing is performed on at least one of the obtained angular velocity information collected by the two gyroscopic sensors to align the two angular velocity information.

For another example, in the case where the two different sensors are both pose sensors, interpolation processing is performed on at least one of the obtained pose rotation matrix information collected by the two pose sensors to align the two angular velocities information.

For another example, in the case where two different sensors are a pose sensor and a gyro sensor, interpolation processing is performed on the pose rotation matrix information collected by the obtained pose sensor to align the two angular velocity information.

It should be noted that performing interpolation processing on at least one of the two angular velocity information includes performing interpolation processing on both angular velocity information and performing interpolation processing on any one of the two angular velocity information.

In an embodiment, performing interpolation processing on the two angular velocity information includes: selecting a data acquisition frequency as the standard data acquisition frequency, and performing interpolation processing on the two angular velocity information according to the standard data acquisition frequency. Wherein, the standard data acquisition frequency may be between the respective data acquisition frequencies of the two sensors or higher than the respective data acquisition frequencies of the two sensors.

Exemplarily, if the data acquisition frequencies of the two sensors are 10 Hz and 15 Hz, respectively, the data acquisition frequency higher than the two sensors can be selected as the standard data acquisition frequency, for example, the standard data acquisition frequency is 17 Hz. Of course, a frequency between the data acquisition frequencies of the two sensors can also be selected as the standard data acquisition frequency, for example, the standard data acquisition frequency is 13 Hz.

If the selected standard data acquisition frequency is 13Hz, the angular velocity information acquired by the two sensors is interpolated according to the standard data acquisition frequency, so that the angular velocity acquisition frequency of the first sensor and the angular velocity acquisition frequency of the second sensor are respectively the same as the standard acquisition frequency , Both are 13Hz. In this way, the problem of deviations in data processing caused by inconsistent data acquisition frequencies of different sensors can be solved, which helps to obtain more accurate delay time information based on the aligned angular velocity information.

In an embodiment, it is assumed that two different sensors are respectively denoted as the first sensor and the second sensor. The data acquisition frequency of the first sensor is higher than the data acquisition frequency of the second sensor; The interpolation processing includes: performing interpolation processing on the acquired angular velocity information collected by the second sensor according to the data acquisition frequency of the first sensor.

Exemplarily, if the data acquisition frequency of the first sensor is 15 Hz and the data acquisition frequency of the second sensor is 10 Hz, then the angular velocity information acquired by the second sensor can be interpolated according to the data acquisition frequency of the first sensor 15 Hz, so that the The angular velocity acquisition frequency of one sensor is the same as the angular velocity information acquisition frequency of the second sensor, and both are 15 Hz. This solves the problem of deviations in data processing due to inconsistent data acquisition frequencies of different sensors, and helps to obtain more accurate delay time information based on the aligned angular velocity information.

In an embodiment, when the two different sensors on the electronic device are a pose sensor and a gyro sensor, the obtained pose rotation matrix information collected by the pose sensor is subjected to interpolation processing to Align the two angular velocity information. Specifically, it can include S01, S02 and S03, as follows:

S01. Obtain adjacent pose rotation matrix information from multiple pose rotation matrix information.

In the embodiment of the present disclosure, after obtaining multiple pose rotation matrix information, adjacent pose rotation matrix information may be obtained from the multiple pose rotation matrix information. It should be noted that the pose rotation matrix information collected by the pose sensor in the preset time period is multiple pose rotation matrix information.

Exemplarily, when the pose sensor is a vision sensor, the pose rotation matrix information of the adjacent frame image can be obtained; when the pose sensor is a lidar sensor, it can be obtained separately by the lidar sensor The pose rotation matrix information in the preset time period is used to estimate the rotation movement of the electronic device.

S02: Perform interpolation processing on adjacent pose rotation matrix information according to the frequency at which the gyro sensor obtains angular velocity information, and obtain interpolated rotation matrix information.

In the embodiment of the present disclosure, after obtaining the adjacent pose rotation matrix information, the interpolated rotation matrix information may be obtained according to the adjacent pose rotation matrix information.

It should be noted that in the process of obtaining the interpolated rotation matrix information according to the adjacent pose rotation matrix information, an interpolation model can be constructed, and then the interpolation model and the adjacent pose rotation matrix information can be used to obtain the interpolated rotation matrix information. Rotation matrix information.

It is understandable that the interpolation model is used to estimate other adjacent pose rotation information between adjacent pose rotation information based on the value status of adjacent pose rotation information at a limited number of points.

Exemplarily, the interpolation model may include a spherical linear interpolation (Spherical Linear Interpolation) model, a cubic spline interpolation (Cubic Spline Interpolation) model, and a nearest neighbor interpolation model. The embodiment of the present disclosure does not limit the difference model.

S03. Perform differentiation processing on the interpolated rotation matrix information to obtain angular velocity information corresponding to the attitude sensor, so as to align the two angular velocity information acquired by the attitude sensor and the gyro sensor.

In the embodiments of the present disclosure, after obtaining the interpolated rotation matrix information, the interpolated rotation matrix information may be differentiated according to a geometric manner to obtain angular velocity information corresponding to the pose sensor.

It should be noted that the differential model can be constructed in the process of performing differentiation processing on the interpolated rotation matrix information to obtain angular velocity information corresponding to the pose sensor. Through the constructed differential model and the interpolated rotation matrix information, the angular velocity information corresponding to the pose sensor is obtained.

Illustratively, the differential model satisfies formula (1).

Among them, q(t)' is the differential pose rotation matrix information, w is the second sub-angular velocity information, and q(t) is the interpolated pose rotation matrix information.

It should be noted that the differential model in the embodiment of the present disclosure is a model used to characterize angular velocity information. In some possible implementations, the differential model may be a quaternion differential model or a model in other mathematical expression forms, which is not limited in the embodiment of the present disclosure.

S102b: Determine the delay time information between two different sensors according to the aligned two angular velocity information.

In the embodiments of the present disclosure, considering the rotational movement process of the electronic device, the angular velocity information acquired by different sensors at the same time is the same. Therefore, the angular velocity information between the two different sensors can be determined according to the aligned two angular velocity information. Delay time information.

It should be noted that before determining the delay time information between two different sensors according to the aligned two angular velocity information, an error model can be constructed first, and then based on the aligned two angular velocity information and the error model , To determine the delay time information between two different sensors.

In the embodiments of the present disclosure, in the process of constructing the error model, since different sensors correspond to different coordinate axes, when comparing angular velocity information between different sensors, it is required to be under the coordinate axes corresponding to the same coordinate system. Therefore, When constructing the error model, the rotation parameters between the different coordinate axes corresponding to the two different sensors are introduced, so that the angular velocity information of the two different sensors at the same time is under the same coordinate axis.

Taking into account the deviation of angular velocity information obtained by different sensors, when constructing the error model, the error parameter between the two sensors is introduced, so that the delay time information determined by the error model can be more accurate.

Exemplarily, the error model can be as shown in formula (2):

f(Q ₁₂ ,td,bg)=||w ₁ (t)-[Q ₁₂ w ₂ (t+td)-bg]|| ² (2)

Among them, Q ₁₂ is the rotation parameter between the different coordinate axes corresponding to two different sensors; td is the delay time information between the two different sensors; bg is the error parameter between the two different sensors, w ₁ (t) and w ₂ (t) are respectively the angular velocity information acquired by two different sensors at the same time t, and f(Q ₁₂ , td, bg) is the error between the angular velocities.

In an embodiment, after aligning the obtained two angular velocity information and determining the delay time information between the two different sensors, the method further includes: aligning the two angular velocity information, Determine the external parameters between two different sensors. The external parameters include the rotation parameters between the different coordinate axes corresponding to the two different sensors and the error parameters between the two different sensors.

In the embodiments of the present disclosure, through the aligned two angular velocity information, not only the delay time information can be obtained, but also the rotation parameters and error parameters between two different sensors can be obtained, so that the functions of the electronic device are more abundant.

FIG. 7 is a schematic diagram of the third implementation flow of a time synchronization processing method provided by an embodiment of the disclosure. As shown in FIG. 7, the delay time information between two different sensors is determined according to the aligned two angular velocity information, namely Step S102b may include S102b1, S102b2, and S102b3, as follows:

S102b1, according to the aligned two angular velocity information, determine the sub-error equations corresponding to the two different sensors at different times.

In the embodiment of the present disclosure, after the two angular velocity information is aligned, the sub-error equations corresponding to the two different sensors at different moments during the rotation of the electronic device can be determined according to the aligned two angular velocity information.

It should be noted that n can be preset at different times within the preset time period according to actual conditions. For example, 15 moments or 20 moments can be set within the preset time period, and then the sub-error equation corresponding to each moment can be obtained.

Exemplarily, if the angular velocity information collected by different sensors corresponding to time 1 are respectively w ₁ (1) and w ₂ (1+td); the angular velocity information collected by different sensors corresponding to time 2 are respectively w ₁ (2) And w ₂ (2+td), and so on, the angular velocity information collected by different sensors corresponding to time n is w ₁ (n) and w ₂ (n+td) respectively, then when the error model satisfies formula (2), The corresponding sub-error equations corresponding to different times n are:

The sub-error equation corresponding to time 1 satisfies formula (3):

f(1)=||w ₁ (1)-[Q ₁₂ w ₂ (1+td)-bg]|| ² (3)

The sub-error equation corresponding to time 2 satisfies formula (4):

f(2)=||w ₁ (2)-[Q ₁₂ w ₂ (2+td)-bg]|| ² (4)

By analogy, the sub-error equation corresponding to time n satisfies formula (5):

f(n)=||w ₁ (n)-[Q ₁₂ w ₂ (n+td)-bg]|| ² (5)

S102b2, sum up the corresponding sub-error equations at different times to obtain the final error equation.

In the embodiment of the present disclosure, after the sub-error equations corresponding to each time are determined, the sub-error equations corresponding to different times are summed to obtain the final error equation.

It should be noted that the embodiment of the present disclosure obtains the delay time information by obtaining the accumulated errors at different times, and further requires summing the sub-error equations corresponding to different times to obtain the final error equation.

Exemplarily, when the corresponding sub-error equations at different times are the above formula (3), formula (4), and formula (5) respectively, the corresponding final error equation satisfies formula (6):

f(Q ₁₂ 、td、bg)=f(1)+f(2).....+f(n) (6)

S102b3: Perform minimum processing on the final error equation to obtain delay time information.

In the embodiments of the present disclosure, after the final error equation is obtained, the final error equation can be processed to a minimum value to obtain delay time information.

It should be noted that the above-mentioned minimum value processing is to minimize the final error equation value, and then estimate the delay time information in the error model, the rotation parameter, and the error parameter between the sensors.

Exemplarily, the minimum processing of the final error equation may be performed through a nonlinear model or an iterative closest point model to perform the minimum processing on the final error equation.

In the embodiment of the present disclosure, the process of performing minimum value processing on the final error equation to obtain delay time information may include: performing iterative nearest point processing on the final error equation to obtain the second minimization equation; and solving the second minimization equation , Until the preset second threshold is met, the delay time information in the second minimization equation is acquired.

It should be noted that the iterative closest point processing is performed on the final error equation to obtain the second minimization equation, and the delay time can be set between the preset time periods. For example, you can use the golden section method to select a delay time and then substitute it into the error model. The error term obtained at this time is only the two unknowns of the rotation parameter and the error parameter between the sensors. Here, it can be processed by iterative closest point. Then the rotation parameters and the error parameters between the sensors can be obtained.

Exemplarily, assuming that the delay time information is between [-1,1], use the golden section method to select a delay time of 0.3. At this time, the second minimization equation obtained satisfies the formula (7):

Among them, f(Q ₁₂ ,bg) is the error between angular velocities, n is the number of the nearest point pairs, Q ₁₂ is the rotation matrix between the two sensors, bg is the error parameter between the two sensors, w _i1 (t) and w _i2 (t) are respectively a point in the point cloud in the angular velocity corresponding to the two sensors.

In the embodiments of the present disclosure, the process of performing minimum value processing on the final error equation to obtain delay time information may further include: performing nonlinear optimization processing on the final error equation to obtain the first minimization equation; Solve, until the preset first threshold is met, obtain the delay time information in the first minimization equation.

It should be noted that since the final error equation is a non-linear function, it needs to be Taylor expanded. By minimizing the final error equation, it is necessary to find the corresponding delay time information when the preset first threshold is met, so as to make the final error The equation drops to a minimum.

In the embodiment of the present disclosure, nonlinear optimization processing is performed on the final error equation, and a nonlinear optimization model can be constructed in the process of obtaining the first minimization equation. Based on the final error equation and the nonlinear optimization model, the first minimization equation is determined.

Exemplarily, the preset nonlinear optimization model may include a Gauss-Newton algorithm model or a Levenberg-Marquardt algorithm model, and the first threshold can be set according to the actual needs of the user. If it is 0.1 or 0.01, the embodiments of the present disclosure are not limited here.

In the embodiment of the present disclosure, when the first minimization equation is solved until the preset first threshold is satisfied, obtaining the delay time information in the first minimization equation includes: when it is determined that the delay time is determined for the first time Information, according to the preset initial variable value and the preset nonlinear optimization model, obtain the current variable value; according to the initial variable value, the current variable value and the first minimization equation, determine the current solution value of the minimized equation, when the current solution value When the preset first threshold is met, the delay time information in the first minimization equation is obtained.

It should be noted that the first minimization equation is composed of the final error equation corresponding to the adjacent variable values. For example, the first minimization equation can be formula (8).

e=f(x ^k+1 )-f(x ^k ) (8)

Among them, x ^k+1 is the current variable value, x ^k is the initial variable value, and e is the current solution value of the minimization equation.

When determining whether it is the first time to determine the delay time information, obtain the last variable value; obtain the current variable value according to the last variable value and the preset nonlinear optimization model; obtain the current variable value according to the last variable value, current variable value and first minimization Equation to determine the current solution value of the minimization equation, and when the current solution value meets the preset first threshold, obtain the delay time information in the first minimization equation.

It should be noted that when the current solution value does not meet the preset first threshold, the next variable value needs to be obtained according to the current variable value and the preset nonlinear optimization model; according to the next variable value, the current variable value and the first Minimize the equation, determine the next solution value of the minimized equation, and determine in turn whether the next solution value meets the preset first threshold, until the next solution value meets the preset first threshold, the iteration ends, and the first Minimize the delay time information in the equation.

In the embodiments of the present disclosure, while obtaining delay time information by minimizing the final error equation, rotation parameters and sensor deviation parameters can also be obtained. In this way, the obtained rotation parameters can transform the measurement information of different coordinate systems into In the same coordinate system.

In an embodiment, after performing time synchronization processing on the respective measurement results of the two different sensors according to the delay time information, it may further include: performing fusion processing on the synchronized measurement results; The measurement result of the fusion processing performs at least one of the following operations: positioning processing, ranging processing, target detection of the scene where the electronic device is located, generating or updating a map.

In the embodiment of the present disclosure, performing fusion processing on two synchronized measurement results includes: analyzing and synthesizing two measurement results at the same time to obtain a reliable fusion processing result.

Exemplarily, applying the result of the fusion processing to the positioning process can achieve precise positioning of the electronic device; applying the result of the fusion processing to the ranging process can improve the measurement accuracy; applying the result of the fusion processing to the In the target detection process of the scene where the electronic device is located, accurate target detection results can be obtained; when the fusion processing result is applied to the process of generating or updating a map, an accurate map can be obtained.

In the embodiments of the present disclosure, in the process of fusing the synchronized measurement results, a fusion algorithm model can be constructed to obtain accurate fusion processing results.

Exemplarily, the fusion algorithm model may include a Kalman filter fusion algorithm model and a cluster analysis recognition algorithm model, which is not limited in the embodiment of the present disclosure.

In an embodiment, the Kalman filter fusion algorithm model can be used to perform fusion processing on the synchronized measurement results, including: using the measurement results of two different sensors set by the electronic device to perform state propagation, so that the previous moment's The pose estimation of the first measurement result obtains the pose estimation at the current moment, and then uses the second measurement result at the current moment as observation information to correct the preliminary estimation of the pose at the current moment, thereby obtaining an optimal pose at the current moment The best estimate is the result of the fusion processing.

It should be noted that the time synchronization processing method in the embodiment of the present disclosure can also be applied to more than three sensors. When at least three sensors are provided on the electronic device, the measurement results after the time synchronization processing of every two sensors are acquired, and the measurement results after the time synchronization processing are fused to obtain the result of the fusion processing. Compared with fusion of measurement results corresponding to only two sensors, more accurate measurement results can be obtained by fusing the measurement results corresponding to at least three sensors.

FIG. 8 is a fourth schematic diagram of the implementation process of a time synchronization processing method provided by an embodiment of the present disclosure. As shown in FIG. 8, in an embodiment of the present disclosure, two different sensors provided on an electronic device are a vision sensor and an inertial sensor. The time synchronization processing method of this embodiment may include the following steps:

S201. Obtain first angular velocity information when the electronic device rotates through the inertial sensor.

In the embodiments of the present disclosure, the inertial sensor on the electronic device can independently obtain angular velocity information. Since the inertial sensor is provided with a gyro unit, the angular velocity information when the electronic device rotates can be directly obtained through the gyro unit.

It should be noted that the inertial sensor is a sensor that measures the three-axis attitude angle (or angular rate) and acceleration. In addition to the gyro unit, the inertial sensor can also include an acceleration unit, where the acceleration unit can detect the three-axis position of the electronic device. Acceleration information of the axis.

S202: Obtain the position and posture rotation matrix information when the electronic device rotates through the vision sensor.

In the embodiment of the present disclosure, the visual sensor on the electronic device can independently obtain the pose rotation matrix information. The vision sensor is provided with a pose estimation unit, which can obtain the pose rotation matrix information when the electronic device rotates.

It should be noted that the vision sensor is mainly composed of one or two graphics sensors, and is also equipped with a light projector and other auxiliary equipment. It can acquire the original image within a preset time period and store the acquired image in the memory. The benchmarks are compared and analyzed to obtain the pose rotation matrix information, and then calculate the pose rotation matrix information of the electronic device.

Exemplarily, the rotational movement of the electronic device can be represented by a quaternion; the rotational movement of the electronic device can also be represented by a three-dimensional rotation group, which is not limited in the embodiment of the present disclosure.

S203: Determine the second angular velocity information according to the pose rotation matrix information, and align the second angular velocity information with the first angular velocity information.

In the embodiments of the present disclosure, since the image frequency of the vision sensor is usually 10 Hz, and the frequency of the inertial sensor is usually 100 Hz, there is close to the first angular velocity information measured by 10 inertial sensors between the two frames of the vision sensor. Therefore, Interpolation processing is performed on the pose rotation matrix information of the vision sensor to make the frequency of the vision sensor and the inertial sensor consistent, and the second angular velocity information obtained through the interpolated pose rotation matrix information can be aligned with the first angular velocity information.

In this embodiment, at least one of the second angular velocity information and the first angular velocity information can be interpolated through an interpolation model. It should be noted that, based on the different representations of the rotational motion of the electronic device, the interpolation models used are also different. For example, for the rotational motion represented by the quaternion, the pose rotation matrix can be interpolated through the spherical linear interpolation model; for the rotational motion represented by the three-dimensional rotation group, the cubic spline interpolation model and the nearest neighbor interpolation model can be used. Interpolate the pose rotation matrix information.

For the rotational motion represented by a quaternion, four parameters are required to represent the three-dimensional rotation; for the rotational motion represented by the three-dimensional rotation group, nine parameters are required to represent the three-dimensional rotation.

In the following, the rotation motion represented by a quaternion is used as an example to illustrate the interpolation processing process of the pose rotation matrix information of the vision sensor using a spherical linear interpolation model.

Exemplarily, a quaternion is composed of a real number plus an imaginary number, as in formula (9):

q(e ₀ ,e ₁ ,e ₂ ,e ₃ )=e ₀ +e ₁ i+e ₂ j+e ₃ k (9)

Among them, e ₀ , e ₁ , e ₂ , and e ₃ are real numbers, i, j, and k are mutually orthogonal imaginary number units, and q is the pose rotation matrix information represented by a quaternion.

The ways of using quaternion to express the pose rotation matrix information include triangle and exponential, and the exponential is expressed as formula (10).

q=e ^wt/2 (10)

Among them, w is the angular velocity, t is the time, and q is the pose rotation matrix information represented by the exponential mapping of quaternion.

Exemplarily, the constructed spherical interpolation model is formula (11).

Among them, q ₀ and q ₁ are adjacent posture rotation matrices, and q(t) is the posture rotation matrix information obtained by spherical interpolation of the posture rotation matrix.

It can be understood that, according to the posture rotation matrix information q ₀ and q ₁ and the spherical interpolation model, the posture rotation matrix information after interpolation can be obtained.

Exemplarily, FIG. 9 is a schematic diagram 1 of the embodiment of the present disclosure using a quaternion sensor to measure the pose rotation matrix information. As shown in FIG. 9, the electronic device continuously moves from the previous moment to the current moment. That is, rotate from a solid circle to a dotted circle. FIG. 10 is a second schematic diagram of the embodiment of the disclosure using a quaternion to represent the pose rotation matrix information measured by the pose sensor. As shown in FIG. 10, it is a plan view of the dotted frame extracted from FIG. 9, q ₀ and q ₁ is the adjacent posture rotation matrix, q(t) is the posture rotation matrix information after spherical interpolation is performed on the posture rotation matrix, the angle of rotation is Δθ, t is the time, and 0≤t≤1.

In the embodiment of the present disclosure, in the process of acquiring the second angular velocity information according to the interpolated pose rotation matrix information, the angular velocity information corresponding to the vision sensor can be determined according to the interpolated pose rotation matrix information and the differential model. The differential model is as in formula (1).

It can be understood that the second angular velocity information can be obtained indirectly by interpolating the pose rotation matrix information first, and then differential processing. In this way, it can adapt to the time synchronization between more sensors and has universal applicability.

S204: Determine the delay time information between the visual sensor and the inertial sensor.

In the embodiment of the present disclosure, after the first angular velocity information and the second angular velocity information are aligned, the delay time information between the visual sensor and the inertial sensor can be determined according to the first angular velocity information and the second angular velocity information.

In other embodiments, through the first angular velocity information and the second angular velocity information, not only the delay time information between the vision sensor and the inertial sensor can be solved, but also the rotation parameter and the error parameter between the sensors can be solved.

S205: Perform time synchronization processing on the measurement result of the visual sensor and the measurement result of the inertial sensor according to the delay time information.

The time synchronization processing method provided by the embodiments of the present disclosure can be applied to unmanned driving or mobile robot navigation. During the unmanned driving or mobile robot navigation, the unmanned electronic device or mobile robot electronic device can be implemented through the present disclosure The time synchronization processing method provided in the example realizes precise positioning.

In addition, the embodiment of the present disclosure only uses a vision sensor and an inertial sensor as an example to illustrate the process of the electronic device implementing the time synchronization processing method. The time synchronization processing method provided by the embodiment of the present disclosure is not limited to the vision sensor and the inertial sensor. Between the two sensors, it can also be applied to other sensors, as long as the other sensors can independently obtain angular velocity information and pose rotation matrix information. At the same time, the embodiments of the present disclosure can also be applied to electronic devices including more than three sensors, that is, the above-mentioned time synchronization method is also applicable to electronic devices with more than three sensors.

Based on the implementation of the above-mentioned time synchronization processing method, an embodiment of the present disclosure provides a time synchronization processing device. FIG. 11 is a schematic diagram of the composition structure of a time synchronization processing device provided by an embodiment of the disclosure. As shown in FIG. The synchronization processing device 300 includes a first acquisition module 301, an alignment module 302, and a synchronization module 303, where:

The first acquisition module 301 is configured to obtain angular velocity information collected by two different sensors; the angular velocity information is the angular velocity information when the electronic device rotates; the two different sensors are both set on the electronic device And rigidly connected;

The alignment module 302 is configured to perform alignment processing on the two obtained angular velocity information, and determine the delay time information between the two different sensors;

The synchronization module 303 is configured to perform time synchronization processing on the respective measurement results of the two different sensors according to the delay time information.

The time synchronization processing device of the embodiment of the present disclosure requires the electronic device to be capable of rotating motion, that is, the electronic device is required to rotate around the rotation axis at least to calibrate the delay time information between different sensors on the electronic device without It is necessary to require movement around multiple axes, which reduces the complexity of time synchronization of different sensors and can adapt to the needs of different scenarios; and the embodiments of the present disclosure can also calibrate the delay time based on the rotational movement generated by the multi-axis rotation of the electronic device. Obtain richer rotation information to improve the accuracy of time synchronization; the embodiments of the present disclosure require sensors to independently obtain angular velocity information to determine the delay time information between different sensors, which can be widely used in a variety of sensors. The time synchronization between the devices has universal adaptability; the embodiment of the present disclosure is a method for realizing time synchronization based on software, and there is no need to deploy additional dedicated hardware for time synchronization; the embodiment of the present disclosure performs time synchronization based on the delay time information obtained in real time , Can realize online time synchronization processing; embodiments of the present disclosure determine delay time information through angular velocity information obtained when the electronic device rotates, instead of treating the delay time information as a constant, and improve the accuracy of time synchronization.

In other embodiments, the first obtaining module 301 is configured to obtain the pose rotation matrix information collected by the pose sensor; the angular velocity information when the electronic device rotates is determined by the pose rotation matrix information, where: At least one of the two different sensors is a pose sensor.

In other embodiments, the first obtaining module 301 is configured to obtain angular velocity information collected by a gyro sensor, wherein at least one of the two different sensors is a gyro sensor.

In other embodiments, the alignment module 302 is configured to perform interpolation processing on at least one of the angular velocity information to align the two angular velocity information; determine the two angular velocity information according to the aligned two angular velocity information Delay time information between two different sensors.

In other embodiments, the alignment module 302 is configured to perform at least one of the obtained angular velocity information collected by the two gyroscopic sensors when the two different sensors are both gyroscopic sensors. One performs interpolation processing to align the two angular velocity information.

In other embodiments, the alignment module 302 is configured to, in a case where the two different sensors are both pose sensors, perform a pose rotation matrix for each of the obtained two pose sensors. At least one of the information is subjected to interpolation processing to align the two angular velocity information.

In other embodiments, the alignment module 302 is configured to rotate the acquired pose collected by the pose sensor when the two different sensors are a pose sensor and a gyro sensor. The matrix information undergoes interpolation processing to align the two angular velocity information.

In other embodiments, the time synchronization processing device 300 further includes: a second acquisition module 304, configured to perform alignment processing on the two angular velocity information through the alignment module 302 to determine the difference between the two different sensors The external parameters include rotation parameters between different coordinate axes corresponding to the two different sensors and error parameters between the two different sensors.

In other embodiments, the time synchronization processing device 300 further includes: a storage module 305, configured to store the delay time information;

The synchronization module is configured to perform time synchronization on the respective measurement results of the two different sensors according to the delay time information stored by the storage module 305 when the electronic device is in a non-motion state deal with.

In other embodiments, the alignment module 302 is configured to determine the sub-error equations corresponding to the two different sensors at different times according to the two aligned angular velocity information; The error equations are summed to obtain the final error equation; the minimum value processing is performed on the final error equation to obtain the delay time information.

In other embodiments, the alignment module 302 is configured to perform nonlinear processing on the final error equation to obtain a first minimization equation; the first minimization equation is solved until the preset first In the case of a threshold value, the delay time information in the first minimization equation is acquired.

In other embodiments, the alignment module 302 is configured to perform iterative closest point processing on the final error equation to obtain a second minimization equation; the second minimization equation is solved until the preset second In the case of a threshold value, the delay time information in the second minimization equation is acquired.

In other embodiments, the time synchronization processing apparatus 300 further includes:

The fusion module 307 is used to perform fusion processing on the synchronized measurement results;

The execution module 308 is configured to perform at least one of the following operations according to the measurement result of the fusion processing: positioning processing, ranging processing, target detection of the scene where the electronic device is located, and generating or updating a map.

It should be noted that when the time synchronization processing device provided in the above embodiment performs time synchronization processing, only the division of the above program modules is used as an example for illustration. In actual applications, the above processing can be allocated to different program modules according to needs. Complete, that is, divide the internal structure of the device into different program modules to complete all or part of the processing described above. In addition, the time synchronization processing device provided in the foregoing embodiment and the time synchronization processing method embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, and will not be repeated here.

The embodiment of the present disclosure also provides an electronic device. FIG. 12 is a schematic diagram of the structure of the electronic device provided by the embodiment of the disclosure. As shown in FIG. 12, the electronic device at least includes a processor 21 and a storage device capable of running on the processor. The memory 23 of the computer program; the processor 21 is used to execute the steps in the time synchronization processing method provided in the above embodiment when the computer program is running.

In this embodiment, the electronic device used to execute the steps in the time synchronization processing method provided in the foregoing embodiment may be the same or different from the electronic device provided with two different sensors.

Optionally, the electronic device may further include a communication interface 24, which is used to obtain angular velocity information and pose rotation matrix information. The various components in the electronic device are coupled together through the bus system 25. It can be understood that the bus system 25 is used to implement connection and communication between these components. In addition to the data bus, the bus system 25 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clear description, various buses are marked as the bus system 25 in FIG. 12.

It can be understood that the memory 23 may be a volatile memory or a non-volatile memory, and may also include both volatile and non-volatile memory. Among them, the non-volatile memory can be a read only memory (ROM, Read Only Memory), a programmable read only memory (PROM, Programmable Read-Only Memory), an erasable programmable read only memory (EPROM, Erasable Programmable Read- Only Memory, Electrically Erasable Programmable Read-Only Memory (EEPROM, Electrically Erasable Programmable Read-Only Memory), magnetic random access memory (FRAM, ferromagnetic random access memory), flash memory (Flash Memory), magnetic surface memory , CD-ROM, or CD-ROM (Compact Disc Read-Only Memory); magnetic surface memory can be magnetic disk storage or tape storage. The volatile memory may be random access memory (RAM, Random Access Memory), which is used as an external cache. By way of exemplary but not restrictive description, many forms of RAM are available, such as static random access memory (SRAM, Static Random Access Memory), synchronous static random access memory (SSRAM, Synchronous Static Random Access Memory), and dynamic random access Memory (DRAM, Dynamic Random Access Memory), Synchronous Dynamic Random Access Memory (SDRAM, Synchronous Dynamic Random Access Memory), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM, Double Data Rate Synchronous Dynamic Random Access Memory), enhanced -Type synchronous dynamic random access memory (ESDRAM, Enhanced Synchronous Dynamic Random Access Memory), synchronous connection dynamic random access memory (SLDRAM, SyncLink Dynamic Random Access Memory), direct memory bus random access memory (DRRAM, Direct Rambus Random Access Memory) ). The memory 23 described in the embodiment of the present invention is intended to include, but is not limited to, these and any other suitable types of memory.

The method disclosed in the foregoing embodiment of the present invention may be applied to the processor 21 or implemented by the processor 21. The processor 21 may be an integrated circuit chip with signal processing capability. In the implementation process, the steps of the foregoing method can be completed by an integrated logic circuit of hardware in the processor 21 or instructions in the form of software. The aforementioned processor 21 may be a general-purpose processor, a digital signal processor (DSP, Digital Signal Processor), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor 21 may implement or execute various methods, steps, and logical block diagrams disclosed in the embodiments of the present invention. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of the present invention can be directly embodied as being executed and completed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium, and the storage medium is located in the memory 23. The processor 21 reads the information in the memory 23 and completes the steps of the foregoing method in combination with its hardware.

The features disclosed in the several methods or device embodiments provided in the embodiments of the present disclosure can be combined arbitrarily without causing execution logic errors or paradoxes to obtain new method embodiments or device embodiments.

Based on the foregoing embodiment, the embodiment of the present disclosure provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by the foregoing processor, the steps in the time synchronization processing in the foregoing embodiment are implemented.

In the several embodiments provided in the present disclosure, it should be understood that the disclosed device and method may be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division, and there may be other divisions in actual implementation, such as: multiple units or components can be combined, or It can be integrated into another system, or some features can be ignored or not implemented. In addition, the coupling, or direct coupling, or communication connection between the components shown or discussed may be indirect coupling or communication connection through some interfaces, devices or units, and may be electrical, mechanical or other forms of.

The units described above as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units; Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, the functional units in the embodiments of the present disclosure can be all integrated into one processing unit, or each unit can be individually used as a unit, or two or more units can be integrated into one unit; The unit can be implemented in the form of hardware, or in the form of hardware plus software functional units.

A person of ordinary skill in the art can understand that all or part of the steps in the above method embodiments can be implemented by a program instructing relevant hardware. The foregoing program can be stored in a computer readable storage medium. When the program is executed, it is executed. Including the steps of the foregoing method embodiment; and the foregoing storage medium includes: various media that can store program codes, such as a mobile storage device, ROM, RAM, magnetic disk, or optical disk.

Alternatively, if the aforementioned integrated unit of the present disclosure is implemented in the form of a software function module and sold or used as an independent product, it may also be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present disclosure can be embodied in the form of a software product in essence or a part that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for A computer device (which may be a personal computer, a server, or a network device, etc.) is allowed to execute all or part of the methods described in the various embodiments of the present disclosure. The aforementioned storage media include: removable storage devices, ROM, RAM, magnetic disks or optical disks and other media that can store program codes.

The above are only specific implementations of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present disclosure. It should be covered within the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be subject to the protection scope of the claims.

Claims

A time synchronization processing method, the method includes:

Obtaining angular velocity information collected by two different sensors; the angular velocity information is angular velocity information when the electronic device rotates, and the two different sensors are both arranged on the electronic device and rigidly connected;

Aligning the obtained two angular velocity information, and determining the delay time information between the two different sensors;

Perform time synchronization processing on the respective measurement results of the two different sensors according to the delay time information.
The method according to claim 1, wherein at least one of the two different sensors is a pose sensor; the obtaining the angular velocity information collected by the two different sensors separately comprises:

The pose rotation matrix information collected by the pose sensor is obtained; the angular velocity information when the electronic device rotates is determined by the pose rotation matrix information.
The method according to claim 1 or 2, wherein at least one of the two different sensors is a gyro sensor; the obtaining the angular velocity information collected by the two different sensors separately includes:

Obtain the angular velocity information collected by the gyro sensor.
The method according to claim 1, wherein the aligning the obtained two angular velocity information to determine the delay time information between the two different sensors comprises:

Performing interpolation processing on at least one of the two angular velocity information to align the two angular velocity information;

Determine the delay time information between the two different sensors according to the aligned two angular velocity information.
The method according to claim 4, wherein the performing interpolation processing on at least one of the two angular velocity information comprises:

In the case where the two different sensors are both gyro sensors, interpolation processing is performed on at least one of the obtained angular velocity information collected by the two gyro sensors.
The method according to claim 4, wherein the performing interpolation processing on at least one of the two angular velocity information comprises:

In the case where the two different sensors are both pose sensors, interpolation processing is performed on at least one of the obtained pose rotation matrix information collected by the two pose sensors.
The method according to claim 4, wherein the performing interpolation processing on at least one of the two angular velocity information comprises:

In the case where the two different sensors are a pose sensor and a gyro sensor, interpolation processing is performed on the obtained pose rotation matrix information collected by the pose sensor.
The method according to any one of claims 1-7, wherein, after aligning the obtained two angular velocity information and determining the delay time information between the two different sensors, the method further comprises :

By aligning the two angular velocity information, the external parameters between the two different sensors are determined, and the external parameters include the rotation parameters and the rotation parameters between the different coordinate axes corresponding to the two different sensors. The error parameter between the two different sensors.
The method according to any one of claims 1-8, wherein, after aligning the obtained two angular velocity information and determining the delay time information between the two different sensors, the method further comprises :

Storing the delay time information;

When the electronic device is in a non-moving state, time synchronization processing is performed on the respective measurement results of the two different sensors according to the stored delay time information.
The method according to claim 4, wherein the determining the delay time information between the two different sensors according to the aligned two angular velocity information includes:

Determine the sub-error equations corresponding to the two different sensors at different times according to the two aligned angular velocity information;

Sum the sub-error equations corresponding to the different times to obtain the final error equation;

Perform minimum processing on the final error equation to obtain the delay time information.
The method according to claim 10, wherein said performing minimum processing on said final error equation to obtain said delay time information comprises:

Performing non-linear processing on the final error equation to obtain a first minimization equation;

The first minimization equation is solved until the preset first threshold is satisfied, and the delay time information in the first minimization equation is acquired.
The method according to claim 10, wherein said performing minimum processing on said final error equation to obtain said delay time information comprises:

Performing iterative nearest point processing on the final error equation to obtain a second minimization equation;

The second minimization equation is solved until the preset second threshold is satisfied, and the delay time information in the second minimization equation is acquired.
The method according to any one of claims 1-12, wherein after the time synchronization processing is performed on the respective measurement results of the two different sensors according to the time delay information, the method further comprises:

Perform fusion processing on the synchronized measurement results;

Perform at least one of the following operations according to the measurement result of the fusion processing: positioning processing, ranging processing, target detection of the scene where the electronic device is located, and generating or updating a map.
A time synchronization processing device, the device includes a first acquisition module, an alignment module, and a synchronization module, wherein:

The first acquisition module is configured to obtain angular velocity information collected by two different sensors; the angular velocity information is angular velocity information when the electronic device rotates, and the two different sensors are both set on the electronic device Upper and rigid connection;

The alignment module is configured to perform alignment processing on the two obtained angular velocity information, and determine the delay time information between the two different sensors;

The synchronization module is configured to perform time synchronization processing on the respective measurement results of the two different sensors according to the delay time information.
The apparatus according to claim 14, wherein the first acquisition module is configured to acquire the pose rotation matrix information collected by the pose sensor; and determine when the electronic device rotates according to the pose rotation matrix information Angular velocity information, wherein at least one of the two different sensors is a pose sensor.
The device according to claim 14 or 15, wherein the first acquisition module is configured to acquire angular velocity information collected by a gyro sensor, wherein at least one of the two different sensors is a gyro sensor.
The device according to claim 14, wherein the alignment module is configured to perform interpolation processing on at least one of the two angular velocity information to align the two angular velocity information; according to the aligned two angular velocity information Information to determine the delay time information between the two different sensors.
The device according to claim 17, wherein the alignment module is configured to compare the obtained angular velocity information collected by the two gyroscopic sensors when the two different sensors are both gyroscopic sensors. At least one of them performs interpolation processing to align the two angular velocity information.
The device according to claim 17, wherein the alignment module is configured to, in the case that the two different sensors are both pose sensors, collect data collected by the two obtained pose sensors. At least one of the pose rotation matrix information undergoes interpolation processing to align the two angular velocity information.
The device according to claim 17, wherein the alignment module is configured to collect the obtained pose sensor when the two different sensors are a pose sensor and a gyro sensor. Perform interpolation processing on the posture rotation matrix information to align the two angular velocity information.
The device according to any one of claims 14-20, wherein the device further comprises:

The second acquisition module is configured to align the two angular velocity information through the alignment module, and determine the external parameters between the two different sensors, and the external parameters include each of the two different sensors. Corresponding rotation parameters between different coordinate axes and error parameters between the two different sensors.
The device according to any one of claims 14-21, wherein the device further comprises:

A storage module for storing the delay time information;

The synchronization module is configured to perform time synchronization processing on the respective measurement results of the two different sensors according to the delay time information stored in the storage module when the electronic device is in a non-moving state .
The device according to claim 17, wherein the alignment module is configured to determine the sub-error equations corresponding to the two different sensors at different moments according to the aligned two angular velocity information; The sub-error equations corresponding to the moments are summed to obtain the final error equation; the minimum value processing is performed on the final error equation to obtain the delay time information.
The device according to claim 23, wherein the alignment module is configured to perform nonlinear processing on the final error equation to obtain a first minimization equation; the first minimization equation is solved until it is satisfied When the first threshold is preset, the delay time information in the first minimization equation is acquired.
The device according to claim 23, wherein the alignment module is configured to perform iterative closest point processing on the final error equation to obtain a second minimization equation; the second minimization equation is solved until it is satisfied When the second threshold is preset, the delay time information in the second minimization equation is acquired.
The device according to any one of claims 14-25, wherein the device further comprises:

Fusion module, used for fusion processing of synchronized measurement results;

The execution module is configured to perform at least one of the following operations according to the measurement result of the fusion processing:

Positioning processing, ranging processing, target detection of the scene where the electronic device is located, generating or updating the map.
An electronic device comprising at least a processor and a memory for storing a computer program that can run on the processor; when the processor is used to run the computer program, it executes any one of claims 1 to 13 The method described in the item.
A computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method according to any one of claims 1 to 13 is implemented.