CN108988974B - Time delay measuring method and device and system for time synchronization of electronic equipment - Google Patents
Time delay measuring method and device and system for time synchronization of electronic equipment Download PDFInfo
- Publication number
- CN108988974B CN108988974B CN201810630502.0A CN201810630502A CN108988974B CN 108988974 B CN108988974 B CN 108988974B CN 201810630502 A CN201810630502 A CN 201810630502A CN 108988974 B CN108988974 B CN 108988974B
- Authority
- CN
- China
- Prior art keywords
- image
- frame
- preset
- inertial sensor
- sensor
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04J—MULTIPLEX COMMUNICATION
- H04J3/00—Time-division multiplex systems
- H04J3/02—Details
- H04J3/06—Synchronising arrangements
- H04J3/0635—Clock or time synchronisation in a network
- H04J3/0638—Clock or time synchronisation among nodes; Internode synchronisation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04J—MULTIPLEX COMMUNICATION
- H04J3/00—Time-division multiplex systems
- H04J3/02—Details
- H04J3/06—Synchronising arrangements
- H04J3/0635—Clock or time synchronisation in a network
- H04J3/0638—Clock or time synchronisation among nodes; Internode synchronisation
- H04J3/0658—Clock or time synchronisation among packet nodes
- H04J3/0661—Clock or time synchronisation among packet nodes using timestamps
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Length Measuring Devices By Optical Means (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a time delay measuring method and device and a system for time synchronization of electronic equipment, and relates to the technical field of electronic equipment and intelligent equipment. The time delay measuring method comprises the following steps: acquiring a first image set acquired by a first image sensor based on a preset event; acquiring a second image set acquired by a second image sensor based on the preset event; acquiring sensing data of the inertial sensor based on the preset event; a time delay between the first image sensor and the inertial sensor is generated based on the first image set, the second image set, and the sensed data. The method solves the problem of time synchronization between the image sensor and the inertial sensor in the SLAM system module of the electronic equipment, improves the time synchronization precision, is simple to operate and is easy to realize compared with the prior art.
Description
Technical Field
The present invention relates to the field of electronic device time synchronization technologies, and in particular, to a method and an apparatus for measuring time delay and a system for time synchronization of an electronic device.
Background
With the development of devices for personal or commercial automation (i.e. robotic equipment), such as cleaning robots, the SLAM system, known in english as the singular localization and mapping (chinese name time localization and mapping system), is widely used for simultaneous localization and mapping. Due to the fact that parameters such as module performance, equipment models and the like of the SLAM system on the robot equipment are different, timestamps transmitted by the multiple sensors at the same moment are different, and therefore in order to enable the robot equipment to work normally, the problem of time synchronization of the multiple sensors in the SLAM system needs to be solved.
In the prior art, synchronization is mostly realized by comparing timestamps of different sensors, but timestamps of the sensors cannot be aligned precisely under normal conditions, for example, although timestamps of an image sensor and an inertial sensor are the same, because periods output by different modules are different, data measured at the same time are not in real conditions, millisecond-level delay may occur, and further normal work of robot equipment is affected.
Disclosure of Invention
Objects of the invention
The invention aims to provide a method and a system for solving the time synchronization between an image sensor and an inertial sensor in an SLAM system module by using the image sensor with higher image acquisition frequency as auxiliary equipment, which improve the time synchronization precision, are simple to operate and are easy to realize compared with the prior art.
(II) technical scheme
To solve the above problem, a first aspect of the present invention provides a method for measuring a time delay, including: acquiring a first image set acquired by a first image sensor based on a preset event; acquiring a second image set acquired by a second image sensor based on the preset event; acquiring sensing data of the inertial sensor based on the preset event; a time delay between the first image sensor and the inertial sensor is generated based on the first image set, the second image set, and the sensed data.
According to another aspect of the present invention, there is provided a time delay measuring apparatus comprising: the first image set acquisition module is used for acquiring a first image set acquired by the first image sensor based on a preset event; a second image set acquisition module for acquiring a second image set acquired by a second image sensor based on the preset event; the sensing data acquisition module is used for acquiring sensing data of the inertial sensor based on the preset event; a time delay generation module to generate a time delay between the first image sensor and the inertial sensor based on the first image set, the second image set, and the sensed data.
According to a further aspect of the invention, there is provided a computer readable storage medium having stored thereon a computer program which, when executed by a processor, carries out the steps of any of the methods described above.
According to still another aspect of the present invention, there is provided a system for time synchronizing electronic devices, comprising: a controller and a second image sensor communicatively coupled to the controller; the second image sensor is used for carrying out image acquisition on a preset event to obtain a second image set; the controller is configured to receive the second set of images, sensed data of an inertial sensor in the electronic device based on the preset event, and a first set of images acquired by a first image sensor in the electronic device based on the preset event; the controller comprises a memory and a processor, the memory having stored thereon a computer program which, when executed by the processor, carries out the steps of the method of any one of the preceding claims.
(III) advantageous effects
The technical scheme of the invention has the following beneficial technical effects: compared with the traditional method for performing time synchronization on each sensor in the SLAM system module by adopting a timestamp contrast time delay technology, the invention provides the method for calculating the time delay among the sensors in the SLAM system module by means of the image acquisition sensor with higher image acquisition frequency, so that the time synchronization precision of each sensor in the SLAM system module is higher, and meanwhile, the method provided by the invention is simple to operate and easy to realize.
Drawings
FIG. 1 is a flowchart illustrating the steps of a method for measuring time delay according to an embodiment of the present invention;
FIG. 2 is a schematic diagram illustrating the relationship between modules of the apparatus for measuring time delay according to an embodiment of the present invention;
FIG. 3 is a block diagram of a system for time synchronization of electronic devices according to an embodiment of the present invention;
FIG. 4 is a hardware configuration diagram of one embodiment of the controller of FIG. 3;
FIG. 5 is sensed data collected by an inertial sensor in one example embodiment.
Reference numerals:
11. a first image set acquisition module; 12. a second image set acquisition module;
13. a sensed data acquisition module; 14. a time delay generating module;
1. a controller; 2. a second image sensor; 3. a first image sensor; 4. an inertial sensor; 5. a SLAM system; 6. an object control device.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in further detail with reference to the accompanying drawings in conjunction with the following detailed description. It should be understood that the description is intended to be exemplary only, and is not intended to limit the scope of the present invention. Moreover, in the following description, descriptions of well-known structures and techniques are omitted so as to not unnecessarily obscure the concepts of the present invention.
In the description of the present invention, it should be noted that the terms "first", "second", and "third" are used only for the purpose of distinguishing descriptive objects, and are not intended to indicate or imply relative importance.
Fig. 1 is a flowchart illustrating steps of a method for measuring a time delay according to an embodiment of the present invention.
As shown in fig. 1, in the first embodiment of the present invention, the method for measuring the time delay includes the following steps S1-S4:
s1, a first set of images acquired by the first image sensor based on the predetermined event is acquired.
Wherein the first image set is acquired by the first image sensor at a first image acquisition frequency; the first image set includes time information and image information of a preset event occurrence process.
The preset event includes a first event and a second event. The first event is an event that can be captured by the image sensor and a process that changes over time occurs. The second event is an event that can be captured by the image sensor and sensed by the inertial sensor. Specifically, the preset event is an event that a preset object moves and hits the inertial sensor in the moving process. For example: the pre-set object falls from high to low and hits the inertial sensor located low. The following steps are repeated: the preset object swings from side to side and hits the inertial sensor on the other side. The predetermined object may be a nail, a bolt, a nut, a wood block, a pellet, etc.
And S2, acquiring a second image set acquired by a second image sensor based on the preset event.
Wherein the second image set is acquired by the second image sensor at a second image acquisition frequency; the second image set includes time information and image information of an occurrence process of a preset event.
S3, acquiring sensing data of the inertial sensor based on the preset event;
wherein the sensing data includes time information and acceleration information when the inertial sensor interacts with a preset event.
S4, generating a time delay between the first image sensor and the inertial sensor based on the first image set, the second image set, and the sensed data.
It should be noted that, the steps S1 and S2 are not executed in sequence, and step S1 may be executed first and then step S2 is executed, or step S2 may be executed first and then step S1 is executed, or may be executed simultaneously or alternately.
In a second embodiment of the method for measuring a time delay, based on the first embodiment, the second image capturing frequency is greater than the first image capturing frequency. For example: the first image acquisition frequency is 24 frames/second; the second image acquisition frequency is 36 frames/second. It should be noted that, since the image capturing frequency of the first image sensor is low, it is not possible to clearly and accurately capture the image of the moment when the preset object just hits the inertial sensor, and therefore, it is necessary to capture the image of the moment when the preset object just hits the inertial sensor by using the second image sensor which has a higher image capturing frequency than the first image sensor.
In a third embodiment of the method for measuring a time delay according to the present invention, on the basis of any one of the above embodiments, the step S4 includes the following steps S41 to S43:
s41, obtaining a time stamp of the hypothesis hit frame based on the identification of the first image set and the second image set; wherein the assumed hit frame is a frame of image acquired by the first image sensor, in which the preset object hits the inertial sensor.
And S42, obtaining a timestamp when the inertial sensor senses the preset event based on the sensing data.
S43, performing a difference calculation based on the timestamp of the assumed hit frame and the timestamp of the preset event, generating a time delay between the first image sensor and the inertial sensor.
In a fourth embodiment of the method for measuring a time delay provided by the present invention, on the basis of the third embodiment, the step S41 includes the following steps S411 to S415:
s411, selecting a frame image of the preset object at a preset distance from the inertial sensor from the first image set to obtain a selected frame.
Specifically, image recognition is performed on all image frames in the first image set; and screening out one frame of image of a preset object at a preset distance from the inertial sensor from the first image set as the selected frame based on the image recognition result. And recording the timestamp of the selected frame and the position of the preset object in the selected frame.
S412, selecting a frame of image, where the preset object and the preset object in the selected frame are located at the same position, from the second image set, to obtain a reference frame.
Specifically, image recognition is performed on all image frames in the second image set; and selecting one frame image which is positioned at the same position as the preset object in the selected frame from the second image set as a reference frame based on the image recognition result, and recording the time stamp of the reference frame.
S413, selecting a frame of image of the inertial sensor hit by the preset object from the second image set to obtain a hit frame; the time stamp of the hit frame is recorded.
And S414, obtaining the time-consuming length from the movement of the preset object from the preset distance to the hitting of the inertial sensor based on the time stamp of the hit frame and the time stamp of the reference frame.
And S415, obtaining the time stamp of the supposed hit frame based on the time stamp of the selected frame and the time-consuming length.
In a fifth embodiment of the method for measuring a time delay, based on the fourth embodiment, in step S411, the selected frame is a frame of image of the preset object in the first image set, which is closest to the inertial sensor.
In a sixth embodiment of the method for measuring a time delay, based on the fourth embodiment, in step S411, the selected frame is a frame of image in which the preset object in the first image set is closest to the inertial sensor and the position of the preset object can be identified.
In a seventh embodiment of the method for measuring a time delay according to the present invention, on the basis of the third embodiment, step S42 includes: and if the sensing data meet the preset change condition, taking the timestamp when the sensing data meet the preset change condition as the timestamp when the inertial sensor senses the preset event. Recording a time stamp of a predetermined object hitting the inertial sensor.
In one particular embodiment, the sensed data collected by the inertial sensor is shown in FIG. 5. As can be seen from fig. 5, the sensed data abruptly changes during the time stamp 75934100000 to the time stamp 75936100000. In the present embodiment, the timestamp5 of the impact of the preset object on the inertial sensor satisfies the following formula: 75934100000< timestamp5< 75936100000.
Fig. 2 is a schematic diagram of a module relationship of a time delay measuring apparatus according to an embodiment of the present invention.
As shown in fig. 2, in a first embodiment of the apparatus for measuring a time delay provided by the present invention, the apparatus for measuring a time delay includes: a first image set acquisition module 11, a second image set acquisition module 12, a sensing data acquisition module 13, and a time delay generation module 14.
A first image set obtaining module 11, configured to obtain a first image set acquired by a first image sensor based on a preset event;
a second image set obtaining module 12, configured to obtain a second image set acquired by a second image sensor based on the preset event;
a sensed data acquiring module 13, configured to acquire sensed data of the inertial sensor based on the preset event;
a time delay generation module 14 configured to generate a time delay between the first image sensor and the inertial sensor based on the first image set, the second image set, and the sensing data.
In another embodiment, based on the first embodiment, the time delay generating module includes: the time delay device comprises a first time stamp generating unit, a second time stamp generating unit and a time delay generating unit.
A first timestamp generation unit for deriving a timestamp of a hypothetical hit frame based on the identification of the first set of images and the second set of images; wherein the assumed hit frame is a frame of image acquired by the first image sensor and in which the preset object hits the inertial sensor;
the second timestamp generating unit is used for obtaining a timestamp when the inertial sensor senses the preset event based on the sensing data;
and the time delay generating unit is used for generating the time delay between the first image sensor and the inertial sensor based on the difference calculation between the timestamp of the supposed hit frame and a preset event.
In a second embodiment of the apparatus for measuring a time delay provided by the present invention, based on the first embodiment, the first timestamp generating unit includes: the device comprises a selected frame generating unit, a reference frame generating unit, a hit frame generating unit, a time consumption length generating unit and a hypothesis hit frame timestamp generating unit.
A selected frame generating unit, configured to select a frame of image of the preset object at a predetermined distance from the inertial sensor from the first image set, so as to obtain a selected frame;
a reference frame generating unit, configured to select, from the second image set, one frame of image in which the preset object and a preset object in the selected frame are located at the same position, and obtain a reference frame;
the hit frame generating unit is used for selecting one frame of image of the preset object hitting the inertial sensor from the second image set to obtain a hit frame;
the time-consuming length generating unit is used for obtaining the time-consuming length from the movement of the preset object from the preset distance to the hitting of the inertial sensor based on the time stamp of the hit frame and the time stamp of the reference frame;
and the hypothesis hit frame timestamp generation unit is used for obtaining the timestamp of the hypothesis hit frame based on the timestamp of the selected frame and the time consumption length.
In a third embodiment of the time delay measuring apparatus provided by the present invention, based on the second embodiment, the second timestamp generating unit is configured to, when the sensing data meets a preset change condition, use a timestamp when the sensing data meets the preset change condition as a timestamp when the inertial sensor senses the preset event.
Fig. 3 is a schematic structural diagram of a framework of an embodiment of the system for time synchronization of electronic devices according to the present invention.
As shown in fig. 3, the electronic device includes: a first image sensor 3, an inertial sensor 4 and a SLAM system 5. The electronic device may be a cleaning robot, a service robot, or the like.
Inertial sensors, i.e., IMUs (Inertial measurement unit), are used to measure the three-axis attitude angle (or angular velocity) and acceleration of an object. The IMU will house three-axis gyroscopes and three-direction accelerometers to measure angular velocity and acceleration of the object in three-dimensional space.
SLAM, which is called simultaneous localization and mapping throughout English, is called a temporal positioning and mapping system, or a concurrent mapping and positioning system, in Chinese.
The SLAM system 5 is in communication connection with the first image sensor 3 and the inertial sensor 4 respectively, and performs time synchronization on data acquired by the first image sensor and the inertial sensor based on the time delay.
As shown in fig. 3, the system for time synchronizing the electronic devices includes: a controller 1 and a second image sensor 2 communicatively coupled to the controller.
The second image sensor 2 is used for carrying out image acquisition on a preset event to obtain a second image set;
the controller 1 is configured to receive the second image set, sensing data of an inertial sensor 4 in the electronic device based on the preset event, and a first image set acquired by a first image sensor 3 in the electronic device based on the preset event;
the controller comprises a memory and a processor, the memory having stored thereon a computer program which, when executed by the processor, generates a time delay between the first image sensor and the inertial sensor based on an embodiment of any of the above-described methods of measuring time delay, and sends the time delay to the electronic device.
Preferably, the system for time synchronization of the electronic device further includes an object control device 6 for carrying a preset object, the object control device 6 is in communication with the controller, and the operation of the object control device 6 according to a control signal of the controller enables the preset object to start moving.
Fig. 4 is a hardware configuration diagram of one embodiment of the controller of fig. 3.
As shown in fig. 4, the controller includes: one or more processors and memory, one processor being exemplified in fig. 4. The processor and the memory may be connected by a bus or other means, and fig. 3 illustrates the connection by the bus as an example.
Those skilled in the art will appreciate that the configuration of the electronic device shown in fig. 4 is not intended to limit embodiments of the present invention, and may be a bus or star configuration, and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components.
The processor may be composed of an Integrated Circuit (IC), for example, a single packaged IC, or a plurality of packaged ICs connected with the same or different functions. For example, the processor may include only a Central Processing Unit (CPU), or may be a combination of a CPU, a Digital Signal Processor (DSP), a Graphics Processing Unit (GPU), and various control chips. In the embodiment of the present invention, the CPU may be a single operation core, or may include multiple operation cores.
The memory, which is a non-transitory computer readable storage medium, may be used to store non-transitory software programs, non-transitory computer executable programs, and modules, such as program modules corresponding to the time delay measuring device in the embodiment of the present application (for example, the first image set acquisition module, the second image set acquisition module, the sensing data acquisition module, and the time delay generation module shown in fig. 2). The processor executes various functional applications and data processing of the server by running the non-transitory software programs and modules stored in the memory, namely, the processing method of the embodiment of the time delay measuring method is realized.
The memory may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; in the embodiment of the present invention, the operating system may be an Android system, an iOS system, a Windows operating system, or the like. The storage data area may store data created in accordance with the use of the measurement device for time delay, and the like. Further, the memory may include high speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, the memory optionally includes memory remotely located from the processor. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The invention is further illustrated by a specific application scenario.
The application scene is as follows: the IMU sensor and the first image sensor in the cleaning robot are time synchronized for data transmitted into the SLAM system. The cleaning robot includes: an IMU sensor, a first image sensor, and a SLAM system. The system for synchronizing the time of the cleaning robot provided by the invention comprises: a controller and a second image sensor communicatively coupled to the controller. And the image acquisition frequency of the second image sensor is higher than that of the first image sensor.
Before time synchronization is performed, the following preparatory work needs to be done: the graduated scale is placed at a position where the first image sensor and the second image sensor can shoot in a state of being vertical to a horizontal plane. Placing an IMU sensor below the graduated scale; a nail is placed directly above the IMU sensor and in a graduated position directly in front of the scale so that the path of the nail fall can pass the scale of the scale and hit the IMU sensor.
In the process of the event that the nail falls from the beginning to hit the IMU sensor, the first image sensor and the second image sensor acquire images of the event, respectively obtain a first image set and a second image set, and respectively send the first image set and the second image set to the controller. The IMU sensor sends its sensing data to the controller.
The controller, upon receiving the first image set, the second image set, and the sensed data, executes a computer program to perform the steps of:
and S01, identifying the first image set, finding out that the nail is closest to the IMU sensor and one frame image (namely, a selected frame) of the corresponding scale on the scale can be clearly seen, and recording the position L1 of the scale corresponding to the nail in the frame image and the timestamp1 of the frame image.
S02, identifying the second image set, finding one frame image (namely reference frame) when the nail falls to the position L1, and recording the timestamp2 of the frame image.
And S03, analyzing the sensing data and finding the timestamp5 when the sensing data is mutated.
S04, finding one frame image (namely, a reference frame) when the nail hits the IMU sensor in the second image set, and recording the time stamp3 of the frame image.
S05, based on the timestamps timestamp3, timestamp2 and equation (1), the elapsed length △ T1 of the nail falling from position L1 to the position just hitting the IMU sensor is obtained.
△ T1 ═ timestamp3-timestamp2 formula (1)
S06, based on the elapsed time length △ T1, the timestamp1 and the formula (2), the timestamp4 of the image of the frame (i.e. the assumed hit frame) when the nail hit the IMU sensor, which is shot by the first image sensor, is presumed.
Timestamp4 ═ Timestamp1+ △ T1 formula (2)
S07, based on the timestamps timestamp5, timestamp4 and equation (3), a time delay △ T2 between the first image sensor and the IMU sensor is obtained.
△ T2 ═ timestamp5-timestamp4 type (3)
S08, sending the time delay △ T2 to the SLAM system.
The SLAM system in the cleaning robot, upon receiving the time delay △ T2 sent by the controller, corrects the time data of the transmission data of the IMU sensor and the first image sensor based on the time delay △ T2 so that the two are time-synchronized.
Further, when data collected by a first image sensor and an IMU sensor in the SLAM system are fused, time delay △ T2 is transmitted into a central processing unit, the system automatically and uniformly subtracts time delay △ T2 from a timestamp of the data collected by the IMU sensor, and then time synchronization of the data collected by the first image sensor is achieved, the data collected by the two sensors are better matched and fused, and time synchronization of the image sensor and the IMU sensor in the SLAM system is completed.
It is to be understood that the above-described embodiments of the present invention are merely illustrative of or explaining the principles of the invention and are not to be construed as limiting the invention. Therefore, any modification, equivalent replacement, improvement and the like made without departing from the spirit and scope of the present invention should be included in the protection scope of the present invention. Further, it is intended that the appended claims cover all such variations and modifications as fall within the scope and boundaries of the appended claims or the equivalents of such scope and boundaries.
Claims (17)
1. A method for measuring time delay, comprising:
acquiring a first image set acquired by a first image sensor at a first image acquisition frequency based on a preset event;
acquiring a second image set acquired by a second image sensor at a second image acquisition frequency based on the preset event, wherein the second image acquisition frequency is greater than the first image acquisition frequency;
acquiring sensing data of an inertial sensor based on the preset event;
a time delay between the first image sensor and the inertial sensor is generated based on the first image set, the second image set, and the sensed data.
2. The method of claim 1, wherein,
the preset event is an event that a preset object moves and hits the inertial sensor in the moving process.
3. The method of claim 2, wherein generating a time delay between the first image sensor and the inertial sensor based on the first image set, second image set, and sensed data comprises:
obtaining a timestamp of a hypothetical hit to a frame based on the identification of the first set of images and the second set of images; wherein the assumed hit frame is a frame of image acquired by the first image sensor and in which the preset object hits the inertial sensor;
obtaining a timestamp of the inertial sensor sensing the preset event based on the sensing data;
and performing difference calculation on the time stamp of the hypothesis hit frame and the time stamp of the preset event to generate the time delay between the first image sensor and the inertial sensor.
4. The method of claim 3, wherein deriving timestamps for hypothetical hits on frames based on the identification of the first set of images and the second set of images comprises:
selecting a frame of image of the preset object at a preset distance from the inertial sensor from the first image set to obtain a selected frame;
selecting a frame image of the preset object and the preset object in the selected frame at the same position from the second image set to obtain a reference frame;
selecting a frame of image of the preset object hitting the inertial sensor from the second image set to obtain a hit frame;
obtaining the time-consuming length from the movement of the preset object from the preset distance to the hitting of the inertial sensor based on the time stamp of the hit frame and the time stamp of the reference frame;
based on the timestamp of the selected frame and the elapsed time length, a timestamp of a hypothetical hit frame is derived.
5. The method of claim 4, wherein,
the selected frame is an image of the preset object in the first image set closest to the inertial sensor.
6. The method of claim 4, wherein,
the selected frame is a frame of image in which the preset object is closest to the inertial sensor in the first image set and the position of the preset object can be identified.
7. The method of claim 3, wherein the deriving, based on the sensed data, a timestamp of when the inertial sensor sensed the preset event comprises:
and if the sensing data meet the preset change condition, taking the timestamp when the sensing data meet the preset change condition as the timestamp when the inertial sensor senses the preset event.
8. A time delay measuring device, comprising:
a first image set acquisition module for acquiring a first image set acquired by a first image sensor based on a preset event at a first image acquisition frequency;
a second image set acquisition module for acquiring a second image set acquired by a second image sensor based on the preset event at a second image acquisition frequency, wherein the second image acquisition frequency is greater than the first image acquisition frequency;
the sensing data acquisition module is used for acquiring sensing data of the inertial sensor based on the preset event;
a time delay generation module to generate a time delay between the first image sensor and the inertial sensor based on the first image set, the second image set, and the sensed data.
9. The apparatus of claim 8, wherein,
the preset event is an event that a preset object moves and hits the inertial sensor in the moving process.
10. The apparatus of claim 9, the time delay generating module comprising:
a first timestamp generation unit for deriving a timestamp of a hypothetical hit frame based on the identification of the first set of images and the second set of images; wherein the assumed hit frame is a frame of image acquired by the first image sensor and in which the preset object hits the inertial sensor;
the second timestamp generating unit is used for obtaining a timestamp when the inertial sensor senses the preset event based on the sensing data;
and the time delay generating unit is used for generating the time delay between the first image sensor and the inertial sensor based on the difference calculation between the timestamp of the supposed hit frame and the preset event.
11. The apparatus of claim 10, wherein the first timestamp generation unit comprises:
a selected frame generating unit, configured to select a frame of image of the preset object at a predetermined distance from the inertial sensor from the first image set, so as to obtain a selected frame;
a reference frame generating unit, configured to select, from the second image set, one frame of image in which the preset object and a preset object in the selected frame are located at the same position, and obtain a reference frame;
the hit frame generating unit is used for selecting one frame of image of the preset object hitting the inertial sensor from the second image set to obtain a hit frame;
the time-consuming length generating unit is used for obtaining the time-consuming length from the movement of the preset object from the preset distance to the hitting of the inertial sensor based on the time stamp of the hit frame and the time stamp of the reference frame;
and the hypothesis hit frame timestamp generation unit is used for obtaining the timestamp of the hypothesis hit frame based on the timestamp of the selected frame and the time consumption length.
12. The apparatus of claim 11, wherein,
the selected frame is an image of the preset object in the first image set closest to the inertial sensor.
13. The apparatus of claim 11, wherein,
the selected frame is a frame of image in which the preset object is closest to the inertial sensor in the first image set and the position of the preset object can be identified.
14. The apparatus according to claim 10, wherein the second timestamp generating unit is configured to take a timestamp of when the sensed data satisfies a preset change condition as the timestamp of when the inertial sensor senses the preset event, when the sensed data satisfies the preset change condition.
15. A system for time synchronizing electronic devices, comprising: a controller and a second image sensor communicatively coupled to the controller;
the second image sensor is used for carrying out image acquisition on a preset event to obtain a second image set;
the controller is configured to receive the second set of images, sensed data of an inertial sensor in the electronic device based on the preset event, and a first set of images acquired by a first image sensor in the electronic device based on the preset event; wherein the image acquisition frequency of the second image sensor is greater than the image acquisition frequency of the first image sensor in the electronic device;
the controller comprises a memory and a processor, the memory having stored thereon a computer program which, when executed by the processor, causes the method of any of claims 1 to 7 to generate a time delay between the first image sensor and the inertial sensor and to transmit the time delay to the electronic device.
16. The system of claim 15, wherein,
the preset event is an event that a preset object moves and hits the inertial sensor in the moving process.
17. The system of claim 16, further comprising an object motion control device communicatively coupled to the controller,
the controller is also used for sending a control signal to the object motion control device;
the object motion control device is used for driving the object to move based on the control signal.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810630502.0A CN108988974B (en) | 2018-06-19 | 2018-06-19 | Time delay measuring method and device and system for time synchronization of electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810630502.0A CN108988974B (en) | 2018-06-19 | 2018-06-19 | Time delay measuring method and device and system for time synchronization of electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108988974A CN108988974A (en) | 2018-12-11 |
| CN108988974B true CN108988974B (en) | 2020-04-07 |
Family
ID=64540694
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810630502.0A Active CN108988974B (en) | 2018-06-19 | 2018-06-19 | Time delay measuring method and device and system for time synchronization of electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108988974B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112414400B (en) * | 2019-08-21 | 2022-07-22 | 浙江商汤科技开发有限公司 | Information processing method and device, electronic equipment and storage medium |
| WO2021093586A1 (en) | 2019-11-11 | 2021-05-20 | 北京嘀嘀无限科技发展有限公司 | Data synchronization system, and method |
| CN111813716A (en) * | 2019-11-11 | 2020-10-23 | 北京嘀嘀无限科技发展有限公司 | Multi-sensor data synchronization, electronic device, and storage medium |
| CN112843677B (en) * | 2021-02-08 | 2023-07-11 | 脸萌有限公司 | System delay determining method, device, equipment and storage medium |
| CN113315926B (en) * | 2021-05-26 | 2022-06-14 | 浙江大华技术股份有限公司 | Image output control method and device, storage medium and electronic device |
| CN113436349B (en) * | 2021-06-28 | 2023-05-16 | 展讯通信(天津)有限公司 | 3D background replacement method and device, storage medium and terminal equipment |
| CN115191871B (en) * | 2022-06-07 | 2024-05-28 | 深圳市倍思科技有限公司 | Method and device for data time synchronization, cleaning robot and storage medium |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120293630A1 (en) * | 2011-05-19 | 2012-11-22 | Qualcomm Incorporated | Method and apparatus for multi-camera motion capture enhancement using proximity sensors |
| US9405972B2 (en) * | 2013-09-27 | 2016-08-02 | Qualcomm Incorporated | Exterior hybrid photo mapping |
| JP6371397B2 (en) * | 2015-05-23 | 2018-08-08 | エスゼット ディージェイアイ テクノロジー カンパニー リミテッドＳｚ Ｄｊｉ Ｔｅｃｈｎｏｌｏｇｙ Ｃｏ．，Ｌｔｄ | Method, system, and program for determining state information regarding unmanned aerial vehicles |
| US10026212B2 (en) * | 2015-11-20 | 2018-07-17 | Google Llc | Electronic display stabilization using pixel velocities |
-
2018
- 2018-06-19 CN CN201810630502.0A patent/CN108988974B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN108988974A (en) | 2018-12-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108988974B (en) | Time delay measuring method and device and system for time synchronization of electronic equipment | |
| CN107888828B (en) | Space positioning method and device, electronic device, and storage medium | |
| CN109194436B (en) | Sensor timestamp synchronous testing method, device, equipment, medium and vehicle | |
| US10789771B2 (en) | Method and apparatus for fusing point cloud data | |
| US10799299B2 (en) | Tracking system and tracking method using same | |
| US20200250429A1 (en) | Attitude calibration method and device, and unmanned aerial vehicle | |
| WO2018077176A1 (en) | Wearable device and method for determining user displacement in wearable device | |
| US20130166061A1 (en) | Object gripping apparatus, control method for object gripping apparatus, and storage medium | |
| CN108871311B (en) | Pose determination method and device | |
| KR20160122709A (en) | Methods and systems for determining elstimation of motion of a device | |
| US9485366B2 (en) | Methods and systems for communicating sensor data on a mobile device | |
| US20160061581A1 (en) | Scale estimating method using smart device | |
| WO2020258901A1 (en) | Method and apparatus for processing data of sensor, electronic device, and system | |
| CN106375706A (en) | Method and apparatus for velocity measurement of moving object by use of double pick-up heads, and terminal | |
| US10796437B2 (en) | System and method for simultaneously reconstructing initial 3D trajectory and velocity of object by using single camera images | |
| US20220088783A1 (en) | Method and Apparatus for Manufacturing Line Simulation | |
| CN112819860A (en) | Visual inertial system initialization method and device, medium and electronic equipment | |
| CN103017676A (en) | Three-dimensional scanning device and three-dimensional scanning method | |
| CN108858251A (en) | A kind of collision avoidance system of high-speed motion manipulator | |
| US9292963B2 (en) | Three-dimensional object model determination using a beacon | |
| WO2019080879A1 (en) | Data processing method, computer device, and storage medium | |
| CN115389246B (en) | Speed precision measuring method, system and device of motion capture system | |
| CN105651201B (en) | Three-dimensional scanner and 3-D scanning method | |
| US20230066821A1 (en) | Information processing apparatus, information processing method, and program | |
| CN115309630A (en) | Method, device and equipment for generating automatic driving simulation data and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |