TWI537747B - Motion initiated time synchronization - Google Patents

Description

Time synchronization technique Field of invention

Embodiments of the present invention generally relate to implementing a time synchronization procedure. In particular, embodiments of the present invention are time synchronization procedures for action triggering.

Background of the invention

Images from multiple digital cameras can be compiled together (eg, via Super Resolution) to produce a higher quality picture, capture a video image pair across two separate cameras, or generate a picture from multiple cameras simultaneously 3D (three-dimensional) model. Reconstructing a given scene, especially if the captured scene includes moving objects, the time synchronization program can be used to ensure that image data from the same time instance is taken from each camera. However, in practice, different cameras typically do not share the same time reference point, and each camera has an independent and spontaneous oscillator. Therefore, time stamps from different cameras may be useless when combining pictures without a network time reference point.

According to an embodiment of the present invention, a system is specifically provided, comprising: a sensor for detecting movement of a first mobile device; and implementing a related one in response to the movement of the first mobile device a synchronization module of a time synchronization program of the second mobile device, wherein the time synchronization program is implemented without the network infrastructure support of the first mobile device; and for capturing an image and synchronizing according to the time The program applies a time stamp to an image capture module of the image.

Simple illustration

Various advantages of the embodiments of the present invention will become apparent to those skilled in the art from the following description and the appended claims. A block diagram of an example of a time synchronization environment in accordance with an embodiment; FIG. 2 is a diagram illustrating a time synchronization procedure in accordance with one embodiment; and FIG. 3 is an implementation of a time synchronization procedure in accordance with an embodiment A flowchart of an example of a method; FIG. 4 is a block diagram of an example of a camera according to an embodiment; and FIG. 5 is a diagram of an example of image reconstruction according to an embodiment.

Detailed description of the preferred embodiment

The embodiment of the present invention may provide a device, including a sensor for detecting movement of a first mobile device, and a synchronization module for implementing a second in response to the movement of the first mobile device One of the mobile devices is a time synchronization program. The time synchronization procedure can be implemented with the support of the first mobile device without a network infrastructure.

The embodiment of the invention also includes a system having a sensor for detecting movement of a first mobile device, and a synchronization module for implementing a time synchronization procedure in response to movement of the first mobile device . The time synchronization procedure can be implemented with the support of the first mobile device without a network infrastructure. The system can also include an image capture module for capturing an image and applying a time stamp to the image in accordance with the time synchronization program.

Other embodiments may include a computer readable storage medium having a set of instructions that, when executed by a processor, cause a first mobile device to detect movement of the first mobile device and to respond to the The movement of the first mobile device implements a time synchronization procedure for one of the second mobile devices. The time synchronization procedure can be implemented with the support of the first mobile device without a network infrastructure.

1 shows an environment 10 in which a plurality of mobile devices 12 (12a-12c) implement a peer-to-peer time synchronization procedure 14 (eg, using a point-to-point wireless interface) without taking a network (eg, access points, Base station, cellular network infrastructure 16 or otherwise implement a common time reference point. The mobile device 12 may include or be part of a camera, a mobile internet device (MID), a personal digital assistant (PDA), a wireless smart phone, a media player, a notebook, an e-book reader, Or any combination thereof, wherein the knowledge of the offset between the internal clocks 18, 20, 22 of the individual devices 12 is beneficial. For example, in the case of an image (still or video) capture application, the clock offsets can be used to reconstruct the time dependent scene captured by the mobile device 12.

In the illustrated example, each mobile device 12 includes a sensor 24, 26, 28 to detect the movement of the mobile device 12 in three dimensions. For example, the detector 26 of the mobile device 12b can include an accelerometer, wherein the detector 24 of the mobile device 12a can include a gyroscope or other motion sensing member. The movement of the detection can be triggered in association with a user of the device 12 shaking the device, tapping two or more of the devices 12, and the like to trigger a time synchronization procedure 14. Each mobile device 12 can include a synchronization module 30 for implementing a time synchronization procedure 14 for one or more other devices in response to detecting the movement. As described above, the time synchronization program 14 can be implemented with the support of the mobile device 12 without a network infrastructure. The clock offsets caused by the time synchronization program 14 can be stored locally in each mobile device 12 as offset data 32, 34, 36.

2, which shows a timing diagram 38 in which a first mobile device ("device 1") is operative to implement a second mobile device ("device 2") in response to movement of the first mobile device. One of the time synchronization programs. In the illustrated example, the first mobile device broadcasts the second mobile device to receive one of the time synchronization requests 40. The time synchronization requirement 40 can include a departure time ("ToD") stamp, as well as several other parameters. Table I below provides an example of one of the time frame formats for the time synchronization request 40.

The frame type can identify whether the frame is a synchronization request, respond to one of the synchronization requirements, and the like, and in the illustrated example, the transmitter address can indicate the address of the first mobile device. The address may be a world-unique MAC (Media Access Control) address, wherein the first mobile device may wait for a random or pseudo-random delay timeout to reduce the time from multiple devices before the synchronization request 40 is sent. The risk of collisions between requirements. Additionally, the receiver address can be a broadcast address that is monitored by other devices in the area. If there are multiple sending devices in the area, the dialog can be used to distinguish the time synchronization request frames from different devices. The ToD stamp may be applied when the frame is sent (eg, after the random delay period expires), and the user affinity name may be an identifier of a user-defined first mobile device (eg, Michael's camera) / string. The user affinity name makes it easier for the user to determine whether to receive time synchronization requests from other devices.

Upon receiving the time synchronization request 40, the second mobile device can determine the time of arrival (ToA) of the time synchronization request 40 and identify the time synchronization request based on the transmitter address and/or the dialog box in the request frame. The source of 40. If the second mobile device has also detected a local movement, such as a pan and/or impact, a response 40 to the time synchronization request 40 may be generated and sent after a random delay period has expired. As described above, the random delay period can be imported such that the device receiving the same time synchronization request 40 does not need to respond at the same time, causing a collision. This random delay can therefore be chosen according to a uniform dispersion (eg [0, max_delay]).

The time synchronization response 42 can be a single frame that identifies the ToA of the request 40 and the ToD of the time synchronization response 42. An example of the one-frame format of the time synchronization response 42 is provided in Table II below.

To further filter out an unexpected time synchronization response 42 transmission, a signature of the movement detected by the first mobile device can be encoded into the time synchronization request 40, wherein the second mobile device can compare the received mobile signature with A signature generated for a portion of the movement detected by the second mobile device. The transmission of the synchronization response 42 may be limited to instances where the two signatures match within a defined deterministic level (e.g., satisfying a predetermined relationship).

When the first mobile device shown receives the synchronization response 42, it can determine the ToA of the response 42 and calculate a clock offset of one of the two devices. This calculation can be implemented as follows,

For example, if the local clock of the first mobile device reads 1:30 pm and the offset between the local clock of the first mobile device and the clock of the second mobile device is +2 minutes, the first mobile device can The clock of the second mobile device is determined to be 1:32 pm. By using this type of method, a device can synchronize with multiple devices at the same time without using one of the master clocks in a network. Moreover, because the time synchronization program is decentralized, the program can be faster and more efficient than each device in order to replicate the clock values of other devices. Moreover, because modern clocks can have fairly high accuracy, the two devices do not need to be resynchronized frequently. For example, a drift between two devices with a clock accuracy of 10 ppm is at most 1 millisecond every 50 seconds. Therefore, to maintain a one millisecond accuracy, the two devices can be resynchronized every 50 seconds.

Figure 3 shows a method 44 of synchronizing a local mobile device with one or more other mobile devices. The method 44 can be executed by executing software such as random access memory (RAM), read only memory (ROM), programmable ROM (PROM), firmware, cache memory, etc. A set of logic/module instructions in a machine or computer readable medium of memory, such as an application specific integrated circuit (ASIC), complementary metal oxide semiconductor (CMOS), or transistor-transistor logic (TTL) The fixed functional hardware of the circuit technology of the technology, or any combination thereof, is implemented. For example, a computer code that can perform the operations shown in method 44 can be written in any combination of one or more programming languages, including an object-oriented programming language such as Java, Smalltalk, C++, and the like. A programming language, such as a C programming language or a similar programming language.

Processing block 46 may provide for receiving user input via a user interface of the device. In block 48, the user input input through a small and/or simple touch screen, button, microphone, etc. can be used to place the mobile device in a motion determination mode. Therefore, the mobile device can determine when the user has enabled time synchronization without continuously monitoring the movement of the mobile device. If motion is detected in block 50 (e.g., shaking, striking, tapping, etc. in a critical range), then block 52 may determine whether a time synchronization request has been received from another mobile device. As described above, the time synchronization request frame may include an address of another mobile device (e.g., output via a user interface of the local mobile device), a broadcast address, a dialog, a ToD stamp, and another One of the mobile devices, the user's affinity name, a mobile signature corresponding to the movement of another mobile device, and the like.

If a time synchronization request has been received, the drawing block 54 can identify the source of the request (eg, another mobile device) based on information such as the sender address and/or the request message frame. . Block 54 also confirms the match between the mobile signatures of the two devices. In addition, the user's affinity name of another mobile device and the option to accept or reject the synchronization request (eg, "Accept synchronization request from Michael's camera?") may be presented to the user of the local mobile device.

Block 56 may provide a random delay period prior to transmission of a response in block 58. As described above, the response frame may include an address of another mobile device such as (via a user interface of another mobile device), an address of the local mobile device, the dialog, a ToA stamp, and a A ToD stamp, a user affinity name for one of the local mobile devices, and a mobile signature corresponding to the movement of the local mobile device. Thus, the response to the transmission can cause another mobile device to calculate the clock offset between the two mobile devices and store the offset data in its internal memory.

The method 44 also provides a determination of the clock offset of the local mobile device. Thus, if a time synchronization request is not received in block 52, block 59 provides a time delay for waiting for a random delay period in which a time synchronization request can be sent. As described above, the time synchronization request frame may include an address of the local mobile device, such as (via a user interface of another mobile device), a broadcast address, a dialog, a ToD stamp, and the local Information about the user's affinity name, a mobile signature corresponding to the movement of the local mobile device, and the like.

If a response to the request is detected before a timeout occurs in block 62, the drawing block 64 may confirm that the mobile signatures of the two devices match, and determine the clock offset based on the response. As described above, the determination of the clock offset can include calculation using equation (1) above. The method 44 can periodically perform repeated re-synchronization of the devices for a plurality of mobile devices.

If it is determined in block 62 that a response has not been received after a timeout occurs, block 61 may make a determination as to whether a retry limit has been reached. If so, the mapping method 44 returns to the determination of the movement of block 50. Otherwise, as described above, after the device waits for a random time period, a time synchronization request in block 60 may be resent.

Figure 4 shows an example of a mobile device, wherein a camera 66 includes an image capture module 67, and a processor 68, which may include one or more processor cores (not shown), each core being fully functional It has an instruction fetch unit, an instruction decoder, a first-order (L1) cache, an execution unit, and the like, and can execute an operating system (OS), a device driver, and various other software applications. The processor 68 can also execute time synchronization logic that can be used by the camera 66 to perform a local clock 76 and one or more other mobile devices in response to one of the cameras 66 moving (eg, shaking, bumping, tapping) Time synchronization of the clock (for example, the camera).

In one example, a user of the camera 66 can place the camera in a motion determination mode via a relatively simple user interface (UI) 74, wherein the motion determination mode can be such as an accelerator or a gyroscope One of the motion sensors 78 detects the movement in three dimensions and generates one of the movement signatures. A wireless transceiver 80 can be used to transmit and receive timestamp synchronization requests/responses to and from other mobile devices, wherein the request/response can support the determination of the clock offset. The clock offsets may be stored locally in random access memory (RAM) 70, programmable ROM (PROM) 72, firmware, cache memory, etc. of camera 66. The time synchronization logic can also be retrieved from the RAM 70, PROM 72, etc. of the camera 66 for execution by the processor 68, with a fixed functionality of the processor 68 or other microprocessor, or any combination thereof. To implement it.

Figure 5 shows a scene 82 with content that changes over time. In the illustrated example, the first image (still or video) 84 of the scene 82 is captured by a first camera, and the second image 86 of the scene 82 is captured by a second camera. If the cameras include time synchronization logic, such as the time synchronization logic of camera 66 (Fig. 4), time stamps (TS) 88, 90 that account for the clock offset between the two cameras can be applied to the images 84, 86, such that the time dependent scene can be reconstructed. For example, the time stamp 88 of the first image 84 can also identify the camera associated with the second image 86 and its clock offset with respect to the camera associated with the first image 84. A similar method can be used for the timestamp 90 of the second image 86. The time synchronization logic can also be used to generate super-resolution images, 3-D models, and the like.

In fact, once the two cameras are time synchronized in response to the impact (or similar movement), the first camera can send a wireless message to the second camera to indicate that the second camera is instantaneous in the future. Take a photo at T. The second camera can confirm that the program responds by taking a photo of one of the wireless messages at time instant T. According to the time instant T, the two cameras can exchange images wirelessly so that each camera can establish a 3D structure.

Embodiments described herein can be applied using all types of semiconductor integrated circuit ("IC") wafers. Examples of such IC chips include, but are not limited to, processors, controllers, chipset components, programmable logic arrays (PLAs), memory chips, network chips, and the like. In addition, in some of these figures, the signal conductors may be represented by line segments. Some line segments may be thicker to indicate a more composed signal path and have a numerical designation to indicate a number of constituent signal paths, and/or have arrows at one or more endpoints to indicate the primary information flow direction. However, this should not be viewed in a limited way. Rather, this class of addition details can be used in conjunction with one or more exemplary embodiments to facilitate a better understanding of a circuit. Any representative signal line, whether or not with additional information, may actually include one or more signals that travel in multiple directions and may be executed in any suitable signal type scheme, for example, may be in different pairs, fiber optic lines, and/or A digital or analog signal line that is executed by a single-ended line.

Although the present embodiment has given exemplary dimensions/models/values/ranges, embodiments of the invention are not limited to this same specification. When manufacturing techniques (eg, photolithography) are matured for some time, it is expected that devices of smaller size will be fabricated. In addition, well known power/ground connections to IC chips and other components may or may not be shown in order to simplify the description and discussion, and to avoid obscuring certain aspects of the embodiments of the present invention. Furthermore, the arrangement of the lines may be shown in block diagram form to avoid obscuring embodiments of the present invention, and in addition, in view of the fact that the performance of the execution of such block diagram arrangements is highly dependent on the platform in which the embodiment is to be executed, That is, such characteristics should be within the scope of the industry. Specific details (e.g., circuits) may be presented to illustrate the exemplary embodiments of the present invention, and it will be apparent to those skilled in the art that the embodiments of the present invention are not, or have a In the case of the state, it can still be implemented. The description is thus to be considered as illustrative and not limiting.

The term "coupled" as used herein may be used to refer to any type of relationship between members, directly or indirectly, of the discussion, and may be applied to electrical, mechanical, fluid, optical, electromagnetic, electrical, or other connections. Moreover, the terms "first", "second", and the like are used herein for convenience of discussion only, and are not particularly limited in terms of time or timing, unless otherwise indicated elsewhere herein.

From the above description, it will be apparent to those skilled in the art that the broad teachings of the embodiments of the present invention can be implemented in various different forms. Therefore, in the case where the embodiments of the present invention have been described in connection with the specific examples thereof, the true scope of the embodiments of the present invention should not be limited thereto, as it is apparent to those skilled in the art that Other modifications may be made by the drawings, the description, and the scope of the following patent application.

10. . . surroundings

12, 12a-12c. . . Mobile device

14. . . Point-to-point time synchronization program

16. . . network

18, 20, 22. . . Internal clock

24, 26, 28. . . Sensor

30. . . Synchronization module

32, 34, 36. . . Offset data

38. . . Timing chart

40. . . Time synchronization requirement

42. . . Time synchronization response

44. . . method

46, 48, 50, 52, 54, 56, 58, 59, 60, 61, 62, 64. . . Square

66. . . camera

67. . . Image capture module

68. . . processor

70. . . Random access memory

72. . . Planable ROM

74. . . user interface

76. . . Local clock

78. . . Motion sensor

80. . . Wireless transceiver

82. . . Scenes

84. . . First image

86. . . Second image

88, 90. . . Timestamp

T. . . Time instant

ToA. . . Time of arrival

ToD. . . departure time

1 is a block diagram of an example of a time synchronization environment in accordance with an embodiment;

2 is a diagram illustrating a time synchronization procedure in accordance with an embodiment;

3 is a flow chart of an example of a method of implementing a time synchronization program, in accordance with an embodiment;

Figure 4 is a block diagram of an example of a camera in accordance with an embodiment;

Figure 5 is a diagram of an example of image reconstruction in accordance with an embodiment.

46, 48, 50, 52, 54, 56, 58, 59, 60, 61, 62, 64. . . Square

Claims (20)

A system for synchronizing a plurality of devices, comprising: a sensor for detecting movement of one of the first mobile devices; and a synchronization module for implementing the movement in response to the movement of the first mobile device a time synchronization procedure of a second mobile device, wherein the time synchronization program is implemented without the network infrastructure support of the first mobile device, and wherein the mobile device is to be detected during a delay period And one or more of a time synchronization request and a response to the request; and an image capture module for capturing an image and applying a time stamp to the image according to the time synchronization program . The system of claim 1, wherein the synchronization module performs the steps of: transmitting the time synchronization request after the delay period expires; receiving the response to the time synchronization request from the second mobile device; The response determines a clock offset between the first mobile device and the second mobile device. The system of claim 2, wherein the time synchronization requirement at least partially includes a mobile signature corresponding to the movement of the first mobile device. The system of claim 1, wherein the synchronization module performs the following steps: receiving the time synchronization request from the second mobile device; The response to the time synchronization request is sent after the delay period has expired. The system of claim 4, wherein the response to the time synchronization request includes, at least in part, a first mobile signature corresponding to the movement of the first mobile device, the time synchronization request including A second mobile signature moved by one of the second mobile devices, and the synchronization program transmits the response when the first mobile signature matches the second mobile signature. The system of claim 1, further comprising a user interface for receiving a user input, the synchronization module placing the first mobile device in a motion determination mode in response to the user input in. An apparatus for synchronizing a plurality of devices, comprising: a synchronization module, configured to implement a time synchronization procedure for a second mobile device in response to the detected movement of one of the first mobile devices, wherein the time The synchronization procedure is implemented without the network infrastructure support of the first mobile device, and wherein when the mobile is detected, a time synchronization request is sent and a response to the request is sent after a delay period is expected One or more of them. The device of claim 7, wherein the synchronization module performs the steps of: transmitting the time synchronization request after a random delay period expires; receiving the response to the time synchronization request from the second mobile device; A clock offset between the first mobile device and the second mobile device is determined based on the response. The apparatus of claim 8, wherein the time synchronization requirement at least partially includes a mobile signature corresponding to the movement of the first mobile device. The apparatus of claim 7, wherein the synchronization module performs the steps of: receiving the time synchronization request from the second mobile device; and transmitting the response to the time synchronization request after the delay period is overdue. The apparatus of claim 10, wherein the response to the time synchronization request includes, at least in part, a first movement signature corresponding to the movement of the first mobile device. The device of claim 11, wherein the time synchronization request includes a second mobile signature corresponding to movement of one of the second mobile devices, and the synchronization module is associated with the first mobile signature The response is sent when the second mobile signature matches. The device of claim 7, further comprising a user interface for receiving a user input, the synchronization module placing the first mobile device in a motion determination mode in response to the user input in. A non-transitory computer readable storage medium comprising a set of instructions that, when executed by a processor, cause a first mobile device to perform the following steps: Detecting movement of one of the first mobile devices; and implementing a time synchronization procedure for a second mobile device in response to the movement of the first mobile device, wherein the time synchronization program is unnetworked at the first mobile device Implemented in the case of infrastructure support, and wherein when the movement is detected, one or more of the time synchronization request and one of the responses to the request will be sent after a delay period is expected. The computer readable storage medium of claim 14, wherein the instructions, when executed, cause the first mobile device to perform the step of: transmitting the time synchronization request after the delay period expires; from the second action The device receives the response to the time synchronization request; and determines a clock offset between the first mobile device and the second mobile device based on the response. The computer readable storage medium of claim 15, wherein the time synchronization requirement at least partially includes a mobile signature corresponding to the movement of the first mobile device. The computer readable storage medium of claim 14, wherein the instructions, when executed, cause the first mobile device to perform the steps of: receiving the time synchronization request from the second mobile device; and during the delay period The response to the time synchronization request is sent after the deadline. The computer readable storage medium of claim 17, wherein the response to the time synchronization request at least partially includes corresponding to the first A first mobile signature of the mobile device's movement. The computer readable storage medium of claim 18, wherein the time synchronization request includes a second mobile signature corresponding to movement of one of the second mobile devices, and the instructions cause the The first mobile device transmits the response when the first mobile signature matches the second mobile signature. The computer readable storage medium of claim 14, wherein the instructions are executed to place the first mobile device in a motion determination mode in response to a user input.