WO2017156642A1 - A system for synchronization of data logged with wearable devices and video streams in sports applications - Google Patents

A system for synchronization of data logged with wearable devices and video streams in sports applications Download PDF

Info

Publication number
WO2017156642A1
WO2017156642A1 PCT/CH2017/000025 CH2017000025W WO2017156642A1 WO 2017156642 A1 WO2017156642 A1 WO 2017156642A1 CH 2017000025 W CH2017000025 W CH 2017000025W WO 2017156642 A1 WO2017156642 A1 WO 2017156642A1
Authority
WO
WIPO (PCT)
Prior art keywords
wearable device
video
wearable
logged
data
Prior art date
Application number
PCT/CH2017/000025
Other languages
French (fr)
Inventor
Christian Züger
Thomas MOIX
Original Assignee
Kitris Ag
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Kitris Ag filed Critical Kitris Ag
Publication of WO2017156642A1 publication Critical patent/WO2017156642A1/en

Links

Classifications

    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H20/00—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance
    • G16H20/30—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance relating to physical therapies or activities, e.g. physiotherapy, acupressure or exercising
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H40/00—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices
    • G16H40/60—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices
    • G16H40/67—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices for remote operation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/414—Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance
    • H04N21/41407—Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/422—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS]
    • H04N21/4223—Cameras
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
    • H04N21/4307—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen
    • H04N21/43074—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen of additional data with content streams on the same device, e.g. of EPG data or interactive icon with a TV program
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/442—Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk
    • H04N21/44213—Monitoring of end-user related data
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/85—Assembly of content; Generation of multimedia applications
    • H04N21/854—Content authoring
    • H04N21/8547—Content authoring involving timestamps for synchronizing content
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/04—Synchronising
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G06V40/23—Recognition of whole body movements, e.g. for sport training

Definitions

  • the present invention applies to the field of wearable devices in sports applications.
  • Wearable devices in sports applications can be used to log a certain number of information during a sports session that can be used to access and analyze performance.
  • this logged data can include, but is not limited to, movements captured by inertial sensors, heart rate or vibration
  • This raw information can be interpreted directly by the wearable device or can be retrieved by an external system (via a wired or wireless communication link) to provide
  • wearable devices are wearable heart rate sensors or activity monitors (based on movements) .
  • the user could in certain applications and if this function is available on the wearable device also enter punctual information about the session with dedicated actions.
  • These input methods could include, but are not limited to, button (s) pressing, activation movements or voice commands.
  • this function are score trackers in racket sports such as, but not limited to, tennis .
  • the present invention focuses on external sources of optical type such as video streams.
  • the video streams can potentially be used for enhanced presentation of the data logged by the wearable device or in a more advanced system used together with the logged data to compute additional information on the recorded session or to replay relevant sequences of recorded video streams autonomously or on demand.
  • One critical issue when using several sources for data logging is to synchronize temporally all sources before combining them.
  • a typical application example is a wearable device used for movement logging and score tracking in tennis
  • the information logged during a session can be retrieved and processed to create higher-level information such as, but not limited to, point and performance statistics, rally lengths or the different strikes performed in rallies.
  • the processed information could in this example be presented to the user via an interactive software or a web-client. If a video stream of the session is available, it could be synchronized with the information from the wearable device and used for an enhanced result display enabling a deeper understanding of the presented results.
  • the user could for instance review strikes in a given rally or part of the session along with the video information. For a more elaborate and specific analysis the user could also for instance visualize all the last strikes of rallies lost by unforced error (detected by the analysis of the data logged by the wearable device) to analyze the root cause and take
  • the object of the present invention is to provide an improved device and method for synchronization of data logged with wearable devices and video streams in sports applications to overcome the mentioned limitations.
  • the wearable device is provided with one or several sensors generating data which can be logged, and in particular wherein the wearable devices has one or more inertial sensors which data can be logged. It is further preferred that the wearable device has one or several user input means which operation by the user can be logged as events. It is further preferred that the optical output of the wearable device is provided by at least one light emitting diode. In another preferred embodiment the optical output of the wearable device is provided by at least one display screen. It is further preferred that the optical synchronization sequence contains encoded communication information that can be extracted during the detection phase on the computing device.
  • the optical synchronization sequence contains encoded identification information such as serial numbers or session identification numbers that can be extracted during the detection phase on the computing device.
  • a further preferred embodiment of the system uses several video capture devices to capture the optical synchronization sequence generated by the wearable device and wherein the resulting video streams are synchronized to the logged data of the wearable device.
  • several video capture devices are used to capture different optical synchronization sequences generated by the wearable device and each video stream is synchronized to the logged data of the wearable device by detecting an optical synchronization sequence that can be different depending on the video capture device that was used to acquire the video stream.
  • the synchronization sequence generated by the wearable device can be initiated by a user input and/or that the synchronization sequence generated by the wearable device can be initiated automatically.
  • a further preferred embodiment of the system includes that the wearable device comprises several optical outputs (light emitting diodes and/or screens) used to generate the optical synchronization sequence.
  • the computing device can be a computer and/or server. It is a further preferred embodiment that the computing device and the wearable device are part of the same physical device. In a preferred system several video capture devices are linked to the video control device and at least one video stream from one of the linked video capture device is used to capture the optical synchronization sequence generated by the wearable device. In a further preferred embodiment the computing device and the video control device are part of the same physical device. As well it is a preferred embodiment wherein the
  • the computing device, the video control device and the wearable device are part of the same physical device.
  • the synchronization communication sequence can be initiated automatically when the wearable device and the video control device have established a communication link with their limited range communication means.
  • the limited range communication means is of near field communication (NFC) type.
  • Fie.l illustrates an example embodiment of the disclosed invention where two independent stand-alone video capture devices are used to capture a sports session of a user with a wearable device on the wrist;
  • Fie.2 illustrates an example embodiment of the disclosed invention where two video capture devices are connected to a video control system used to capture a sports session of a user with a wearable device on the wrist.
  • One of the video capture device has a wireless communication link to the video control system while the second relies on a wired communication link;
  • FIG. 3 illustrates an example embodiment of the disclosed invention where a video capture device is connected to a video control system to a video control system used to capture a sports session of a user with a wearable device on the wrist.
  • a limited range RF device is used to communicate with the wearable device when in proximity.
  • the term “wearable” when in' relation with “device” is to be understood as attached to the user's body by any means such as, but not limited to, a strap, bracelet, adhesive patch or as embedded in any piece of clothing or equipment worn or carried by the user.
  • the wearable device in the disclosed invention comprises at least a logging function for at least a sensor (possibly external to the wearable device ⁇ and/or at least one input means for user information about the session in progress (for instance for entering scoring information) .
  • the logged information can be retrieved or accessed by the means of a communication link (wired or wireless communication link) .
  • the wearable device has at least an optical output that can be used to generate an optical
  • optical output is a light emitting diode (LED) for its compact size, easy operation and low cost. Nevertheless, other embodiments are possible and could consist on other optical output methods such as, but not limited to, displays. In certain embodiments of the disclosed invention, the optical output could be composed of several physical hardware parts (of similar or different types or colors) used in a predefined combination (spatially and/or temporally) to create an optical synchronization signal.
  • LED light emitting diode
  • the optical output could be composed of several physical hardware parts (of similar or different types or colors) used in a predefined combination (spatially and/or temporally) to create an optical synchronization signal.
  • the video capture device in the disclosed invention comprises at least a vision sensor that can provide a video stream.
  • the video stream can either be stored on the device itself and/or can be broadcasted.
  • the data is transmitted by the means of a communication link (wired or wireless communication link) to an external storage means.
  • the stored video stream can be retrieved or accessed by the means of a communication link (wired or wireless communication link) .
  • the wearable device and the video stream ⁇ s) have individual and independent time bases and that timestamps are available for specific events. Such events can for instance be, but are not limited to, a recording session ⁇ either data logging or video stream) start or end.
  • FIG. 1 A first embodiment of the disclosed invention is illustrated in figure 1.
  • the user 101 has a wearable device 102 attached to the wrist.
  • ⁇ video capture device 103 is available to capture a session.
  • a video capture device can in the description of the disclosed invention be of any type such as, but not limited to, camcorder, computer with a webcam, smartphone, smartwatch or tablet.
  • the video capture device can be arranged in a fixed setup or can be a hand-held device manipulated by an external person or left in a given position.
  • the user starts a logging session on the wearable device and starts the recording of the video stream on the video capture device.
  • the wearable device is held in the field of view of the video capture device while the wearable device generates the synchronization signal.
  • the synchronization signal can either be initiated automatically when the logging session starts on the wearable device or can possibly require the user acknowledgement to be started (by an available input means) .
  • the synchronization signal could also in variants of the embodiment be generated at any other point in time during the session (for instance on request by the user or at the end of the session) .
  • At least one timestamp is kept with the logged data on the wearable device to identify when the synchronization signal was generated.
  • the single or multiple timestamps can be taken for instance at, but not limited to, the start or end of the synchronization signal. It is understood that these timestamps are related to the wearable device time base. The order in which the devices are started at the beginning of the session is not important as long as the optical synchronization signal is fully captured on the video stream. Once the synchronization signal has been captured on the video stream, the content of the field of view of the video capture device can be changed at will and possibly not include the wearable device anymore. One could imagine in an example application to record the opponent (s) in a match based sport.
  • the synchronization signal composition can vary in different variations of the embodiment and could be of any composition.
  • a known alternating on and off sequence is generated as a synchronization signal.
  • encoded data for instance, but not limited to, a serial and/or session number
  • a serial and/or session number could be encapsulated in the synchronization sequence using previously disclosed data encoding and framing techniques (such as for instance as techniques used in telecommunication
  • the logged data and video stream can be retrieved by the user or by software tools. They are then processed by software tools on a computing device 105 to generate and format feedback data that can be presented to the user for analysis and review.
  • software tools on a computing device 105 to generate and format feedback data that can be presented to the user for analysis and review.
  • a specific algorithm analyses the video stream ⁇ with prior
  • additional video capture devices can similarly and simultaneously be used in a session as illustrated in figure 1 where a second video capture device 104 is depicted.
  • each video capture device can be synchronized with a single generated
  • synchronization signal is generated) or a separate (and possibly dedicated) synchronization signal can be generated for each video capture device.
  • a separate (and possibly dedicated) synchronization signal can be generated for each video capture device.
  • a mixture of the two approaches is also possible.
  • the optical source generating the optical synchronization signal could be part of a device external to the wearable device.
  • the wearable device would therefore require a proper light sensor (or image sensor) to at least log and/or detect the synchronization signal. It is understood that in such variants, both the video capture device and the wearable device would require to have the synchronization signal in their field of vision while it is generated by the external device. In alternate variants, separate synchronization signals could be used for the wearable device and the video capture devices. Additionally, in all these variants the
  • synchronization signal would also have to be detected in the logged data from the wearable device (if detection is not performed by the wearable device itself either offline or in real time) by a dedicated algorithm during data processing.
  • FIG. 2 A second embodiment of the disclosed invention is illustrated in figure 2.
  • the user 201 has a wearable 202 device attached to his wrist.
  • At least one video capture device 203 is available to capture sports sessions of the user.
  • the video capture device 203 is connected by a communication link 206 to a video control system 205.
  • a communication link 206 to a video control system 205.
  • video capture devices can be used and connected to the video control system.
  • the communication link between a video capture device and the video control system could be of any type (wired or wireless) .
  • two video capture devices, 203 and 204 are connected to a single video control system 205 with a wireless communication link 206 and a wired communication link 207 respectively.
  • the user starts a logging session on the wearable device and starts the capture of at least one video stream on the video control system.
  • the video control system is in charge of controlling the different connected video capture devices through their communication link. The exact nature of the control depends greatly on the video capture device (s) used. It could be, but is not limited to, sending commands to the video capture device (s) (for instance: start the video stream recording, stop the video stream recording, get current timestamp, etc..) or simply storing the broadcasted video stream(s) . It is understood that the video control system has a time base. Since the video control system has control over the video capture device (s), it is understood that it has a knowledge of the video streams and their
  • the wearable device is held in the field of view of at least one video capture device while the wearable device generates the optical
  • the video capture device (s) used to capture the synchronization signal can be predefined (by setup or user) or detected automatically during the video processing step that detects the synchronization signal.
  • the generation of the synchronization signal on the wearable device and the type of synchronization signal used are similar to what was previously described in the first example embodiment of the disclosed invention.
  • the order in which the devices are started is not important as long as the optical synchronization is fully captured by at least one video stream. Once the synchronization signal has been captured on at least one of the video streams, the content of the field of view can be changed at will and possibly not include the wearable device anymore. Furthermore, since the video control system monitors and controls the
  • Video capture devices it is even possible in this embodiment of the disclosed invention to start and resume some of the video streams depending on the session type. Such action could be performed automatically by the video control system (out of a predefined pattern or by on the fly analysis of the session) or by input from the user or another third person on the video control system (could even be remotely performed if the video control system has such capability) .
  • One variant of the disclosed embodiment can therefore use a dedicated video capture device (possibly part of the same physical device as the video control system) to capture the synchronization signal during a given time period while other video capture devices are used to record the actual sport session.
  • the detection of the synchronization signal can be performed similarly as described in the first embodiment. In this case the processing steps are carried out in the computing device (208) .
  • the computing device can physically be part of the video control system, an external computing system or the computation can be carried out partially on both.
  • detection of the synchronization processing step could be carried out in real-time on at least one of the video streams.
  • the video control device or the external computing devise carrying out this task could provide a feedback of any available means to the user or another person when the
  • synchronization signal detection is successful or failed, potentially enabling a new synchronization attempt.
  • the video control system and at least one video capture device could be part of a single physical device.
  • Examples of such devices are, but are not limited to, a computer with an integrated webcam (most laptop computers for instance), a tablet, a smartphone or a smartwatch.
  • the optical source generating the synchronization could be part of an external device (for instance stand-alone or part of the video control device) to the wearable device.
  • the wearable device would therefore require a proper light sensor (or image sensor) to log or detect the synchronization signal. It is understood that in such variants, at least one video capture device and the wearable device would require to have the synchronization signal in their field of vision while it is generated by the external device.
  • separate synchronization signals could be used for the wearable device and at least one the video capture devices. Additionally, in all these variants the synchronization signal would also have to be detected in the logged data from the wearable device (if detection is not performed by the wearable device itself) by a dedicated algorithm during processing.
  • FIG. 3 A third embodiment of the disclosed invention is illustrated in figure 3.
  • the user 301 has a wearable device 302 attached to his wrist.
  • At least one video capture device 303 is available to capture sports sessions of the user.
  • the video capture device 303 is connected by a communication link 306 to a video control system 305.
  • a communication link 306 to a video control system 305.
  • additional video capture devices can be used and connected to the video control system.
  • the communication link between a video capture device and the video control system could be of any type (wired or wireless) .
  • ⁇ limited range wireless communication device 304 is linked to the video control system 305 by a communication link that can be of any type (wired or wireless) .
  • additional limited range can be used and connected to the video control system.
  • the wearable device 302 includes a means to establish a communication link 308 when in range with the limited range communication device 304.
  • the wearable device can but does not necessarily contain an optical output (as the first two described
  • the video control device controls the different video capture devices as previously described in the second embodiment of the invention with the exception of the synchronization process with the wearable device.
  • the user starts a logging session on the wearable device and starts a session on the video control device (but not necessarily a video stream) .
  • the session synchronization is performed by saving time stamps on the wearable device and on the video control device when a specific synchronization communication cycle between the two devices is successfully performed. This implies that the user comes into range of the limited range communication device at least once during the session. Depending on the variant, this moment can be predefined (for instance but not limited to, the start of session) or at any time during the session.
  • the video control device can provide a feedback by any- available means to the user that the synchronization was
  • the wearable device Since, the synchronization is performed by wireless communication, the wearable device does not have to be in the field of view of any video capture device or to appear in any video stream during the synchronization.
  • the logged data and video streams can be retrieved by the user or by software tools. They are then processed by software tools on a computing device 309 to generate and format feedback data that can be presented to the user for analysis and review.
  • the synchronization of the of the different information sources is simply performed by using the timestamps logged during the synchronization communication during the session.
  • a typical implementation of a limited range communication link in a variant of the embodiment could be, but not limited to, near field communication (NFC) .
  • NFC near field communication
  • both devices could potentially generate requests and answers based on previous communication and on the state of each device (although for practical reasons and power efficiency a
  • the communication link could in such variants be used for synchronization as described previously but also to transmit additional information during the session (for instance updated score information during a session that could be used for a live scoring system) .
  • the user could therefore optionally come in communication range at several occasions during the session to update this information.
  • video capture devices the video control device or limited range communication devices could be part of the same physical device.

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Public Health (AREA)
  • Primary Health Care (AREA)
  • Medical Informatics (AREA)
  • Epidemiology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biophysics (AREA)
  • Physical Education & Sports Medicine (AREA)
  • Business, Economics & Management (AREA)
  • General Business, Economics & Management (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Security & Cryptography (AREA)
  • Databases & Information Systems (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Social Psychology (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

In a system including a wearable device (102), a video capture device (103) and a computing device, the wearable device comprises a housing adapted to be worn on and fixed to the body of a person (101), means for logging data, at least one optical output, at least one internal time source that can be used to generate timestamps and at least one communication means to retrieve logged data. The video capture device (103) comprises at least one means for acquiring video streams using at least an internal time source to synchronize frame capture and at least one communication means to retrieve video streams. The computing device is adapted to synchronize the retrieved logged data by the wearable device and the video stream captured by the video capture device by matching the detection by a video processing algorithm of the temporal position of an optical synchronization sequence generated by the wearable device and sent out through its optical output and captured in the video stream and a timestamp in the logged data of the wearable device indicating when the synchronization sequence was generated.

Description

A system for synchronisation of data logged with wearable devices and video streams in sports applications
CROSS REFERENCE TO RELATED APPLICATIONS
This application claims the priority of Swiss Patent Application No. 0355/16 filed on March 15, 2016.
TECHNICAL FIELD
The present invention applies to the field of wearable devices in sports applications.
BACKGROUND ART
Wearable devices in sports applications can be used to log a certain number of information during a sports session that can be used to access and analyze performance. Depending on the wearable device hardware and on the considered sport, this logged data can include, but is not limited to, movements captured by inertial sensors, heart rate or vibration
measurements. This raw information can be interpreted directly by the wearable device or can be retrieved by an external system (via a wired or wireless communication link) to provide
additional information on the session to the user. Typical examples of such wearable devices are wearable heart rate sensors or activity monitors (based on movements) . Additionally, the user could in certain applications and if this function is available on the wearable device also enter punctual information about the session with dedicated actions. These input methods could include, but are not limited to, button (s) pressing, activation movements or voice commands. Typical examples of this function are score trackers in racket sports such as, but not limited to, tennis .
In some sports applications it is of great interest to pair the information gathered by a wearable device with other external and possibly independent data sources in order to expand the analysis capabilities of the user performance in the captured sports sessions. The present invention focuses on external sources of optical type such as video streams. The video streams can potentially be used for enhanced presentation of the data logged by the wearable device or in a more advanced system used together with the logged data to compute additional information on the recorded session or to replay relevant sequences of recorded video streams autonomously or on demand.
One critical issue when using several sources for data logging (such as, in this case, with a wearable device and video streams) is to synchronize temporally all sources before combining them.
A typical application example is a wearable device used for movement logging and score tracking in tennis
applications. The information logged during a session can be retrieved and processed to create higher-level information such as, but not limited to, point and performance statistics, rally lengths or the different strikes performed in rallies. The processed information could in this example be presented to the user via an interactive software or a web-client. If a video stream of the session is available, it could be synchronized with the information from the wearable device and used for an enhanced result display enabling a deeper understanding of the presented results. The user could for instance review strikes in a given rally or part of the session along with the video information. For a more elaborate and specific analysis the user could also for instance visualize all the last strikes of rallies lost by unforced error (detected by the analysis of the data logged by the wearable device) to analyze the root cause and take
corrective actions. In the current state of the art, most wearable devices have at least one RF (radio frequency) wireless link (typically standard technologies such as Wi-Fi or Bluetooth) that could possibly be used to synchronize logged data with video streams based on previously disclosed methods (for instance triggering of the video recording or timestamps exchange) . There are nevertheless cases where a wireless communication link is not available (for instance on some simple video capture devices) or simply not allowed or limited in range in specific conditions (for instance during official matches in tennis to prevent the possible violation of a non-coaching rule) . Another straight forward synchronization alternate in these cases would be to establish a communication link (wired or wireless) between the wearable device and the video capture devices before, after or during a pause of the session. Nevertheless, this method can possibly be complicated and awkward for the user, which could be detrimental to concentration and focus (especially during a competition) .
DISCLOSURE OF THX INVENTION
The object of the present invention is to provide an improved device and method for synchronization of data logged with wearable devices and video streams in sports applications to overcome the mentioned limitations.
In order to meet this object, the invention
introduces a novel system and thus a device and a method, as defined by the independent claims.
By synchronizing a wearable device using an optical synchronization signal or a short range RF communication link (typically near field communication NFC) as claimed, the above disadvantages can be overcome.
In a preferred embodiment of the device and method or system, respectively, the wearable device is provided with one or several sensors generating data which can be logged, and in particular wherein the wearable devices has one or more inertial sensors which data can be logged. It is further preferred that the wearable device has one or several user input means which operation by the user can be logged as events. It is further preferred that the optical output of the wearable device is provided by at least one light emitting diode. In another preferred embodiment the optical output of the wearable device is provided by at least one display screen. It is further preferred that the optical synchronization sequence contains encoded communication information that can be extracted during the detection phase on the computing device. It is in particular preferred that the optical synchronization sequence contains encoded identification information such as serial numbers or session identification numbers that can be extracted during the detection phase on the computing device. A further preferred embodiment of the system uses several video capture devices to capture the optical synchronization sequence generated by the wearable device and wherein the resulting video streams are synchronized to the logged data of the wearable device. In a further preferred embodiment of the system several video capture devices are used to capture different optical synchronization sequences generated by the wearable device and each video stream is synchronized to the logged data of the wearable device by detecting an optical synchronization sequence that can be different depending on the video capture device that was used to acquire the video stream. It is further preferred that the synchronization sequence generated by the wearable device can be initiated by a user input and/or that the synchronization sequence generated by the wearable device can be initiated automatically. A further preferred embodiment of the system includes that the wearable device comprises several optical outputs (light emitting diodes and/or screens) used to generate the optical synchronization sequence. The computing device can be a computer and/or server. It is a further preferred embodiment that the computing device and the wearable device are part of the same physical device. In a preferred system several video capture devices are linked to the video control device and at least one video stream from one of the linked video capture device is used to capture the optical synchronization sequence generated by the wearable device. In a further preferred embodiment the computing device and the video control device are part of the same physical device. As well it is a preferred embodiment wherein the
computing device, the video control device and the wearable device are part of the same physical device. The synchronization communication sequence can be initiated automatically when the wearable device and the video control device have established a communication link with their limited range communication means. In a preferred embodiment the limited range communication means is of near field communication (NFC) type.
BRIEF DESCRIPTION OF DRAWINGS
The above and other features and advantages of the present invention will become more apparent to those of ordinary skill in the art by describing in detail preferred embodiments thereof with reference to the attached drawings in which:
Fie.l illustrates an example embodiment of the disclosed invention where two independent stand-alone video capture devices are used to capture a sports session of a user with a wearable device on the wrist;
Fie.2 illustrates an example embodiment of the disclosed invention where two video capture devices are connected to a video control system used to capture a sports session of a user with a wearable device on the wrist. One of the video capture device has a wireless communication link to the video control system while the second relies on a wired communication link; and
FIG. 3 illustrates an example embodiment of the disclosed invention where a video capture device is connected to a video control system to a video control system used to capture a sports session of a user with a wearable device on the wrist. A limited range RF device is used to communicate with the wearable device when in proximity. DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS
In the disclosed invention./ the term "wearable" when in' relation with "device" is to be understood as attached to the user's body by any means such as, but not limited to, a strap, bracelet, adhesive patch or as embedded in any piece of clothing or equipment worn or carried by the user.
The wearable device in the disclosed invention comprises at least a logging function for at least a sensor (possibly external to the wearable device} and/or at least one input means for user information about the session in progress (for instance for entering scoring information) . The logged information can be retrieved or accessed by the means of a communication link (wired or wireless communication link) .
Additionally, the wearable device has at least an optical output that can be used to generate an optical
synchronization signal. The preferred embodiment of the optical output is a light emitting diode (LED) for its compact size, easy operation and low cost. Nevertheless, other embodiments are possible and could consist on other optical output methods such as, but not limited to, displays. In certain embodiments of the disclosed invention, the optical output could be composed of several physical hardware parts (of similar or different types or colors) used in a predefined combination (spatially and/or temporally) to create an optical synchronization signal.
The video capture device in the disclosed invention comprises at least a vision sensor that can provide a video stream. The video stream can either be stored on the device itself and/or can be broadcasted. In cases where the video stream is broadcasted, the data is transmitted by the means of a communication link (wired or wireless communication link) to an external storage means. It is understood that in any case the stored video stream can be retrieved or accessed by the means of a communication link (wired or wireless communication link) . In the description of the disclosed invention it is understood that the wearable device and the video stream {s) have individual and independent time bases and that timestamps are available for specific events. Such events can for instance be, but are not limited to, a recording session {either data logging or video stream) start or end.
A first embodiment of the disclosed invention is illustrated in figure 1. The user 101 has a wearable device 102 attached to the wrist. Ά video capture device 103 is available to capture a session.
A video capture device can in the description of the disclosed invention be of any type such as, but not limited to, camcorder, computer with a webcam, smartphone, smartwatch or tablet. The video capture device can be arranged in a fixed setup or can be a hand-held device manipulated by an external person or left in a given position.
The user starts a logging session on the wearable device and starts the recording of the video stream on the video capture device. The wearable device is held in the field of view of the video capture device while the wearable device generates the synchronization signal. The synchronization signal can either be initiated automatically when the logging session starts on the wearable device or can possibly require the user acknowledgement to be started (by an available input means) . Note that the synchronization signal could also in variants of the embodiment be generated at any other point in time during the session (for instance on request by the user or at the end of the session) . At least one timestamp is kept with the logged data on the wearable device to identify when the synchronization signal was generated. The single or multiple timestamps can be taken for instance at, but not limited to, the start or end of the synchronization signal. It is understood that these timestamps are related to the wearable device time base. The order in which the devices are started at the beginning of the session is not important as long as the optical synchronization signal is fully captured on the video stream. Once the synchronization signal has been captured on the video stream, the content of the field of view of the video capture device can be changed at will and possibly not include the wearable device anymore. One could imagine in an example application to record the opponent (s) in a match based sport.
The synchronization signal composition can vary in different variations of the embodiment and could be of any composition. In a preferred variant of the embodiment using a light emitting diode as optical output, a known alternating on and off sequence is generated as a synchronization signal.
In variants of the embodiment, encoded data (for instance, but not limited to, a serial and/or session number) could be encapsulated in the synchronization sequence using previously disclosed data encoding and framing techniques (such as for instance as techniques used in telecommunication
protocols) .
Once the session is completed, the logged data and video stream can be retrieved by the user or by software tools. They are then processed by software tools on a computing device 105 to generate and format feedback data that can be presented to the user for analysis and review. During the processing phase, a specific algorithm analyses the video stream {with prior
knowledge of the type of optical synchronization signal used) and determines the position of the synchronization signal in the video stream. This information can be used in further processing steps in conjunction with the timestamps stored when generating the synchronization signal on the wearable device to temporally align information from the logged data and the video stream.
In variants of the disclosed embodiment, additional video capture devices can similarly and simultaneously be used in a session as illustrated in figure 1 where a second video capture device 104 is depicted. In such variants, each video capture device can be synchronized with a single generated
synchronization signal (provided that each video capture device has the wearable device in its field of view when the
synchronization signal is generated) or a separate (and possibly dedicated) synchronization signal can be generated for each video capture device. In other variants a mixture of the two approaches is also possible.
In variants of the embodiment, the optical source generating the optical synchronization signal could be part of a device external to the wearable device. The wearable device would therefore require a proper light sensor (or image sensor) to at least log and/or detect the synchronization signal. It is understood that in such variants, both the video capture device and the wearable device would require to have the synchronization signal in their field of vision while it is generated by the external device. In alternate variants, separate synchronization signals could be used for the wearable device and the video capture devices. Additionally, in all these variants the
synchronization signal would also have to be detected in the logged data from the wearable device (if detection is not performed by the wearable device itself either offline or in real time) by a dedicated algorithm during data processing.
A second embodiment of the disclosed invention is illustrated in figure 2. The user 201 has a wearable 202 device attached to his wrist. At least one video capture device 203 is available to capture sports sessions of the user. The video capture device 203 is connected by a communication link 206 to a video control system 205. In variants of the embodiment,
additional video capture devices can be used and connected to the video control system. The communication link between a video capture device and the video control system could be of any type (wired or wireless) . In the example illustrated in figure 2, two video capture devices, 203 and 204, are connected to a single video control system 205 with a wireless communication link 206 and a wired communication link 207 respectively.
The user starts a logging session on the wearable device and starts the capture of at least one video stream on the video control system. The video control system is in charge of controlling the different connected video capture devices through their communication link. The exact nature of the control depends greatly on the video capture device (s) used. It could be, but is not limited to, sending commands to the video capture device (s) (for instance: start the video stream recording, stop the video stream recording, get current timestamp, etc..) or simply storing the broadcasted video stream(s) . It is understood that the video control system has a time base. Since the video control system has control over the video capture device (s), it is understood that it has a knowledge of the video streams and their
synchronization relative to its own time base. The wearable device is held in the field of view of at least one video capture device while the wearable device generates the optical
synchronization signal. The video capture device (s) used to capture the synchronization signal can be predefined (by setup or user) or detected automatically during the video processing step that detects the synchronization signal. The generation of the synchronization signal on the wearable device and the type of synchronization signal used are similar to what was previously described in the first example embodiment of the disclosed invention. The order in which the devices are started is not important as long as the optical synchronization is fully captured by at least one video stream. Once the synchronization signal has been captured on at least one of the video streams, the content of the field of view can be changed at will and possibly not include the wearable device anymore. Furthermore, since the video control system monitors and controls the
different video capture devices, it is even possible in this embodiment of the disclosed invention to start and resume some of the video streams depending on the session type. Such action could be performed automatically by the video control system (out of a predefined pattern or by on the fly analysis of the session) or by input from the user or another third person on the video control system (could even be remotely performed if the video control system has such capability) . One variant of the disclosed embodiment can therefore use a dedicated video capture device (possibly part of the same physical device as the video control system) to capture the synchronization signal during a given time period while other video capture devices are used to record the actual sport session.
The detection of the synchronization signal can be performed similarly as described in the first embodiment. In this case the processing steps are carried out in the computing device (208) . In variants of the embodiment the computing device can physically be part of the video control system, an external computing system or the computation can be carried out partially on both.
In variants of the disclosed embodiment, the
detection of the synchronization processing step could be carried out in real-time on at least one of the video streams. In such a variant, the video control device or the external computing devise carrying out this task could provide a feedback of any available means to the user or another person when the
synchronization signal detection is successful or failed, potentially enabling a new synchronization attempt.
In variants of the embodiment the video control system and at least one video capture device could be part of a single physical device. Examples of such devices are, but are not limited to, a computer with an integrated webcam (most laptop computers for instance), a tablet, a smartphone or a smartwatch.
In variants of the disclosed embodiment, the optical source generating the synchronization could be part of an external device (for instance stand-alone or part of the video control device) to the wearable device. The wearable device would therefore require a proper light sensor (or image sensor) to log or detect the synchronization signal. It is understood that in such variants, at least one video capture device and the wearable device would require to have the synchronization signal in their field of vision while it is generated by the external device. In alternate variants, separate synchronization signals could be used for the wearable device and at least one the video capture devices. Additionally, in all these variants the synchronization signal would also have to be detected in the logged data from the wearable device (if detection is not performed by the wearable device itself) by a dedicated algorithm during processing.
A third embodiment of the disclosed invention is illustrated in figure 3. The user 301 has a wearable device 302 attached to his wrist. At least one video capture device 303 is available to capture sports sessions of the user. The video capture device 303 is connected by a communication link 306 to a video control system 305. In variants of the embodiment,
additional video capture devices can be used and connected to the video control system. The communication link between a video capture device and the video control system could be of any type (wired or wireless) . Ά limited range wireless communication device 304 is linked to the video control system 305 by a communication link that can be of any type (wired or wireless) . In variants of the embodiment, additional limited range
communication devices could be used and be connected to the video control device. The wearable device 302 includes a means to establish a communication link 308 when in range with the limited range communication device 304. In this embodiment of the invention the wearable device can but does not necessarily contain an optical output (as the first two described
embodiments) .
The video control device controls the different video capture devices as previously described in the second embodiment of the invention with the exception of the synchronization process with the wearable device. The user starts a logging session on the wearable device and starts a session on the video control device (but not necessarily a video stream) . The session synchronization is performed by saving time stamps on the wearable device and on the video control device when a specific synchronization communication cycle between the two devices is successfully performed. This implies that the user comes into range of the limited range communication device at least once during the session. Depending on the variant, this moment can be predefined (for instance but not limited to, the start of session) or at any time during the session. Depending on the variant, the video control device can provide a feedback by any- available means to the user that the synchronization was
performed and/or failed, possibly enabling another
synchronization trial.
Since, the synchronization is performed by wireless communication, the wearable device does not have to be in the field of view of any video capture device or to appear in any video stream during the synchronization.
Once the session is completed, the logged data and video streams can be retrieved by the user or by software tools. They are then processed by software tools on a computing device 309 to generate and format feedback data that can be presented to the user for analysis and review. The synchronization of the of the different information sources is simply performed by using the timestamps logged during the synchronization communication during the session.
A typical implementation of a limited range communication link in a variant of the embodiment could be, but not limited to, near field communication (NFC) . In such a case both devices could potentially generate requests and answers based on previous communication and on the state of each device (although for practical reasons and power efficiency a
master/slave approach is a preferred variant) . The communication link could in such variants be used for synchronization as described previously but also to transmit additional information during the session (for instance updated score information during a session that could be used for a live scoring system) . The user could therefore optionally come in communication range at several occasions during the session to update this information.
In all embodiments, it is understood that, if present, video capture devices, the video control device or limited range communication devices could be part of the same physical device.
It is understood that a mixture of the three
described embodiments would be possible and be part of the disclosed invention. Readers of skill In the art to which this invention pertains will understand that the foregoing description of the details of preferred embodiments is not to be construed in any manner as to limit the invention. Such readers will understand that other embodiments may be made which fall within the scope of the invention, which is defined by the following claims and their legal equivalents.

Claims

1. A system including a wearable device, a video capture device and a computing device, wherein
said wearable device comprises
a housing adapted to be worn on and fixed to the body of a person, and in particular a housing adapted to be worn on and to be fixed to an arm of a person,
a means for logging data,
at least one optical output,
at least one internal time source that can be used to generate timestamps,
at least one communication means to retrieve logged data;
said video capture device comprises
at least one means for acquiring video streams using at least an internal time source to synchronize frame capture,
at least one communication means to retrieve video streams;
and said computing device being adapted to synchronize the retrieved logged data by the wearable device and the video stream captured by the video capture device by matching:
the detection by a video processing algorithm of the temporal position of an optical synchronization sequence generated by the wearable device and sent out through its optical output and captured in the video stream, and
a timestamp in the logged data of the wearable device indicating when the synchronization sequence was generated.
2. The system according to claim 1 wherein the wearable device has one or several sensors which data can be logged, and in particular wherein the wearable devices has one or more inertial sensors which data can be logged.
3. The system according to claim 1 or claim 2 wherein the wearable device has one or several user input means which can be logged as events.
4. The system according to one of claims 1 to 3 wherein the wearable device optical output is at least one light emitting diode (LED) ,
5. The system according to one of claims 1 to 4 wherein the wearable device optical output is at least one display.
6. The system according to one of claims 1 to 5 wherein the optical synchronization sequence contains encoded communication information that can be extracted during the detection phase on the computing device, in particular wherein the optical synchronization sequence contains encoded identification information such as serial numbers or session identification numbers that can be extracted during the detection phase on the computing device.
7. The system according to one of claims 1 to 6 wherein several video capture devices are used to capture the optical synchronization sequence generated by the wearable device and wherein the resulting video streams are synchronized to the logged data of the wearable device.
8. The system according to one of claims 1 to 7 wherein several video capture devices are used to capture different optical synchronization sequences generated by the wearable device and wherein each video stream is
synchronized to the logged data of the wearable device by detecting an optical synchronization sequence that can be different depending on the video capture device that was used to acquire the video stream.
9. The system according to one of claims 1 to 8 wherein the synchronization sequence generated by the wearable device can be initiated by a user input.
10. The system according to one of claims 1 to 9 wherein the synchronization sequence generated by the wearable device can be initiated automatically.
11. The system according to one of claims 1 to 10 wherein the wearable device comprises several optical outputs used to generate the optical synchronization sequence.
12. The system according to one of claims 1 to 11 wherein the computing device is a computer and/or server.
13. The system according to one of claims 1 to 11 wherein the computing device and the wearable device are part of the same physical device.
14. A system including a wearable device, a video control device, a video capture device and a computing device, wherein
said wearable device comprises
a housing adapted to be worn on and fixed to the body of a person, and in particular a housing adapted to be worn on and to be fixed to an arm of a person,
a means for logging data,
at least one optical output,
at least an internal time source that can be used to generate timestamps,
at least a communication means to retrieve logged data;
said video control device comprises
a means to control at least one video capture device through a communication link,
at least an internal time source that can be used to generate timestamps,
a means for logging events such as in particular camera control commands and timestamps; said video capture device comprises
at least one means for acquiring video streams using at least an internal time source to synchronize frame capture,
at least one means to retrieve video streams, at least one communication link to the video control device; and
said computing device being adapted to
synchronize the retrieved logged data by the wearable device and the video stream captured by the video capture device by matching:
the detection by a video processing algorithm of the temporal position of an optical synchronization sequence generated by the wearable device and sent out through its optical output and captured in the video stream,
timestamps logged in the video control devices corresponding to commands to the video capture devices, and
a timestamp in the logged data of the wearable device indicating when the synchronization sequence was generated.
15. The system according to claim 14 wherein the wearable device has one or several sensors which data can be logged, and in particular wherein the wearable devices has one or more inertial sensors which data can be logged.
16. The system according to claiml4 or claim 15 wherein the wearable device has one or several input means which can be logged as events.
17. The system according to one of claims 14 to
16 wherein the wearable device optical output is at least one light emitting diode LED.
18. The system according to one of claims 14 to
17 wherein the wearable device optical output is at least one display.
19. The system according to one of claims 14 to 18 wherein the optical synchronization sequence contains encoded communication information that can be extracted by during the detection phase on the computing device, in particular wherein the optical synchronization sequence contains encoded identification information such serial numbers or session identification numbers that can be extracted by during the detection phase on the computing device .
20. The system according to claims 14 to 19 wherein several video capture devices are linked to the video control device and wherein at least one video stream from one of the linked video capture device is used to capture the optical synchronization sequence generated by the wearable device.
21. The system according to one of claims 14 to
20 wherein the synchronization sequence generated by the wearable device can be initiated by a user input.
22. The system according to one of claims 14 to
21 wherein the synchronization sequence generated by the wearable device can be initiated automatically.
23. The system according to one of claims 14 to
22 wherein the wearable device comprises several optical outputs used to generate the optical synchronization sequence .
24. The system according to one of claims 14 to
23 wherein the computing device is a computer and/or server.
25. The system according to one of claims 14 to 23 wherein the computing device and the wearable device are part of the same physical device.
26. The system according to one of claims 14 to 23 wherein the computing device and the video control device are part of the same physical device.
27. The system according to one of claims 14 to 23 wherein the computing device, the video control device and the wearable device are part of the same physical device .
28. A system including a wearable device, a video control device and a video capture device, wherein
said wearable device comprises
a housing adapted to be worn on and fixed to the body of a person, and in particular a housing adapted to be worn on and to be fixed to an arm of a person,
a means for logging data,
at least one limited range communication means ,
at least one internal time source that can be used to generate timestamps,
at least one communication means to retrieve logged data;
said video control device comprises
a means for controlling at least one video capture device through a communication link,
at least one internal time source that can be used to generate timestamps,
at least one limited range communication means that can create a communication link to the wearable device; and
said video capture device comprises
at least one means for acquiring video streams using at least one internal time source to synchronize frame capture,
at least one means to retrieve video streams, at least one communication link to the video control system; and
wherein the person with the wearable device can establish by said wearable device a communication link with the video control device using the limited range
communication means and wherein a synchronization
communication exchange is performed between the wearable device and the video control device that triggers the logging of timestamps on the wearable device and on the video control device and wherein these timestamps can be used on a computing device at a later stage to synchronize the retrieved logged data from the wearable device and the video streams from the video capture devices.
29. The system according to claim 28 wherein the wearable device has one or several sensors which data can be logged, and in particular wherein the wearable devices has one or more inertial sensors which data can be logged.
30. The system according to claim 28 or claim 29 wherein the wearable device has one or several input means which can be logged as events.
31. The system according to one of claims 28 to
30 wherein the synchronization communication sequence can be initiated by a user input on the wearable device and/or the video control device.
32. The system according to one of claims 28 to
31 wherein the synchronization communication sequence can be initiated automatically when the wearable device and the video control device have established a communication link with their limited range communication means .
33. The system according to one of claims 28 to
32 wherein the limited range communication means is of near field communication (NFC) type.
34. The system according to one of claims 28 to
33 wherein the computing device and the wearable device are part of the same physical device.
35. The system according to one of claims 28 to 33 wherein the computing device and the video control device are part of the same physical device.
36. The system according to one of claims 28 to 33 wherein the computing device, the video control device and the wearable device are part of the same physical device.
PCT/CH2017/000025 2016-03-15 2017-03-08 A system for synchronization of data logged with wearable devices and video streams in sports applications WO2017156642A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CH3552016 2016-03-15
CH0355/16 2016-03-15

Publications (1)

Publication Number Publication Date
WO2017156642A1 true WO2017156642A1 (en) 2017-09-21

Family

ID=58360770

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CH2017/000025 WO2017156642A1 (en) 2016-03-15 2017-03-08 A system for synchronization of data logged with wearable devices and video streams in sports applications

Country Status (1)

Country Link
WO (1) WO2017156642A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3888346A4 (en) * 2019-02-19 2022-08-24 Edgy Bees Ltd. Estimating real-time delay of a video data stream

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150317801A1 (en) * 2010-08-26 2015-11-05 Blast Motion Inc. Event analysis system

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150317801A1 (en) * 2010-08-26 2015-11-05 Blast Motion Inc. Event analysis system

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
MEHRABI MAZIAR ET AL: "Frame Synchronization of Live Video Streams Using Visible Light Communication", 2015 IEEE INTERNATIONAL SYMPOSIUM ON MULTIMEDIA (ISM), IEEE, 14 December 2015 (2015-12-14), pages 128 - 131, XP032886742, DOI: 10.1109/ISM.2015.26 *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3888346A4 (en) * 2019-02-19 2022-08-24 Edgy Bees Ltd. Estimating real-time delay of a video data stream
US11563932B2 (en) 2019-02-19 2023-01-24 Edgy Bees Ltd. Estimating real-time delay of a video data stream
US11849105B2 (en) 2019-02-19 2023-12-19 Edgy Bees Ltd. Estimating real-time delay of a video data stream

Similar Documents

Publication Publication Date Title
US10070046B2 (en) Information processing device, recording medium, and information processing method
JP5867432B2 (en) Information processing apparatus, recording medium, and information processing system
JP5743221B2 (en) Wireless synchronization system, wireless device, sensor device, wireless synchronization method, and program
US20170221379A1 (en) Information terminal, motion evaluating system, motion evaluating method, and recording medium
EP3186599B1 (en) Feedback provision system
US20180063421A1 (en) Wearable camera, wearable camera system, and recording control method
EP3226229A1 (en) Motion evaluation method and system in a sport context
US10942968B2 (en) Frameworks, devices and methodologies configured to enable automated categorisation and/or searching of media data based on user performance attributes derived from performance sensor units
JP2015116288A (en) Exercise information display system, exercise information display method, and exercise information display program
US20150314164A1 (en) Combining signal information from shoes and sports racket
JP7307447B2 (en) MOTION CAPTURE SYSTEM, MOTION CAPTURE PROGRAM AND MOTION CAPTURE METHOD
JP6094476B2 (en) Imaging system, control method therefor, and control program therefor
CN108970091B (en) Badminton action analysis method and system
WO2017156642A1 (en) A system for synchronization of data logged with wearable devices and video streams in sports applications
US20230402066A1 (en) Intelligent sports video and data generation from ai recognition events
US10569135B2 (en) Analysis device, recording medium, and analysis method
JP7276677B2 (en) DATA ACQUISITION DEVICE, CONTROL METHOD AND CONTROL PROGRAM THEREOF, CONTROL DEVICE, DATA ACQUISITION DEVICE
JP6888250B2 (en) Image processing equipment, measuring equipment, image processing system, image processing method and program
CN105718038B (en) A kind of gesture gesture recognition equipment for near-eye display device
US20220001263A1 (en) Smart Wearable Device, a Motion Detector, and a Feedback Mechanism Thereof
JP2016015780A (en) Information processor, recording medium and information processing system
Malawski et al. Synchronization of External Inertial Sensors and Built-in Camera on Mobile Devices
WO2024056153A1 (en) A method of monitoring a device and synchronising sensor data

Legal Events

Date Code Title Description
NENP Non-entry into the national phase

Ref country code: DE

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17711998

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 17711998

Country of ref document: EP

Kind code of ref document: A1