GB2595879A

GB2595879A - Method for controlling an image capture device

Info

Publication number: GB2595879A
Application number: GB2008728.4A
Authority: GB
Inventors: El Kolli Yacine; Guignard Romain; Halna Du Fretay Tristan; Tocze Lionel
Original assignee: Canon Inc
Current assignee: Canon Inc
Priority date: 2020-06-09
Filing date: 2020-06-09
Publication date: 2021-12-15
Anticipated expiration: 2040-06-09
Also published as: GB2595879B; GB202008728D0

Abstract

An image capture device 110 is controlled by a control device 140 comprising a first synchronization unit and a second synchronization unit and is connected to a remote time server 130 through a communication network 120. The method of controlling the image capture device 110 comprises: synchronizing the first synchronization unit with the remote time server 130 according to a time synchronization process; synchronizing the second synchronization unit with the first synchronization unit; obtaining first time capture control information at the second synchronization unit; detecting a failure of the communication network 120; determining second time capture control information at the second synchronization unit, and ; controlling the image capture device 100 to capture a sequence of images according the determined second time capture control information. In some embodiments the image capture device is part of a system comprising plural image capture devices and control devices. The first and second capture time information may be capture start times or information relevant to capture start times. The synchronization signals may be generated according to the SMPTE 2059 standard.

Description

METHOD FOR CONTROLLING AN IMAGE CAPTURE DEVICE FIELD OF THE INVENTION

The present invention relates to a method, a device, and a computer program for controlling an image capture device among a plurality of image capture devices connected to a same network, allowing synchronous image capture by all the image capture devices.

BACKGROUND OF THE INVENTION

Video systems allowing a user to interactively control a viewpoint and generate new views of a dynamic scene from any 3D position have received increasing to attention during last years. Such systems involve capturing a scene from a plurality of viewpoints, using a plurality of image capturing apparatuses (e.g. cameras) placed at different positions. A virtual viewpoint content may then be generated from the images captured from the plurality of viewpoints, offering the user a better sense of realism.

The plurality of image capturing apparatuses is generally set up all around the scene (e.g. a sports field), and the distance between two image capturing apparatuses can be very large. Because of that, it is not possible to connect all the image capturing apparatuses to a central image processing device by a conventional image capture interconnect technology, such as Serial Digital Interface (SDI).

Therefore, it is necessary to use digital networking technologies, such as Ethernet.

The quality of the reconstructed images relies, among other parameters, on the synchronization precision of all the image capturing apparatuses: all the image capturing apparatuses must activate their respective shutter simultaneously, to avoid inconsistencies and artefacts in the reconstructed sequence of images.

Precisely synchronizing the start of capture of a sequence of images by all the image capturing apparatuses of a network is a real challenge.

The synchronization methods of the prior art are mainly based on the use of a time source taken as a reference for all the image capturing apparatuses of the network, on which each image capturing apparatus is synchronized, and from which start times of image capture can be calculated.

In case of failure in the network, an image capturing apparatus may not have a reliable access to this reference time source, which may lead to a significant shift between a clock of this image capturing apparatus and clocks of other image capturing apparatuses of the network. As a result, the captures of an image sequence do not start at the exact same time for all the image capturing apparatuses, which impacts the quality of reconstructed image sequences.

There is thus a need for a method for synchronizing a plurality of image capturing apparatuses of a network that is robust in case of a failure of this network.

SUMMARY OF THE INVENTION

It is provided a method for controlling an image capture device, the method being performed in a control device comprising a first synchronization unit and a second synchronization unit, the control device being connected to a remote time server through a communication network. The method may comprise: synchronizing the first synchronization unit with the remote time server according to a time synchronization protocol; synchronizing the second synchronization unit with the first synchronization unit; obtaining, at the second synchronization unit, a first capture time control information; detecting a communication failure of the communication network; upon said detecting, determining, at the second synchronization unit, a second capture time control information; and controlling the image capture device to capture a sequence of images according to the determined second capture time control information.

The first capture time information and the second capture time information may be, for instance, capture start times or information relative to capture start times. By "capture start time", it is meant a time for starting or restarting image capture by the image capture device.

Such method advantageously makes it possible to readjust, at the control device, a capture time information in the event of a failure in the communication network. A failure may occur, for instance, when too much data flows through the communication network and does not necessarily reach its respective destination on time. For example, a first capture time control information may be transmitted to the control device by a remote entity, but the transmission may be delayed, so the control device cannot control the capture device to start capturing images according to the received first capture time control information. Another possible failure may consist in a loss of synchronization between the first synchronization unit with the remote time server. In such case, readjusting the value of the previous capture time control information by the control device as soon as the synchronization is recovered makes it possible to quickly resynchronize the image capture between the image capture device concerned by the loss of synchronization and the other image capture devices, and to therefore minimize errors in the reconstruction of images during the post-processing of the data.

In one or several embodiments, the first capture time control information may be received from a remote control apparatus. Alternatively, the first capture time control information may be determined by the control device itself. Several methods may be used for determining, by the remote control apparatus or by the control device, the first capture time control information, as described below.

In one or several embodiments, the first synchronization unit may comprise an internal clock, and the first capture time control information may be determined by the control device. The controlling of the image capture device may then comprise: transmitting synchronization signals to the image capture device, wherein the synchronization signals are generated according to the SMPTE 2059 standard. The first capture time control information may then correspond to an alignment point calculated based on a current value of the internal clock.

Definition and computation methods of alignment points are provided in the SMPTE 2059 standard.

The above method may further comprise: obtaining, at the second synchronization unit, a first value of a time code; upon the detecting of the communication failure of the communication network, determining, at the second synchronization unit, a second value of the time code; and the controlling of the image capture device may further comprise: transmitting the second value of the time code to the image capture device.

Time codes are classically used in the post-processing of data received from image capture devices, in particular for synchronizing frames corresponding to a capture of the scene by the different image capture devices at a same time. Adjusting the value of the time code in the event of a network failure makes it possible to quickly re-establish the correspondence between sequences of images captured at same times, and to reduce reconstruction errors.

In one or several embodiments, the first synchronization unit may comprise an internal clock, and the communication failure may be detected by comparing the first capture time control information with a threshold time, said threshold time being function of a current value of the internal clock.

For instance, the threshold time may be equal to a sum of the current value of the internal clock and a predefined processing delay, and the communication failure may be detected when the first capture time control information is lower than the threshold time.

By "current value" of the internal clock, it is meant the value of the internal clock at the time at which the comparison between the first capture time control information and the threshold is performed.

When the first synchronization unit is synchronized with the remote time server, the value of the internal clock is set based on the value of the reference clock of the remote time server. In some cases, e.g. when the data transmissions within the communication network are delayed, it is not possible to start the image capture according to the first capture time control information (for example because the first capture time control information is lower than or too close to the current value of the current value of the internal clock). In such cases, it is necessary to readjust the

S

capture time control information for starting the image capture by the capture image device.

As mentioned above, a communication failure may also correspond to a loss of synchronization between the first synchronization unit and the remote time server.

In such cases, the method may further comprise, in one or several embodiments: upon receiving the first capture time control information, controlling the image capture device to capture a sequence of images according to the first capture time control information; upon detecting the loss of synchronization: receiving data relative to a synchronization recovery between the first synchronization unit and the remote time server; and upon receiving of said data, performing the determining of the second capture time control information.

The second capture time control information is thus determined when the synchronization between the first synchronization unit and the remote time server has been recovered.

In addition, the method may further comprise, upon detecting the loss of 20 synchronization: stopping synchronizing the second synchronization unit with the first synchronization unit; controlling the image capture device to capture a sequence of images independently from a clock of the first synchronization unit until receiving the data relative to the synchronization recovery; and upon receiving the data relative to the synchronization recovery, resynchronizing the second synchronization unit with the first synchronization unit.

When a loss of synchronization is detected, stopping the synchronization between the second synchronization unit and the first synchronization unit may advantageously reduce the lag between the image sequences captured by the image capture device controlled by the control device affected by the loss of synchronization compared to other image capture devices of the network.

In one or several embodiments wherein the first synchronization unit comprises an internal clock, the second capture time control information may be determined based on a current value of the internal clock.

In particular, the controlling of the image capture device may comprise: transmitting synchronization signals to the image capture device; and the synchronization signals may be generated according to the SMPTE 2059 standard The second capture time control information may then correspond to an alignment point calculated based on a function of the current value of the internal clock.

In one or several embodiments, the second capture time control information may be function of the first capture time control information.

In particular, when upon receiving the first capture time control information, the image capture device is controlled to capture a sequence of images according to the first capture time control information, and when upon detecting the loss of synchronization, the method comprises: receiving data relative to a synchronization recovery between the first synchronization unit and the remote time server; and upon receiving of said data, performing the determining of the second capture time control information, the method may further comprise: determining a number of frames captured by the image capture device between the first capture time control information and a time at which the second capture time control information is determined; wherein the second capture time control information is function of the first capture time control information, the determined number of frames and a clock period of the image capture device.

For instance, the second capture time control information may be set to to + N x T, where to is the first capture time control information, N is the number of frames, and T is the clock period.

For instance, the number of frames may be equal to: N cell (t1 ± St tel) where ceil() is a ceiling function, t, is the time at which the second capture time control information is determined, and St is a predefined processing delay value.

Another aspect of the invention relates to a computer program product for a programmable apparatus, the computer program product comprising a sequence of instructions for implementing a method for controlling an image capture device as presented above, when loaded into and executed by the programmable apparatus.

Yet another aspect relates to a control device comprising a first synchronization unit and a second synchronization unit, the control device being configured to be connected to a remote time server through a communication network, the control device comprising a processor configured for: synchronizing the first synchronization unit with the remote time server according to a time synchronization protocol; synchronizing the second synchronization unit with the first synchronization unit; obtaining, at the second synchronization unit, a first capture time control

B

information; detecting a communication failure of the communication network; upon said detecting, determining, at the second synchronization unit, a second capture time control information; and controlling the image capture device to capture a sequence of images according to the determined second capture time control information.

Another aspect of the invention relates to a method for controlling a plurality of image capture devices in a system comprising the plurality of image capture devices, to a plurality of control devices, each control device being associated with a respective image capture device, a remote time server and a remote control apparatus, wherein each control device comprises a respective first synchronization unit and a respective second synchronization unit, wherein each control device is respectively connected to the remote time server and to the remote control apparatus through a communication network, and wherein the remote control apparatus is connected to the remote time server through the communication network; the method comprising: synchronizing, at each control device, the respective first synchronization unit with the remote time server according to a time synchronization protocol; synchronizing, at each control device, the respective second synchronization unit with the respective first synchronization unit; determining, at the remote control apparatus, an intermediate capture time control information and sending the intermediate capture time control information to each control device; obtaining, at the respective second synchronization unit of each control device, a first capture time control information based on the received intermediate capture time control information; wherein the method further comprises, at a given control device among the plurality of control devices: detecting a communication failure of the communication network; upon said detecting, determining, at the second synchronization unit, a second capture time control information; and controlling the image capture device to capture a sequence of images according to the determined second capture time control information.

In one or several embodiments, the determining, at the remote control apparatus, of the intermediate capture time control information may comprise: receiving, from the remote time server, an indication relative to a current time of the time server; and setting the intermediate capture time control information as a function of the received indication relative to the current time of the time server; wherein the first capture time control information is function of the intermediate capture time control information.

Alternatively, the time synchronization protocol may conform to the SMPTE 2059 standard, and the determining, at the remote control apparatus, of the intermediate capture time control information may comprise: receiving, from the remote time server, an indication relative to a current time of the time server; and setting the intermediate capture time control information as a function of an alignment point calculated based on the received indication relative to the current time of the time server.

The first capture time control information may then be function of the intermediate capture time control information.

Alternatively, when the time synchronization protocol conforms to the SMPTE 2059 standard, the determining, at the remote control apparatus, of the first capture time control information may comprise: receiving, from the remote time server, an indication relative to a current time of the time server; and setting the intermediate capture time control information as a function of the received indication relative to the current time of the time server.

The first capture time control information may then be function of an alignment point calculated based on the received intermediate capture time control information.

Another aspect of the invention relates to a computer program product for a programmable apparatus, the computer program product comprising a sequence of instructions for implementing a method for controlling a plurality of image capture devices as presented above, when loaded into and executed by the programmable apparatus.

It is also provided a communication system comprising a remote time server, a remote control apparatus, a plurality of image capture devices and a plurality of control devices, each control device being associated with a respective image capture device, wherein each control device comprises a respective first synchronization unit and a respective second synchronization unit, wherein each control device is respectively connected to the remote time server and to the remote control apparatus through a communication network, and wherein the remote control apparatus is connected to the remote time server through the communication network; each control device comprising: a first synchronizing circuit for synchronizing the respective first synchronization unit with the remote time server according to a time synchronization protocol; a second synchronizing circuit for synchronizing the respective second synchronization unit with the respective first synchronization unit; the remote control apparatus comprising a determining circuit for determining an intermediate capture time control information and sending the intermediate capture time control information to each control device; each control device further comprising: a circuit for obtaining, at the respective second synchronization unit, a first capture time control information based on the received intermediate capture time control information; a circuit for detecting a communication failure of the communication network; a circuit for determining, at the second synchronization unit and upon said detecting, a second capture time control information; and a circuit for controlling the image capture device to capture a sequence of images according to the determined second capture time control information.

At least parts of the methods according to the invention may be computer implemented. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a "circuit", "module" or "system". Furthermore, the present invention may take the form of a computer program product embodied in any tangible medium of expression having computer usable program code embodied in the medium.

Since the present invention can be implemented in software, the present invention can be embodied as computer readable code for provision to a programmable apparatus on any suitable carrier medium. A tangible carrier medium may comprise a storage medium such as a floppy disk, a CD-ROM, a hard disk drive, a magnetic tape device or a solid state memory device and the like. A transient carrier medium may include a signal such as an electrical signal, an electronic signal, an optical signal, an acoustic signal, a magnetic signal or an electromagnetic signal, e.g. a microwave or SF signal.

Other features and advantages of the method and apparatus disclosed herein will become apparent from the following description of non-limiting embodiments, with reference to the appended drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

Some embodiments of the present invention are illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings, in which like reference numerals refer to similar elements and in which: -Figure la and lb represent an example of configuration of a system in which some embodiments of the present invention can be used; -Figure 2 is a block diagram showing an example of an arrangement of a synchronization signal output apparatus in one or several embodiments; -Figure 3 is an example of a flow chart describing an embodiment of the invention; and -Figures 4a and 4b illustrate an embodiment for calculating a new value of the start target time.

DESCRIPTION OF SOME EMBODIMENTS

According to embodiments, the invention allows a video device to capture images synchronously with other video devices connected to a same network, even in the event of a network failure. By "video device", it is meant a combination of an image capture device (hereinafter referred to as "image capturing apparatus"), configured for capturing images of a scene, and a control device (hereinafter referred to as "synchronization signal output apparatus"), configured for controlling the image capturing apparatus. In classical systems, a time information corresponding to a moment for starting the image capture may be received by the synchronization signal output apparatus, and the synchronization signal output apparatus may control the image capturing apparatus so that it begins capturing images according to the information received. However, when a network failure occurs, this information is no longer reliable and usable. Instead of waiting for new time information and thus accumulating discrepancies between the image capturing apparatuses, the present invention proposes to calculate, at the synchronization signal output apparatus, an updated value of the next moment for re-starting the image capture. Thus, a re-synchronization of the synchronization signal output apparatuses of the system is carried out in the event of a network failure, which makes it possible to minimize the errors in the reconstructed images during the post-processing of the data.

In fact, in multi-equipment systems, moments of start of capture can be received by dedicated equipment (hereinafter called "synchronization equipment") Figure la represents an example of configuration of a system wherein some embodiments of the present invention can be used. This system includes, for example, a plurality of image capturing apparatuses 110a to 110z, a plurality of synchronization signal output apparatuses 111a to 111z respectively corresponding to the plurality of image capturing apparatuses 110a to 110z, and a time server 130.

The plurality of image capturing apparatuses 110a to 110z may be connected to the corresponding synchronization signal output apparatuses 111a to 111z using respective wired lines or wireless channels 112a to 112z. In the following, the image capturing apparatuses 110a to 110z, the synchronization signal output apparatuses 111a to 111z and the wired lines or wireless channels 112a to 112z are generically referred to as image capturing apparatuses 110, synchronization signal output apparatuses 111 and wired lines or wireless channels 112, respectively, when it is unnecessary to distinguish them.

In addition, the system may comprise a generation unit (not shown) that generates a virtual viewpoint image (or a virtual viewpoint video) based on a plurality of captured images (or captured videos) obtained by performing image capturing from a plurality of directions by the plurality of image capturing apparatuses 110.

Each of the plurality of synchronization signal output apparatuses 111 may be connected to the time server 130 via a network 120. The plurality of synchronization signal output apparatuses 111 may be directly connected to the time server 130 by a wired line or wireless channel, or connected to the time server 130 via another apparatus on the network 120. The network 120 may be for example a star topology Ethernet network, or a daisy chained Ethernet network, or any kind of network compatible with the synchronization protocol.

Schematically, several distinct synchronization mechanisms may be carried out within the elements of this system, as shown in Figure lb. During a first synchronization mechanism, for each synchronization signal output apparatus 111 of the system, a respective clock of the synchronization signal output apparatus 111 may be synchronized to a reference time held by a time server 130.

For instance, this first synchronization mechanism may be carried out by using first synchronization signals ("Sync_sig_1" in Figure lb) respectively outputted from the time server 130 to the synchronization signal output apparatuses 111.

In one or several embodiments, this first synchronization mechanism may be carried out using a time synchronization protocol such as NIP (for "Network Time Protocol") or IEEE 1588 PIP (for "Precision Time Protocol"), as set forth in the document entitled "IEEE Standard for a Precision Clock Synchronization Protocol for Networked Measurements and Control Systems" by IEEE Instrumentation and Measurements Society, Jul. 24, 2008.

Then, the synchronization signal output apparatus 111 may control the image capturing apparatus 110 to which it is connected so that it begins capturing images of a scene at a given time. This control mechanism may actually be seen as a second synchronization mechanism. Indeed, for controlling the image capturing apparatus 110, the synchronization signal output apparatus 111 may output second synchronization signals ("Sync_sig_2" in Figure lb) to the respective image capturing apparatus 110. Upon receiving the synchronization signal, the respective image capturing apparatus 110 may start capturing the image sequence. In alternative embodiments, a control command might be used to instruct the image capturing apparatus 110 to start capturing the image sequence, if provided by the protocol used.

In one or several embodiments, this second synchronization mechanism may be carried out using the Serial Digital Interface (SDI) Genlock and timecode mechanisms The time at which the second synchronization mechanism is performed is referred to as "start target time" hereinafter. This start target time corresponds to the time at which the image capture by the respective image capturing apparatus 110 starts. In one or several embodiments, the start target time may be provided by a control apparatus 140, as further detailed below. Alternatively, no control apparatus 140 is used and the start target time may be determined by each synchronization signal output apparatus 111. For instance, the start target time may be determined based on a common reference time for all the synchronization signal output apparatuses 111 (e.g. SMPTE Epoch, as defined in the SMPTE ST 2059-1:2015 standard), as described below with reference to Figure 4b.

Schematically, the synchronization signal output apparatus 111 may be divided into two main units: a first synchronizing unit for synchronizing the value of the clock of the apparatus to the value of the reference time of the time server 130, and a second synchronizing unit for generating and outputting synchronization signals in order to control the image capturing apparatus 110. The second synchronization unit may be synchronized with the first synchronization unit via an internal signal ("Int_sig" in Figure lb) which will be described in detail below, with reference to Figure 2. A detailed description of the components of the synchronization signal output apparatus 111 will also be provided below with reference to Figure 2. It is noted that, referring to Figure 2, the first synchronization unit may comprise at least the internal clock 201, and the second synchronization unit may comprise at least the synchronization signal generation unit 205.

When all the synchronization signal output apparatuses 111 align timings of starting to output the synchronization signals to their respective image capturing apparatuses 110, all the image capturing apparatuses 110 can start capturing a respective sequence of images at the same time, in synchronism with each other.

In one or several embodiments, the second synchronization signals may be signals in a form processed as "Generator Lock" (or "Genlock") signals in the image capturing apparatuses 110. It is recalled that a Genlock signal is a type of signal commonly used in the video systems for synchronization processing: generally, the Genlock signal is received by each device (cameras, production switchers, etc.) and used as a timing reference for the internal video handling performed by the device, and for synchronous output.

It is noted that the terms "first" and "second" synchronization signals have no specific technical meaning, and are used only for the sake of clarity, to distinguish the types of signals. Subsequently, these terms may be omitted if there is no reasonably possible confusion.

In one or several embodiments, a control apparatus 140 may be used for controlling time at which the plurality of synchronization signal output apparatuses 111 output the synchronization signals to the respective image capturing apparatus 110. The synchronization output apparatuses 111 may be connected to the control apparatus 140 via the network 120, or via another dedicated network.

For example, the control apparatus 140 may notify each synchronization signal output apparatus 111 of the time (i.e. the "start target time") at which the synchronization signal is output, and the synchronization signal output apparatus 111 can start to output the synchronization signal at the notified time. Since the synchronization signal output apparatuses 111 have established time synchronization with the time server 130, when the control apparatus 140 instructs synchronization signal transmission start time, the plurality of synchronization signal output apparatuses 111 can align timings of starting to output the synchronization signals. It is noted that, after the start of outputting the synchronization signal, each synchronization signal output apparatus 111 can continue outputting the synchronization signal while correcting a timing of generating the synchronization signal based on the time synchronized with the time server 130. This allows the plurality of image capturing apparatuses 110 to perform image capturing at the common timing while ensuring time synchronization.

Furthermore, simultaneously with the second synchronization signal, a time code may be sent from the synchronization signal output apparatus 111 to the respective image capturing apparatus 110. Time codes are generally used in the post-processing of the captured image sequences, e.g. for synchronizing frames corresponding to a capture of the scene by the different image capturing apparatuses 110 at a same time. Any appropriate format may be used, for instance the formats defined by SMPTE ST 12-1, which are traditionally encoded in Linear Time Code (LTC), Vertical Interval Time Code (VITC) or Ancillary Time Code (ATC) variants.

For instance, the control apparatus 140 may send a respective message (hereinafter referred to as "START_TARGET_TIME message") to each of the synchronization signal output apparatuses 111. Each START_TARGET_TIME message may include a first parameter indicating the "start target time", i.e. the time to start the capture of the scene by the respective image capturing apparatus 110.

Advantageously, the START_TARGET_TIME message may also include a second parameter corresponding to the "start time code" associated with the captured image or sequence of images. The start time code may indicate for instance the value of the time code associated with the first frame of the image sequence to capture.

In one embodiment, the "start target time" parameter may be formatted in accordance with the PTP time representation as defined in the IEEE 155-2008 standard, and the "start time code" parameter may be formatted in accordance with the ST 12-1: 2014 SMPTE Standard.

The time server 130 may be a server that distributes, via the network 120, information of the current time held in the self-apparatus (i.e. the time server 130), in accordance with a protocol such as NTP or IEEE 1588 PTP. For example, the time server 130 may obtain the current time from GPS, a standard radio wave, or an atomic clock, and distribute the obtained current time. Alternatively, the time server 130 may distribute, as time information, information of an elapsed time after the time set in the self-apparatus (i.e. the time server 130). The protocol used when the time server 130 distributes the time information is not limited to NTP or PTP, and other methods of clock synchronization of the prior art may be used.

The image capturing apparatus 110 is an apparatus that performs image capturing based on the (second) synchronization signal received from the synchronization signal output apparatus 111. It is noted that the image capturing apparatus 110 can perform image capturing without using the synchronization signal. However, when performing image capturing in synchronism with another image capturing apparatus 110, the image capturing apparatus 110 establishes time synchronization with the other image capturing apparatus 110 using, for example, the synchronization signal according to this embodiment. A video captured by the image capturing apparatus 110 may be either a moving image or a still image, and may include data such as a sound. The video captured by the image capturing apparatus 110 can be saved in a storage incorporated in the image capturing apparatus 110 or in the synchronization signal output apparatus 111, in a server accessible via the network 120, or in a storage on a cloud. It is possible to generate a virtual viewpoint content based on videos synchronously captured by the plurality of image capturing apparatuses 110 and accumulated. Alternatively, videos captured by the image capturing apparatuses 110 may be used to generate a free viewpoint content in real time without being saved in a storage.

The network 120 is a communication network that allows transmission/reception of signals between the apparatuses such as the synchronization signal output apparatuses 111 and the time server 130 via, for example, a wired line, a wireless channel using Wi-Fie, Bluetoothe, or the like, or a combination thereof. The network may include a relay apparatus such as a hub, a wireless LAN station, or a PC that can relay data between the plurality of synchronization signal output apparatuses 111 and the time server 130. It is noted that, in the example of Figure 1a, each of the plurality of synchronization signal output apparatuses 111 is connected to the network 120. However, the present invention is not limited to this architecture. For example, the system may be configured so that the plurality of synchronization signal output apparatuses 111 may be cascade-connected and one of the plurality of synchronization signal output apparatuses 111 is connected to the time server 130 via the network 120.

The line or channel 112 may be a coaxial cable such as a BNC cable that connects the synchronization signal output apparatus 111 and the corresponding image capturing apparatus 110. Of course, different kinds of wired line, or wireless channels may be used, such as close proximity wireless communication.

It is known that if time synchronization is established via the network 120, as shown in Figures la or lb, the time synchronization accuracy lowers due to the number of relay apparatuses on the network 120 and a latency and a jitter when each relay apparatus relays data. To solve this problem, NTP provides a mechanism in which, using a hierarchical structure called stratum, the top-level NTP server obtains the standard time from an accurate clock such as GPS and a lower-level NTP server adjusts time with reference to the standard time. With this mechanism, it is expected that even a low-level NTP server distributes high-accuracy time. However, the time synchronization accuracy is lower for a lower-level NTP server. In PTP, it is possible to distribute high-accuracy time even via a plurality of relay apparatuses using the BC (Boundary Clock) or TC (Transparent Clock) mechanism.

However, to establish time synchronization between the time server 130 and the synchronization signal output apparatus 111 with high accuracy, all the relay apparatuses that relay communication between the apparatuses need to have a high-accuracy BC or IC function. If the plurality of synchronization signal output apparatuses 111 are cascade-connected, a synchronization signal output apparatus 111 can implement both high-accuracy time synchronization and a relay by having OC (Ordinary Clock) and BC or TC. If a relay apparatus supporting no BC or TC exists in the middle of a relay path, it is impossible to distribute the time to apparatuses on the downstream side of the relay apparatus. And even if all the relay apparatuses support BC or TC, if the relay apparatus has low performance, the time accuracy may degrade.

It is noted that Figures la and lb illustrate an example in which the synchronization signal output apparatus 111 and the image capturing apparatus 110 are formed as separate apparatuses. However, in one or several embodiments, the synchronization signal output apparatus 111 and the image capturing apparatus 110 may be formed as a single apparatus by, for example, including the synchronization signal output apparatus 111 in the image capturing apparatus 110.

Also, even if the example of Figure la includes a plurality of synchronization signal output apparatuses 111, each of them being respectively associated with one image capturing apparatus 110, there may be only one synchronization signal output apparatus 111 configured for outputting respective synchronization signals to the plurality of image capturing apparatuses 110. Alternatively, there may be more than one synchronization signal output apparatuses 111, each of the synchronization signal output apparatuses 111 being configured for outputting synchronization signals to a respective subset of image capturing apparatuses 110 among the plurality of image capturing apparatuses 110. In other words, the number of synchronization signal output apparatuses 111 is not limited.

Figure 2 is a block diagram showing an example of an arrangement of a synchronization signal output apparatus 111 in one or several embodiments.

The arrangement shown in Figure 2 can be implemented when, for example, one or more processors such as a CPU (for "central processing unit") executes a program stored in a storage device such as a ROM (for "read only memory") or RAM (for "random access memory"), unless otherwise specified. As the processor, for example, a processor other than a CPU, such as an ASIC (Application Specific Integrated Circuit), FPGA (Field Programmable Gate Array), or DSP (Digital Signal Processor) may be used. The example of the arrangement shown in Figure 2 is presented for the descriptive purpose. Other functional components may be included in addition to functions shown in Figure 2. If possible, at least some of the functions shown in Figure 2 may be omitted/integrated.

In one or several embodiments, the synchronization signal output apparatus 111 may include an internal clock 201, a data transmission/reception unit 202, a time synchronization unit 203, a time control unit 204, a synchronization signal generation unit 205, a synchronization accuracy determination unit 206, and a synchronization controller 207.

The internal clock 201 may be, for example, a hardware clock holding the current time. For example, based on a hardware clock, the internal clock 201 may periodically output a reference signal as a time reference in the synchronization signal output apparatus 111.

The data transmission/reception unit 202 may transmit/receive data to/from the time server 130 via the network 120. The data transmission/reception unit 202 can be, for example, a NIC (Network Interface Card). However, the present invention is not limited to this, and another component capable of transmitting/receiving data to/from the time server 130 may be used. The data transmission/reception unit 202 complies with, for example, the IEEE 1588 standard and has a function of saving a timestamp obtained when transmitting/receiving data to/from the time server 130. In some embodiments, the function of the internal clock 201 may be included in the data transmission/reception unit 202.

For example, according to a method complying with the IEEE 1588 standard, the time synchronization unit 203 may obtain time information from the time server 130, and synchronize the internal clock 201 with the time in the time server 130. It is noted that the time synchronization unit 203 may establish time synchronization with the time server 130 by a proprietary protocol or another standard such as the NTP or Ethernet AVB standard, instead of the IEEE 1588 standard. Even if the IEEE 1588 standard is used, the time synchronization unit 203 may comply with the current or future standard such as IEEE 1588-2002 or IEEE 1588-2008. It is also noted that the standard for time synchronization by IEEE 1588-2008 is also called PTPv2 (Precision Time Protocol Version 2). For example, the time synchronization unit 203 can calculate an error (offset) with respect to the time of the time server 130 by transmitting/receiving data to/from the time server 130 and calculating a transmission delay between the time server 130 and the synchronization signal output apparatus 111. The time synchronization unit 203 may hold the calculated error, and supply information of the error to the time control unit 204, the synchronization accuracy determination unit 206 and/or the synchronization controller unit 207.

The time control unit 204 may adjust the internal clock 201 based on the time of the time server 130 obtained by the time synchronization unit 203 and the time error between the time server 130 and the synchronization signal output apparatus 111.

For example, the time control unit 204 may define a threshold for the error with respect to the time held by the time server 130. If the error is larger than the threshold, the time control unit 204 largely changes the time of the internal clock 201; otherwise, the time control unit 204 may gradually correct the time of the internal clock 201.

The synchronization signal generation unit 205 may generate a (second) synchronization signal, and output it to the image capturing apparatus 110. In one or several embodiments, the synchronization signal may be for example a Genlock signal conforming to the SMPTE 274M standard, and/or a Linear Time Code (LTC) time code in accordance to the SMPTE ST 12-1 specification. However, the present invention is not limited to this, and an arbitrary periodic signal can be used as a synchronization signal. The synchronization signal generation unit 205 may include, for example, a crystal oscillator such as a VCXO (for "Voltage Controlled Crystal Oscillator") or TCXO (for "Temperature Compensated Crystal Oscillator"), and a ceramic resonator. The synchronization signal generation unit 205 may include a PLL (for "Phase Locked Loop") formed by combining a phase comparator and the like for generating a stable synchronization signal. Furthermore, to improve the accuracy, the synchronization signal generation unit 205 may be configured to apply in advance a correction value for correcting the individual difference of the crystal oscillator.

It is noted that, if possible in terms of downsizing/simplification of a circuit, the synchronization signal generation unit 205 may use a high-accuracy atomic clock or the like.

For instance, the synchronization signal generation unit 205 may generate an internal signal of the same frequency as that of the reference signal generated by the internal clock 201, and adjust a timing of generating a synchronization signal based on a phase difference between the internal signal and the reference signal.

If a jitter of the reference signal generated by the internal clock 201 is large, that is, for example, a jitter of the transmission delay with respect to the time server 130 is large, a fluctuation of the frequency of the reference signal generated by the internal clock 201 is large. As a result, the phase difference between the internal signal and the reference signal fluctuates. If the phase difference is fed back to the internal signal, the generation timing of the synchronization signal may largely fluctuate. Consequently, the image capturing apparatus 110 may not be able to perform image capturing stably.

The synchronization accuracy determination unit 206 determines the magnitude of the time error between the time server 130 and the synchronization signal output apparatus 111, that is held in the time synchronization unit 203. Then, in accordance with the determination result, the synchronization accuracy determination unit 206 may set an adjustment mode of the internal signal generated by the synchronization signal generation unit 205.

The synchronization signal output apparatus 111 may further comprise a synchronization controller 207 configured for enabling and phase aligning the synchronization signal generation unit 205, to allow synchronization signals to be outputted from the synchronization signal generation unit 205 to the image capture apparatus 110 according to start target times or new target times. Therefore, image capture may be started or restarted according to these times. In particular, the synchronization controller 207 may interact with the synchronization signal generation unit 205 by providing times (i.e. "start target times" or "target times"), either to start or to re-align the synchronization signal carried on the line or channel 112. The start time at which a (second) synchronization signal is transmitted to the image capturing apparatus 110 may be controlled after a power up, and may be realigned after recovery from a network failure, as will be detailed hereinafter with reference to Figure 3.

In one or several embodiments, the synchronization controller 207 may transmit messages to the synchronization signal generation unit 205, said messages comprising the following parameters: -a "start clock synchronization time", indicating the target time for generating or re-aligning the (second) synchronization signals transmitted from the synchronization signal output apparatus 111 to the respective image capturing apparatus 110 on the line or channel 112; -a "start clock synchronization time code", defining the time code to be transmitted from the synchronization signal output apparatus 111 to the respective image capturing apparatus 110 on the line or channel 112 when the internal clock reaches the "start clock synchronization time" value. From this time, the synchronization signal generation unit 205 may increment the value of the time code according to the image standard; -a "start" parameter, indicating that a new value of the "start clock synchronization time" parameter and/or the "start clock synchronization time code" parameter is received; -a "free run" parameter, indicating that the synchronization signal generation unit 205 must no longer be synchronized with the internal clock 201. Such mode may be useful in case of a network failure causing a synchronization loss with the time server 130. Indeed, the second synchronization (i.e. the image capturing) starts to naturally drift with respect to the other image capturing apparatuses 110 of the system, but the large jitter caused by the synchronization loss and recovery will be avoided.

Figure 3 is a flow chart describing a method for controlling the time of transmission of a synchronization signal from a synchronization signal output apparatus 111 to the respective image capturing apparatus 110 on the line or channel 112.

Part of this flow chart can represent steps of an example of a computer program which may be executed by the synchronization controller 207.

These steps may be executed, for instance, after powering on, at a step 300, the synchronization signal output apparatus 111 (and its components 201-207) and the respective image capturing apparatus 110.

Then, the synchronization between the synchronization signal output apparatus 111 and the reference time held by the time server 130 is performed (step 310).

In one or several embodiments, an indication that this synchronization is completed may be received by the synchronization controller 207. For instance, such indication may indicate that the time synchronization unit 203 is synchronized to the time server 130.

By way of example and without limitation, the synchronization between the synchronization signal output apparatus 111 and the time server 130 may be performed according to the precision time protocol (PTP) defined in the IEEE 15882008 protocol, as set forth in the document entitled "IEEE Standard for a Precision Clock Synchronization Protocol for Networked Measurements and Control Systems" by IEEE Instrumentation and Measurements Society, Jul. 24, 2008. This protocol defines a number of states and possible events for the slave nodes On the system of Figure la, the slave nodes may correspond to the synchronization signal output apparatuses 111, and the master node may correspond to the time server 130). In particular, the "SLAVE" state indicates that the slave node has selected a remote entity as the master node and that its clock synchronized with the selected master node clock. In this case, the indication that the synchronization with the time server 130 has been achieved may be a message outputted by the time synchronization unit 203 when the standard defined state "SLAVE" is reached.

Of course, other synchronization protocols and other types of indications may be used. For instance, a network time protocol (NTP) may be used. Moreover, it may happen that the synchronization is considered to be achieved according to the protocol used (e.g. when the slave node enters a "SLAVE" state in PTP), but that this synchronization is not sufficiently accurate or stable. Such situation may especially occur in case of synchronization recovery after a network failure. In this case, complementary methods (not described in the standard) can be used to determine when synchronization is "sufficiently reliable".

At step 320, the synchronization controller 207 may receive an indication relative to the value of a start target time, i.e. the time at which the capture of the image sequence has to be started by the image capturing apparatus 110. In the context of the invention, the reception of this indication may cover either a reception of this indication from the control apparatus 140, or a determination of this indication by the synchronization signal output apparatuses 111 itself (e.g. by the synchronization controller 207).

For instance, the synchronization controller 207 may receive a START TARGET TIME message from the control apparatus 140 indicating the value of the start target time.. As mentioned above, the START_TARGET_TIME message may also indicate the value of the time code associated with the first frame of the image sequence to capture. The time code value may be used for post-processing image sequences captured by a plurality of image capturing apparatuses 110, in particular for synchronizing images corresponding of a same time. The received values of the start target time and the start time code may be stored in a memory of the synchronization signal output apparatus 111.

As mentioned above, the start target time may be determined by the control apparatus 140 and sent to all the synchronization signal output apparatuses 111 to be processed by their respective synchronization controllers 207. In one or several embodiments, the control apparatus 140 may first wait for an indication that all apparatuses 111 are synchronized to the time server 130, and then determine the start target time. For instance, the start target time may be determined as being at least equal to a minimal time, said minimal time being the sum of: -the current master time (i.e. the value of the clock of the time server 130 at the moment when the start target time is determined); and -the largest possible transmission time to reach all the synchronization signal output apparatus 111 of the system; and -a predefined margin (for example 5 seconds).

Alternatively, the minimal time may be only the sum of the current master time and the predefined margin.

The current master time may be obtained by querying any one of the synchronized apparatuses 111, or querying the time server 130, or by any other

method of the prior art.

In other embodiments, the start target time may be adjusted by the control apparatus 140 to be aligned with the SMPTE-2059-1 Epoch and the image frequency of the image capturing apparatuses 110. In particular, the calculation described below with reference to Figure 4b may be applied for determining the start target time. An advantage of such determination is the interoperability that can be achieved thanks to the conformance with the ST2059-1 standard.

When the start target time is determined by the control apparatus 140, the START TARGET_TI ME message is sent by the control apparatus 140, but in some situations, this START_TARGET_TIME message may not be received by the synchronization controller 207. Such situations may occur, for instance, when the synchronization signal output apparatus 111 is cycle powered just after sending a synchronization status to the control apparatus 140 indicating that the synchronization with the time server 130 is correctly performed, or when the message is dropped in the network 120 Advantageously, the control apparatus 140 may periodically resend the START_TARGET_TI ME message. However, if the message is retransmitted too late to one of the synchronization signal output apparatuses 111, the value of the start target time may be lower than the time reflected by its internal clock 201, or greater but too close to the time reflected by its internal clock 201, which does not allow to start the capture at the indicated start target time.

The present invention then proposes, in this case, to readjust the value of the start target time, i.e. to calculate a new value of the start target time which is greater than the time reflected by the internal clock 201 (and not too close to it).

To that end, at step 325, the value of the start target time received at step 320 may be checked to determine whether an adjustment is necessary.

Therefore, at step 325, it is verified whether the received value of the start target time is greater than or equal to a time wherein tmin is the sum of the current time indicated by the internal clock 201 and a predefined delay (e.g. 1 second). The predefined delay makes it possible to compensate the delays for processing the message and/or transmitting the synchronization signal to the of image capturing apparatus 110.

If the received value of the start target time is greater than or equal to rm., no adjustment has to be performed (arrow "N" of step 325).

In that case, the image capturing apparatus 110 starts capturing the image sequence when the start target time is reached (step 340).

In one or several embodiments, the step 340 may be performed as follows. First, the synchronization controller 207 may send a message to the synchronization signal generation unit 205, wherein the value of the "start clock synchronization time" parameter is equal to the value of the start target time received at step 320, together with a "start" parameter. If, at step 320, the synchronization controller 207 also received a value of the start time code, the value of the "start clock synchronization time code" parameter may be set to this received value. The generation unit 205 may then send a synchronization signal to the image capturing apparatus 110 when the start target time is reached.

After starting capturing the image sequence (step 340), a monitoring step begins to detect a potential loss of synchronization between the synchronization signal output apparatus 111 and the time server 130. Such synchronization loss may occur in case of a network failure (for instance, due to network congestion, or link disconnection/reconnection).

For instance, in case of a PTP standard protocol, a synchronization loss may be detected when the synchronization signal output apparatus 111 leaves the "SLAVE" state.

Other embodiments are possible. For instance, a network failure indication may be received from the data transmission reception unit 202. This may be advantageous, because network failure indications are often available before synchronization loss notifications. Therefore, the system can be more reactive in case of network failure.

While no synchronization loss or network failure is detected (arrow "N" of step 345), the synchronization signal output apparatus 111 continues to be synchronized with the time server 130, and synchronization signals continue to be sent to the image capturing apparatus 110 When a synchronization loss or a network failure is detected (arrow "Y" of step 345), the coupling of the synchronization signal generation unit 205 and the internal clock 201 may be stopped. In such mode (referred to as "free running mode", or "self-running mode"), the synchronization signal generation unit 205 does not update the internal signal based on the phase difference between the internal signal and the reference signal generated by the internal clock 201.

The synchronization controller may then wait for a time synchronization recovery of the synchronization signal output apparatus 111 and the reference time held by the time server 130. This synchronization recovery is achieved at step 360. In one or several embodiments, an indication that the synchronization recovery is achieved may be received by the synchronization controller 207, similarly to step 310. In case of a synchronization recovery after a network failure, it is particularly interesting to wait until the synchronization is stable. Indeed, during the process of synchronization recovery or just after it, it is often observed that the local copy of the network time first strongly diverges from the actual value of the time held by the time server, before converging slowly towards it.

At step 330, after synchronization recovery 360, a new value of the start target time is calculated by the synchronization signal output apparatus 111 (e.g. by the synchronization controller 207), to resynchronize the image capture performed by the image capturing apparatus 110 on which there was a loss of synchronization with the image captures performed by the other image capturing apparatuses 110 (on which there was no loss of synchronization). In other words, in step 330, an adjustment of the value of the start target time is performed. The new value of the start target time may be calculated from the former available start target time (e.g. the start target time received at step 320, or a value obtained after a previous start target time adjustment 330) and the current time of the internal clock 201. The new value of the start target time calculated at step 320 actually corresponds to the moment at which other image capturing apparatuses 110 (those that have not lost synchronization with the time server 130) will capture the next image sequence. A new value of the time code may also be calculated at step 330.

Then, the process is back to step 340, i.e. the image capturing apparatus 110 starts capturing the image sequence when the new start target time is reached.

As a result, since all the synchronization signal output apparatuses 111 use a same reference time (via the time server synchronization), and start outputting synchronization signals to their respective to image capturing apparatuses 110 at a simultaneous time (the start target time received at step 320) or at an identical calculated time (the new value calculated at step 330), a simultaneous capture of the scene by all the image capturing apparatuses 110 is guaranteed.

If, at step 325, it is determined that the start target time needs to be adjusted, i.e. the received value of the start target time is lower than the time Tniin (arrow "Y" of step 325), step 330 is performed for achieving this adjustment as previously described, and the process may continue as described above.

In some embodiments, when a stable synchronization with the time server 130 is achieved (steps 310, 360), the synchronization signal generation unit 205 may change from "free running" to "network synchronization". This means that from there and while no network failure is detected, the synchronization signal generation unit 205 may be synchronized with the internal clock 201. In these embodiments, the free running mode may be de-asserted during steps 320 and 330, while a time counter integrated into the synchronization signal generation unit 205 may be synchronized to the internal clock 201.

These embodiments are possible when the value of the internal clock 201 is available at the synchronization signal generation unit 205. According to these embodiments, the synchronization signal generation unit 205 has advantageously more time to stabilize before starting outputting synchronization signals via the line or channel 112.

In these embodiments, the first output of the line or channel 112 represents an internal phase reset of the unit since the start of the synchronization happened at a previous random time. When the time counter is equal to the start target time, a reset signal may be generated and sent to a time code generator and to a Genlock generator, these both entities being integrated into the synchronization signal generation unit 205 and being configured for generating time code values and Genlock synchronization signals, respectively, to be outputted to the image capturing device 110 via the respective line or channel 112. This reset signal causes these time code values and Genlock signals phase aligned with all other synchronization signal apparatuses 111 of the network.

In alternative embodiments, the synchronization signal generation unit 205 may start transmitting synchronization signals to the image capturing apparatus 110 when the target time is reached. These embodiments may be used for instance if the interface between the internal clock 201 and the synchronization signal generation unit 205 is implemented as an electrical signal oscillating at a frequency representative of the image format. These embodiments are particularly advantageous when the internal clock 201 and the synchronization signal generation unit 205 are not implemented in the same integrated circuit (IC), saving a considerable amount of external pins. These embodiments are also particularly advantageous in the all integrated IC for reducing the number of connections between two internal modules.

In these embodiments, the synchronization of the synchronization signal generation unit 205 and the output of the line or channel 112 occur at the same time (i.e. the start target time), and resetting the internal phase is not necessary.

In these embodiments, the free running mode is de-asserted during step 335. The reference signal generated by the internal clock 201is started exactly at start target time value. So at start target time, a reset signal is generated, making the time code generator and the Genlock generator phase aligned with other synchronization signal apparatuses 111 of the network.

Figures 4a and 4b illustrate possible embodiments for calculating a new value of the start target time.

In the example of Figure 4a, the previous available value of the start target time (i.e. the start target time received at step 320, or a value obtained after a previous start target time adjustment 330) is noted to. The "current time", i.e. the time at which the adjustment must be performed, is noted t1. It is assumed that, between to and t1, a synchronization loss occurs, and a stable synchronization is recovered a little before t1. Between to and t1, the image capturing apparatus 110 continues to capture images, with a frame rate that is set according to the image format used by the image capturing apparatus 110.

Based on this frame rate, which is known, and on the time elapsed between to and ti, it is possible to determine the number N of clock periods elapsed since to. By "clock period" it is meant the time between two consecutive frames captured by the considered image capturing apparatus 110. In other words, N represents the (integer) number of frames captured between to and ti by the image capturing apparatus 110.

For instance, the number N of clock periods elapsed between to and ti may be given by: N= t{)

T

where T denotes the clock period, and ceil() is the ceiling function (i.e. the function that takes as input a real number x and gives as output the lowest integer greater than or equal to x).

In alternative embodiments, the calculation of N may take into account a delay value 5t representative of the processing delay for starting the synchronization with the image capturing apparatus 110. In such embodiments, the number N of clock periods elapsed between to and ti may be given by: + 6t-to) N = ceil The new value of the start target time may be set to to + N x T, and the new value of the time code may be set to TC0 + N x T, where TC0 is the value of the time code received at step 320, or a value obtained after a previous adjustment 330.

In one or several embodiments, the "start clock synchronization time" parameter may be set to the new calculated value of the start target time, the "start clock synchronization time code" parameter may be set to the new calculated value of the time code, and these parameters may be transmitted in a message from the synchronization controller 207 to the synchronization signal generation unit 205.

A similar method may be used in the case of an inconsistent received target time requiring adjustment (arrow "Y" of step 325 of Figure 3), even if there has been no detected failure. In such a case, to is still the previous available value of the start target time (i.e. the start target time received at step 320, or a value obtained after a previous start target time adjustment 330), and t1 corresponds to the current time, i.e. the time (according to the internal clock 201) at which the adjustment is performed.

Alternatively, the value of the start target time may be computed based on a common reference time provided by the standard itself (rather than based on an initial value of the start target time, as above). Such embodiments may be used, for instance, when the system does not comprise any control station 140 or when the system comprises a control station 140, but also requires a strict SMPTE standard 2059-1 compliancy, for example to achieve interoperability with other broadcasting equipment.

For instance, the SMPTE standard 2059-1 defines a common initial time called "SMPTE Epoch" 401, as represented in Figure 4b. All signals are generated in relation to this common initial time 401. In particular, the synchronization signals to the image capturing apparatus 110 may be generated by the synchronization signal generation unit 205 at moments called "alignment points".

In one or several embodiments, the new value of the start target time may be set to the next alignment point from ti, ti being the "current time", i.e. the time at which the adjustment must be performed.

For example, for digital high definition television, at a given time t1 (element 402 of Figure 4b), the next alignment point ("Next AP") 403 is given by the following 15 formula: NextAP = (int (H xVx T+ 1) x (H x V x T) ) where: -int(.) denotes the floor function (i.e. the function that takes as input a real number x and gives as output the greatest integer lower than or equal to x); -T is the clock period expressed in seconds, i.e. T = , where SR denotes the frequency of the sample clock, expressed in Hz; -H is the duration of an entire line of the video signal, expressed in sample clock periods; -V is the duration of a complete frame, expressed in lines; -t (element 404) is the elapsed time since the SMPTE Epoch, expressed in seconds, and rounded down to an integer if its value is not an integer. In other words, ti may be written: ti = t + At, where At (element 405) is greater than or equal to 0 second, and strictly lower than 1 second.

Alternatively, a processing delay Ot may be taken into account, and the next alignment point may be calculated from ti + The alignment point may be the horizontal alignment sample P of line 1 of the video signal. For instance, the horizontal alignment (Y sample) P is defined in the standard as 260 samples before the first sample of the digital active line (immediately after the end of the start of active video, SAV) for 750 line formats, and 192 samples before the first sample of the digital active line (immediately after the end of SAV) for 1125 line formats. Line numbers increment immediately after the last sample of the digital active line (at the first word of the end of active video, EAV, sequence).

Since the synchronization signal output apparatuses 111 are all synchronized with the time server 130, they all calculate the same value t (element 404). And since all other parameters of the above formula are constants depending only on the image format, all the synchronization signal output apparatuses 111 can calculate the same value for the next alignment point, i.e. the same moment for sending the next synchronization signals to their respective image capturing apparatuses 110.

Similarly, for a given time ti (element 402) the next value of the Linear Time Code (LTC) in accordance to the SMPTE ST 12-1 specification is given by: Next LTC -Ff where: -Fl is the LTC codeword rate, expressed in codewords per second. The value of Fl can be indicated for example by a "Default System Frame Rate" field in the SMPTE Profile; and -t is defined as above.

Alternatively, a processing delay Ot may be taken into account, and the next alignment point may be calculated from ti + 6t.

In one or several embodiments, the "start clock synchronization time" parameter may be set to the new calculated value of the start target time (i.e. the next alignment point), the "start clock synchronization time code" parameter may be set to the new calculated value of the time code, and these parameters may be transmitted in a int(t x Pt) + 1 message from the synchronization controller 207 to the synchronization signal generation unit 205 A similar method may be used both in the case of a synchronization recovery after a network failure (step 330 following step 360 of Figure 3), and in the case of an inconsistent received target time requiring adjustment (step 330 following arrow "Y" of step 325 of Figure 3).

It is noted that the determination of the value of the start target time according to Figure 4b may be used by the control apparatus 140 or by each of the synchronization signal output apparatuses 111 (when the system does not comprise any control apparatus 140, or when a strict SMPTE standard 2059-1 compliancy is required, for example to achieve interoperability between all the apparatuses of the system).

For instance, when the value of the start target time is determined by the control apparatus 140, the value of the start target time may be computed based on the next alignment point from the current master time (i.e. the value of the reference time indicated by the time server 130 at which the determination of the start target time is performed). For instance, the value of the start target time may be equal to (or may be a function of) the sum of the value of the next alignment point from the current master time and of a predetermined delay (e.g. 5 seconds), or to the next alignment point from a time equal to the sum of the current master time and of a predetermined delay.

When the value of the start target time is determined by each synchronization signal output apparatus 111, the value of the start target time may be computed based on the next alignment point from the current value of the internal clock 201 (which is synchronized with the reference time of the time server 130), i.e. the value of the internal clock 201 at the time at which the determination is performed. For instance, the value of the start target time may be equal to (or may be a function of) the sum of the value of the next alignment point from the current value of the internal clock 201 and of a predetermined delay (e.g. 5 seconds), or to the next alignment point from a time equal to the sum of the current value of the internal clock 201 and of a predetermined delay. Such determination may be used, for instance, when there is no control apparatus 140.

Alternatively, the control apparatus 140 may determine a "intermediate" value of the start target time, for instance equal to (or function of) the current master time. This intermediate value may be for instance function of the sum of the current master time and of a predetermined delay (e.g. 5 seconds). The intermediate value of the start target time is then sent to each of the synchronization signal output apparatuses 111, and each of the synchronization signal output apparatuses 111 may readjust this value based on the next alignment point from the received intermediate value of the start target time. In Figure 4b, the intermediate value may correspond to ti (element 402), and the next alignment point is represented by element 403.

In a purely illustrative and non-limiting manner, four examples of embodiments are now provided. Of course, other embodiments are possible. In particular, the various methods provided above for determining an initial value of the start target time and/or an adjusted value of the start target time may be combined in one or several embodiments, at different steps of the global method represented in Figure 3.

In a first embodiment, the communication system comprises a control apparatus 140 configured for determining the start target time. The start target time may be set, for example, to the value of the current master time plus a predetermined delay value (e.g. 5 seconds). The determined value of the start target time (or information relative to this value) is then sent from the control apparatus 140 to each synchronization signal output apparatus 111 of the system (step 320 in Figure 3). If the received value of the start target time is not incompatible with the current value of the internal clock, i.e. no adjustment needs to be performed (arrow "N" of step 325 in Figure 3), each synchronization signal output apparatus 111 starts synchronizing its respective image capturing apparatus 110 (step 340 of Figure 3) according to the received value of the start target time. If a failure occurs and the synchronization between a synchronization signal output apparatus 111 and the time server 130 is lost (arrow "Y" of step 345 in Figure 3), the synchronization signal output apparatus 111 may determine, when the synchronization with the time server 130 is recovered (step 360 in Figure 3), an adjusted value of the start target time according to the method described with reference to Figure 4a and restart synchronizing the respective image capturing apparatus 110 according to this adjusted value (step 340 in Figure 3).

In a second embodiment, the communication system also comprises a control apparatus 140, and the control apparatus is configured for determining the start target time based on the next alignment point from the current master time (e.g. the next alignment point from the current master time plus a predefined delay value).

The determined value of the start target time (or information relative to this value) is then sent from the control apparatus 140 to each synchronization signal output apparatus 111 of the system (step 320 of Figure 3). If the received value of the start target time is not incompatible with the current value of the internal clock, i.e. no adjustment needs to be performed (arrow "N" of step 325 in Figure 3), each synchronization signal output apparatus 111 starts synchronizing its respective image capturing apparatus 110 (step 340 of Figure 3) according to the received value of the start target time. If a failure occurs and the synchronization between a synchronization signal output apparatus 111 and the time server 130 is lost (arrow "Y" of step 345 in Figure 3), the synchronization signal output apparatus 111 may determine, when the synchronization with the time server 130 is recovered (step 360 in Figure 3), an adjusted value of the start target time according to the method described with reference to Figure 4a and restart synchronizing the respective image capturing apparatus 110 according to this adjusted value (step 340 in Figure 3).

In a third embodiment, the communication system does not comprise any control apparatus 140. Each synchronization signal output apparatus 111 may determine the start target time. For instance, the start target time may be set to the value of the next alignment point from the current value of the internal clock 201 plus a predefined delay value. Each synchronization signal output apparatus 111 then starts synchronizing its respective image capturing apparatus 110 (step 340 in Figure 3) according to the determined value of the start target time. If a failure occurs and the synchronization between a synchronization signal output apparatus 111 and the time server 130 is lost (arrow "Y" of step 345 in Figure 3), the synchronization signal output apparatus 111 may determine, when the synchronization with the time server is recovered (step 360 in Figure 3), an adjusted value of the start target time according to the method described with reference to Figure 4a and restart synchronizing the respective image capturing apparatus 110 according to this adjusted value (step 340 in Figure 3).

In a fourth embodiment, the communication system comprises a control apparatus 140 configured for determining an intermediate value of the start target time. This intermediate value may be set, for example, to the value of the current master time plus a predetermined delay value (e.g. 5 seconds). The intermediate value (or information relative to this value) is then sent from the control apparatus to each synchronization signal output apparatus 111 of the system (step 320 in Figure 3). Each synchronization signal output apparatus 111 may then readjust this value as being the next alignment point from the received intermediate value. Then, each synchronization signal output apparatus 111 starts synchronizing its respective to image capturing apparatus 110 (step 340 in Figure 3) according to the determined adjusted value of the start target time. If a failure occurs and the synchronization between a synchronization signal output apparatus 111 and the time server 130 is lost (arrow "Y" of step 345 in Figure 3), the synchronization signal output apparatus 111 may determine, when the synchronization with the time server 130 is recovered (step 360 in Figure 3), a new adjusted value of the start target time according to the method described with reference to Figure 4a and restart synchronizing the respective image capturing apparatus 110 according to this adjusted value (step 340 in Figure 3).

Embodiment(s) of the present invention can also be realized by a computer of a system or apparatus that reads out and executes computer executable instructions (e.g., one or more programs) recorded on a storage medium (which may also be referred to more fully as a "non-transitory computer-readable storage medium") to perform the functions of one or more of the above-described embodiment(s) and/or that includes one or more circuits (e.g., application specific integrated circuit (ASIC)) for performing the functions of one or more of the above-described embodiment(s), and by a method performed by the computer of the system or apparatus by, for example, reading out and executing the computer executable instructions from the storage medium to perform the functions of one or more of the above-described embodiment(s) and/or controlling the one or more circuits to perform the functions of one or more of the above-described embodiment(s). The computer may comprise one or more processors (e.g., central processing unit (CPU), micro processing unit (MPU)) and may include a network of separate computers or separate processors to read out and execute the computer executable instructions. The computer executable instructions may be provided to the computer, for example, from a network or the storage medium. The storage medium may include, for example, one or more of a hard disk, a random-access memory (RAM), a read only memory (ROM), a storage of distributed computing systems, an optical disk (such as a compact disc (CD), digital versatile disc (DVD), etc.), a flash memory device, a memory card, and the like.

Expressions such as "comprise", "include", "incorporate", "contain", "is" and "have" are to be construed in a non-exclusive manner when interpreting the description and its associated claims, namely construed to allow for other items or to components which are not explicitly defined also to be present. Reference to the singular is also to be construed in be a reference to the plural and vice versa.

A person skilled in the art will readily appreciate that various parameters disclosed in the description may be modified and that various embodiments disclosed may be combined without departing from the scope of the invention.

Claims

CLAIMS1. A method for controlling an image capture device, the method being performed in a control device comprising a first synchronization unit and a second synchronization unit, the control device being connected to a remote time server through a communication network, the method comprising: synchronizing the first synchronization unit with the remote time server according to a time synchronization protocol; synchronizing the second synchronization unit with the first synchronization unit; obtaining, at the second synchronization unit, a first capture time control information; detecting a communication failure of the communication network; upon said detecting, determining, at the second synchronization unit, a second capture time control information; and controlling the image capture device to capture a sequence of images according to the determined second capture time control information.
2. The method of claim 1, wherein the first capture time control information is received from a remote control apparatus.
3. The method of claim 1 or 2, wherein the first synchronization unit comprises an internal clock; wherein the first capture time control information is determined by the control device; wherein the controlling of the image capture device comprises: transmitting synchronization signals to the image capture device; wherein the synchronization signals are generated according to the SMPTE 2059 standard, and wherein the first capture time control information corresponds to an alignment point calculated based on a current value of the internal clock.
4. The method of any one of claims 1 to 3, further comprising: obtaining, at the second synchronization unit, a first value of a time code; upon the detecting of the communication failure of the communication network, determining, at the second synchronization unit, a second value of the time code; wherein the controlling of the image capture device further comprises: transmitting the second value of the time code to the image capture device.
5. The method of any one of claims 1 to 4, wherein the first synchronization unit comprises an internal clock, to wherein the communication failure is detected by comparing the first capture time control information with a threshold time, said threshold time being function of a current value of the internal clock.
6. The method of claim 5, wherein the threshold time is equal to a sum of the current value of the internal clock and a predefined processing delay, wherein the communication failure is detected when the first capture time control information is lower than the threshold time.
7. The method of any one of claims 1 to 4, wherein the communication failure corresponds to a loss of synchronization between the first synchronization unit and the remote time server.
8. The method of claim 7, further comprising: upon receiving the first capture time control information, controlling the image capture device to capture a sequence of images according to the first capture time control information; upon detecting the loss of synchronization: receiving data relative to a synchronization recovery between the first synchronization unit and the remote time server; and upon receiving of said data, performing the determining of the second capture time control information.
9. The method of claim 8, further comprising, upon detecting the loss of synchronization: stopping synchronizing the second synchronization unit with the first synchronization unit; controlling the image capture device to capture a sequence of images independently from a clock of the first synchronization unit until receiving the data relative to the synchronization recovery; and upon receiving the data relative to the synchronization recovery, to resynchronizing the second synchronization unit with the first synchronization unit.
10. The method of any one of claims 1 to 9, wherein the first synchronization unit comprises an internal clock, wherein the second capture time control information is determined based on a current value of the internal clock.
11. The method of claim 10, wherein the controlling of the image capture device comprises: transmitting synchronization signals to the image capture device; wherein the synchronization signals are generated according to the SMPTE 2059 standard, and wherein the second capture time control information corresponds to an alignment point calculated based on a function of the current value of the internal clock.
12. The method of any one of claims 1 to 9, wherein the second capture time control information is function of the first capture time control information.
13. The method of claim 12 in combination with claim 8 or 9, further comprising: determining a number of frames captured by the image capture device between the first capture time control information and a time at which the second capture time control information is determined; wherein the second capture time control information is function of the first capture time control information, the determined number of frames and a clock period of the image capture device
14. A control device comprising a first synchronization unit and a second synchronization unit, the control device being configured to be connected to a remote time server through a communication network, the control device comprising a processor configured for: synchronizing the first synchronization unit with the remote time server according to a time synchronization protocol; synchronizing the second synchronization unit with the first synchronization unit; obtaining, at the second synchronization unit, a first capture time control information; detecting a communication failure of the communication network; upon said detecting, determining, at the second synchronization unit, a second capture time control information; and controlling the image capture device to capture a sequence of images according to the determined second capture time control information.
15. A method for controlling a plurality of image capture devices in a system comprising the plurality of image capture devices, a plurality of control devices, each control device being associated with a respective image capture device, a remote time server and a remote control apparatus, wherein each control device comprises a respective first synchronization unit and a respective second synchronization unit, wherein each control device is respectively connected to the remote time server and to the remote control apparatus through a communication network, and wherein the remote control apparatus is connected to the remote time server through the communication network; the method comprising: synchronizing, at each control device, the respective first synchronization unit with the remote time server according to a time synchronization protocol; synchronizing, at each control device, the respective second synchronization unit with the respective first synchronization unit; determining, at the remote control apparatus, an intermediate capture time control information and sending the intermediate capture time control information to each control device, obtaining, at the respective second synchronization unit of each control device, a first capture time control information based on the received intermediate capture time control information; wherein the method further comprises, at a given control device among the lo plurality of control devices: detecting a communication failure of the communication network; upon said detecting, determining, at the second synchronization unit, a second capture time control information; and controlling the image capture device to capture a sequence of images according is to the determined second capture time control information.
16. The method of claim 15, wherein the determining, at the remote control apparatus, of the intermediate capture time control information comprises: receiving, from the remote time server, an indication relative to a current time of the time server; and setting the intermediate capture time control information as a function of the received indication relative to the current time of the time server; wherein the first capture time control information is function of the intermediate capture time control information.
17. The method of claim 15, wherein the time synchronization protocol conforms 25 to the SMPTE 2059 standard; wherein the determining, at the remote control apparatus, of the intermediate capture time control information comprises: receiving, from the remote time server, an indication relative to a current time of the time server, and setting the intermediate capture time control information as a function of an alignment point calculated based on the received indication relative to the current time of the time server; wherein the first capture time control information is function of the intermediate capture time control information.
18. The method of claim 15, wherein the time synchronization protocol conforms to the SMPTE 2059 standard; wherein the determining, at the remote control apparatus, of the first capture time control information comprises: receiving, from the remote time server, an indication relative to a current time of the time server; and setting the intermediate capture time control information as a function of the received indication relative to the current time of the time server; is wherein the first capture time control information is function of an alignment point calculated based on the received intermediate capture time control information.
19. A communication system comprising a remote time server, a remote control apparatus, a plurality of image capture devices and a plurality of control devices, each control device being associated with a respective image capture device, wherein each control device comprises a respective first synchronization unit and a respective second synchronization unit, wherein each control device is respectively connected to the remote time server and to the remote control apparatus through a communication network, and wherein the remote control apparatus is connected to the remote time server through the communication network; each control device comprising: a first synchronizing circuit for synchronizing the respective first synchronization unit with the remote time server according to a time synchronization protocol; a second synchronizing circuit for synchronizing the respective second synchronization unit with the respective first synchronization unit; the remote control apparatus comprising a determining circuit for determining an intermediate capture time control information and sending the intermediate capture time control information to each control device; each control device further comprising: a circuit for obtaining, at the respective second synchronization unit, a first capture time control information based on the received intermediate capture time control information; a circuit for detecting a communication failure of the communication network; a circuit for determining, at the second synchronization unit and upon said detecting, a second capture time control information; and a circuit for controlling the image capture device to capture a sequence of images according to the determined second capture time control information.
20. A computer program product for a programmable apparatus, the computer program product comprising a sequence of instructions for implementing a method according to any one of claims 1 to 13 or to any one of claims 15 to 18, when loaded into and executed by the programmable apparatus.