WO2015193049A1 - Automated camera-based time measurement system and method - Google Patents

Automated camera-based time measurement system and method Download PDF

Info

Publication number
WO2015193049A1
WO2015193049A1 PCT/EP2015/061137 EP2015061137W WO2015193049A1 WO 2015193049 A1 WO2015193049 A1 WO 2015193049A1 EP 2015061137 W EP2015061137 W EP 2015061137W WO 2015193049 A1 WO2015193049 A1 WO 2015193049A1
Authority
WO
Grant status
Application
Patent type
Prior art keywords
competitor
code
optical
time
time measurement
Prior art date
Application number
PCT/EP2015/061137
Other languages
French (fr)
Inventor
Oliver Gress
Benedikt JUSTEN
Jens Hermann Müller
Original Assignee
Swiss Timing Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date

Links

Classifications

    • GPHYSICS
    • G07CHECKING-DEVICES
    • G07CTIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE
    • G07C1/00Registering, indicating or recording the time of events or elapsed time, e.g. time-recorders for work people
    • G07C1/22Registering, indicating or recording the time of events or elapsed time, e.g. time-recorders for work people in connection with sports or games

Abstract

Timing system (10) for automatically measuring split and/or finish times of competitors during a race, comprising a photofinish camera (11) centered on a finish or intermediate line and a control unit for triggering the capture of line scan images by the photofinish camera (11), wherein optical markers (M) are provided and respectively associated with each competitor. The timing system (10) further comprises a detection module (13) for automatically detecting the optical markers (M), thereby identifying each competitor and assigning a timing result (t_f) to each said competitor.

Description

Auto mated camera-based ti me measu rement system and

method

Technical domain

The present invention deals with a system and method that automatically measures split and finish times of race participants.

State of the art

Timing systems for sport events often use camera-based systems in order to measure the time of contestants when crossing a finish line. Such so-called photofinish systems consist of a camera using a single column of pixels centered on a finish line, which records a sequence of shots at a very high frame rate. Each shot corresponds to a given time, and the aggregation of the successive columns of pixels of each shot then yields a picture that hopefully allows to visually disambiguate contestants whose respective times are close to each other at the finish line level. Screening the picture column by column also allows, on the other hand, for a precise time assignment to each of them.

However, this kind of system involves manual operation and thus delays the delivery of exact timing results. Moreover, extensive practical experience is required from the operator carrying out the time assignment operation because he would need to know which part of the body is supposed to be taken into account (e.g. the chest for running competitions) for spotting the exact moment of the line crossing, and also be able to recognize and cope with perturbations. Therefore, it is most of the time only used as an auxiliary timing system, whenever required, yet only if the main timing system assigns times that cannot discriminate between several participants of a race, especially amongst the top finishers. As an alternative to manual timing, automatic timing systems have been known for a long time, and their precision has steadily increased since they were launched. More specifically, current timing systems working with transponders now provide a timing precision of 1 /100 of second, or even better. A strong limitation of these systems though is that they usually involve loop detectors which are not suited for water sports, due to the signal disturbance induced by random water reflection. As a result, these systems are confined to terrestrial kinds of events, but cannot be applied, unfortunately, to nautical or maritime sporting events.

For such events, such as regattas, other automatic timings systems such as G PS-assisted ones are known; yet they obviously do not provide comparable timing precision.

In order to automate the provision of race results, the patent US6545705 teaches a solution where a digital line scan camera is coupled with a microprocessor that is able to detect special marking patterns in a photofinish image in order to identify contestants, whereas crossing times of contestants are determined separately without the help of these markers, e.g. through detection of body features. In order to assess crossing times, preferably start and end movements of moving objects are flagged by the microprocessor when the latter detects pixel changes; in contrast, identification may involve a specific color assignment for each contestant, different types of passive optical markers or optical character recognition of alphanumeric characters. The reliability of the identification process is yet reduced due to the possibility of occlusion and misreading of the sought-for markers, and the automatic timing precision is poor since it cannot be ascertained which part of the body of which contestant crossed the line first.

On the other hand, optical-marker systems using active patterns are also known, yet in a very different context. The JP2006-033329 document deals for example with such an optical-marker system which is aimed at transmitting tags that jointly convey the current time as well as identification information about predetermined locations to a plurality of independent photographing devices, so that photographing time differences and the position from which pictures were taken from different devices can be assessed. The tags preferably consist of encoded blinking patterns emitted by IR LEDs disposed at various fixed locations and thus constantly evolve over time. Such evolving blinking patterns are however totally inapplicable as an input parameter for automatic race timing results calculation, where precisely unknown crossing times need to be ascertained. Moreover, the heterogeneous encoding schemes used therein for identification purposes would not ensure any fairness amongst participants of a race for automatically detecting crossing times, which makes these patterns not suitable at all for timing applications.

As a result, there is therefore a need for an new timing system and time measurement method that would be free from these above mentioned limitations.

Brief summary of the present invention

A goal of the present invention is to provide an alternative timing solution to transponder-based systems, showing similar features such as automated timing and identification of participants of a race.

Another goal of the present invention is to provide an improved automated timing solution showing better timing precision, and that would be available for a wider range of sport types as well.

These goals are achieved by a timing system for automatically measuring split and/or finish times of competitors during a race, comprising a photofinish camera centered on a finish or intermediate line and a control unit for triggering the capture of line scan images by said photofinish camera, wherein optical markers are provided and respectively associated with each competitor, characterized in that said timing system further comprises a detection module for automatically detecting said optical markers, whereby said optical markers are not only intended to identify each competitor, but also to help assign a timing result to each said competitor..

These goals are also achieved by a time measurement method using such a timing system hardware, comprising a first step of triggering the capture of a line scan image by a photofinish camera, characterized in that it comprises a second step of automatically detecting optical codes respectively associated to each competitor on said line scan image, whereby said optical codes are not only intended to identify each competitor, but also to help assign a timing result to each said competitor.

An advantage of the timing system and time measurement method according to the present invention is that it provides a timing precision close to the one of a photofinish apparatus, without yet requiring any manual operation any more. Another advantage of the timing system and time measurement method according to the present invention is that they are now totally compliant with water sports events.

An additional advantage of the timing system and time measurement method according to the present invention is that it requires limited capital and operational expenditures, because it is possible to leverage existing photofinish infrastructures.

Still another advantage of the proposed timing system and time measurement method according to the present invention as compared to the prior art solution is that it uses a the same optical markers and codes for both identification purposes and time assignment for each competitor, which greatly simplifies the overall processing through the synergies realized between these two computation steps.

According to a preferred embodiment, the markers are LED active markers, and the encoding scheme consists of back to back sequences of ON/OFF pulse signals sent during fixed time intervals. Preferred coding patterns use a same number of ON and OFF pulses for each participant, in order to ensure timing fairness amongst participants, from a statistical point of view, as well as a limited number of OFF intervals, so that timing errors inherent to the system a minimized. Indeed, the latter cannot detect when a competitor enters the field of view of the camera at this very OFF moment. According to a particularly preferred embodiment using such an ON/OFF coding pattern, the code frequency is augmented in order to better discriminate the code signals from strong background noise.

According to another preferred embodiment, infra-red LEDs are employed as active optical markers, so that there is no negative side effects for the attending audience, and that background noise is also limited to that spectrum band at the same time.

Brief description of the drawings

Examples of preferred embodiment for the present invention are detailed in the following description and illustrated by the appended figures, in which:

- Figure 1 illustrates a schematic view of the timing system according to a preferred embodiment for the present invention;

- Figure 2 illustrates the impact of ON/OFF interval duration with respect to the sampling rate of a photofinish image;

- Figure 3 illustrates different codes obtained for two competitors in an ideal photofinish image according to a preferred embodiment for the present invention; - Figure 4 illustrates a real photofinish image taken at 10000 frames per second of a high frequency infra-red code in an environment of strong background noise due to sunlight reflections over water surface; Figure 5 illustrates a real photofinish image of the same high frequency code as the one shown on figure 4, for the first competitor of a race having nine such competitors;

Figure 6 illustrates a real photofinish image of the same high frequency code as the one of figure 4, for the second competitor of a race having nine such competitors;

Figure 7 illustrates a real photofinish image of the same high frequency code as the one of figure 4, for the third competitor of a race having nine such competitors;

Figure 8 illustrates a discrete fourier transform (DFT) of so-called prototype code functions, showing translation invariance of absolute spectrum;

Figure 9 illustrates a cross-correlation of a prototype code function and a circularly translated version of the same function, showing the same similarity score;

Figure 10 illustrates the code detection process and the finish time computation according to a preferred embodiment for the present invention.

Detailed embodiments for the present invention

In the following, a timing system is provided that automatically measures split and finish times of race participants based on a photo finish camera system, but that does not require any manual operation any more. Such an automated time measurement not only provides nearly live information, but also relieves manual decision of operators. Such an automated timing system is especially also advantageous for high speed camera systems taking pictures at a frame rate F of 10000 frames per second (fps), whereby manual timing becomes challenging because edges appear smeared over a large number of frames and thus decision is hard and prone to error.

Fig. 1 shows a schematic overview of such a timing system 10 according to a preferred embodiment using a high speed photofinish camera 1 1 centered on a finish or intermediate line 2, that detects active optical markers M formed by LEDs 16 emitting light codes by being switching ON and OFF in a predefined sequence. The first marker M1 corresponds to a first competitor while the second marker M2 corresponds to a second competitor, etc. depending on the number of competitors that move in the direction D of travel towards the line 2. Each marker M, and in turn each competitor, is identified when passing the line 2, i.e. during the time when the optical markers M are visible in the field of view 15 of the photofinish camera 1 1 . Subsequently, and yet almost simultaneously, their finish time is computed.

As illustrated on fig. 1 , according to a preferred embodiment for the present invention, the optical markers M are active markers integrated in the competitors' clothes or gear, i.e. any kind of race vehicle. Since they are intended to help measure time, these optical markers M need to be arranged at places that comply with international sports federation regulations for determining the exact moment when a line is crossed; therefore, they are to be preferably installed on specific parts of race vehicles themselves, wherever available, rather than on the participants directly, due to their possible significant relative movement with respect to the vehicles that would completely flaw the timing results. A typical example for a place of these optical markers M is the forward tip of a boat in the frame of rowing competitions.

Due to the limited field of view of the photofinish camera 1 1 , which is usually restricted to a single column of pixels, or a single active pixel wide array, active markers will be preferred over reflective passive markers such as e.g. infra-red markers used for motion capture, that would not be suited for the very high alignment constraints implied by the present invention. Moreover, they are subject to deformation depending on their speed and acceleration, which makes it extremely hard to encode the identity of a competitor, and the high level of light energy that needs to be emitted towards the finish line in order to have enough light reflected back at the photofinish camera level makes it quite impractical. However, the active optical markers M also preferably emit in the infra-red spectrum band and the photofinish camera 1 1 comprises an infra-red - also just referred to as IR - sensor 14, so that the signalling and recording occurs only in a small spectral range outside the visible spectrum, which alleviates the problems that arise when working with visible light signals. Indeed, beside the fact that the optical markers M would not be visible and thus would not disturb the audience, in contrast to visible light, the intensity of background IR light is very low compared to the optical markers' M signals in these spectrum band. Hence the optical markers' M signals are well detectable. In order to cope with the only major exception consisting in directed reflection of sunlight, which appears at the surface of water or on polished objects, and may prove to be a critical issue for nautical kind of sports the invention is especially aimed at, high frequency codes are proposed, as the ones discussed later in this document and are illustrated on figs. 4-8.

In order to avoid any interference with the visible spectrum, other spectral bands outside of it could of course also be considered in the frame of the present invention, such as the neighbouring UV spectral bands. However, the fact that sensors - especially CMOS - are usually by far more sensitive to IR than UV light, integration of the proposed timing solution within existing photofinish hardware is made easier, since it does not necessarily require the integration of a new dedicated sensor within the photofinish camera.

In the schematic overview provided in Fig.1 , it can be appreciated that two boxes are connected to the photofinish camera 1 1 ; actually those boxes do not necessarily represent physical devices, but rather consist of logical modules that may be integrated into dedicated hardware and/or software. The control unit 12 is among others in charge of performing the first step (A) of providing trigger signals for activating the capture mode of the photofinish camera 1 1 , which will take so called line scan images, i.e. subsequent columns of pixels each corresponding to a given time, whereby a full photofinish image is then obtained by aggregating several pixel columns. It thus is triggering the capture of line scan images. The control unit 12 can also possibly be in charge of adjusting all the degrees of freedom of the camera, typically one in translation along the direction of travel D of the competitors, and three in rotation, in order to precisely align the single of respectively active pixel column on the finish or intermediate line 2. The detection module 13 is rather a software treatment module that may be e.g. integrated into a remote PC, and is in charge of performing a second step (B) of automatically detecting codes associated to each competitor on the line scan images taken by the photofinish camera 1 1 . This second step (B) consists first in identifying a participant crossing the line 2 in a first substep (B1 ), and then computing its timing result in a second substep (B2), that may involve a number of signal treatment processing operations upon reception of the line scan images provided by the photofinish camera 1 1 and having recognized an emitted code. A preferred method for computing a timing result t_f associated to a preferred encoding scheme for the present invention is explained later further in view of Fig. 10. In fact, this detection module 13 is the one performing the automated participant recognition and time assignment operations, which were hitherto performed manually in the frame of a known photofinish system.

As explained before, the optical markers M preferably send signals by turning their IR-LEDs 16 of ON and OFF for a fixed time interval. Figure 2 shows how this time interval needs to be set with respect to the frame rate acquisition of the photofinish camera 12. Four different situations S1 , S2, S3 & S4 are shown. In the first situation S1 and the second situation S2, one ON/OFF interval is exactly equal to one frame time acquisition T, i.e. the inverse as the frame rate F (T=1 /F). In the first situation S1 , the marker signals and the camera are synchronized, which is fine. However, if synchronization should fail for any reason and that the two signals are not synchronized any more, as in the second situation S2, the ON and OFF periods, which are represented as "0" and "1 " values, may not be discriminable any more: here, pixel values P are obtained by integration over half of a "1 " and half of a "0" period, which yields average identical values for each pixel. A solution to this problem is provided by determining the ON/OFF time interval to last at least twice the time it takes for a single camera frame to be captured, as shown in the third situation S3 and fourth situation S4 on the right of figure 2. Indeed, the third situation S3 and the fourth situation S4 are similar to the first situation S1 and S2, in the sense that marker signals and camera capture are respectively synchronized and unsynchronized. The only difference is that ON/OFF has now a duration of I = 2*T, and it can be appreciated that in the fourth situation S4, the ON and OFF periods are still well discriminable, even in the worst case. As a result, in the following the term ON/OFF period will always be set to an interval of time Ί" having a length of two frames, so as to meet the requirements of the Nyquist's sampling theorem.

With this specification any code of ON - also referred to as "1 " - and OFF - also referred to as "0" - periods occurs in the photofinish image 3 similar to a bar code, as shown on Figure 3, where the photofinish image 3 is an ideal image corresponding to a concatenation of subsequent line scan images of one column of pixels, each corresponding to a given time. It can be appreciated that figure 3 shows a first code C1 and a second code C2 pertaining to two competitors (the first code C1 of the first participant is 001 1 10101 , whereas the second code C2 of the second participant is 00101 1 101 , as explained hereafter). Some real examples of these codes recorded by a photofinish camera are shown on Figures 4-8, also discussed later in this document.

This kind of encoding has been proven empirically to be more reliable than other kinds of encoding techniques such as analog amplitude or wavelength modulation due to potential noise, over- and undersaturation, etc.

According to the preferred embodiment for the present invention using this kind of bar code encoding, we try to meet the following requirements:

- codes should be sent repeatedly back to back, since the arrival time at the finish or intermediate line 2 is not known in advance. If referring to a code cycle for single transmission of the code, sending repeatedly back to back means that the code is sent over again and again instantly after the previous cycle has finished.

- the code length L (see e.g. fig. 6 hereafter) should be preferably taken as small as possible. Indeed, this length L of a code cycle determines the maximum speed of an object allowed for the system to work because the optical marker M must be present in the field of view 15 of the photofinish camera 1 1 for at least the time of one whole cycle. Therefore the longer the cycle is, the lower is the maximum allowed speed of a competitor to cross the finish line. An absolute value for the maximum allowed speed cannot be easily calculated exactly, since it depends on the sensor pixel size, the lens of the photofinish camera 1 1 and also the distance of the optical markers M from the photofinish camera 12, which can change from one competition to another. However, it can be appreciated that this maximum speed is inversely proportional to the cycle length L, which itself is defined inversely proportional to the frame rate F of the photofinish camera 1 1 (NB: if the overall number of ON/OFF intervals I is referenced as K, the cycle length is L = Κ = 2*K*T = 2*K/F). As a result, the higher the frame rate F of the photofinish camera 1 1 , the higher the maximum allowed speed as well, and in practice with a frame rate F of 10000 fps or above, this will not be a limiting factor for nautical or maritime races, to which the invention is primarily directed. Yet another practical reason to keep the code length L minimal is that it directly impacts the image processing time that is required for detection and timing result assignment.

- the number of total OFF periods should be minimal because it determines the probability of a marker to enter the field of view 15 of the photofinish camera 1 1 in an OFF period which cannot be detected, and hence increases potential timing errors.

- the number of subsequent OFF intervals should also be kept minimal in order to limit the maximum time when LEDs 16 are turned OFF. This reduces the time that an optical marker M is potentially OFF when it enters the field of view of the camera, but cannot be detected as it is OFF, and hence improves the timing resolution of the proposed timing system a time measurement method.

- in order to ensure timing fairness between participants, codes of the different markers/participants must have equal statistical properties of OFF periods at least in a statistical sense, in order to ensure that the probability to enter and leave the field of view during an OFF period. Indeed, when the first ON period of a code is visible in the camera's field of view 15 that is known to be preceded by an OFF period in the code, one cannot decide if the marker has just entered the field of view or if it already had entered but was not visible due to this OFF interval. The same problem might arise when the marker leaves the field of view 15 and an OFF period might follow. Therefore, at least the chances of these events should be equal for each of the different codes. In order to facilitate the detection process of the codes, according to a preferred embodiment, every code of each competitor will be chosen with an equal overall number K of ON/OFF periods, hence determining the same code length L for every competitor. The fairness constraints explained throughout this paragraph will then in this case determine a same first number k1 of ON periods within each optical code C, and in turn, a same second number k2 of OFF periods within each optical code C. In order to guarantee an identical error distribution, , the different codes representing each participant also require an identical amount of subsequent OFF periods in a row, and then accordingly the same amount of subsequent ON periods.

In the following, two preferred code patterns are proposed and discussed in terms of performance and preferred usage scenarios.

Each of these code patterns are both made of a reference block, i.e. a flag block 4 allowing to detect the beginning and the end of a code cycle, and a position block 5, that can in practice correspond to a lane number, e.g. for rowing competitions. The flag block 4 is chosen to show a unique reference sequence in the code C, whereas the position block 5 determines a participant's number by the position of a specific sequence in that position block 5 that differs from the remainder of the position block 5 signal.

For a number N of participants, a first encoding scheme is to identify a participant by an OFF position - i.e. "0" period - within the position block 5. The code C is can e.g. composed of:

- A flag block 4 showing the sequence of periods 1010,

and

- A position block 5 showing the sequence of periods, 1 XXXX1 where the number of X = N and all X are 1 except for the n-th X = 0 to identify the n-th participant.

For N = 4, this first code is then:

1010 101 1 1 1 - for the 1 st participant

1010 1 101 1 1 - for the 2nd participant

1010 1 1 101 1 - for the 3rd participant

1010 1 1 1 101 - for the 4th participant It can be appreciated that the sequence of the flag block 4 can never be found in the position block 5; other options such as "00" could also be taken for this flag block 4; this would result in an overall shorter cycle length L, but at the same time, also in a worse intrinsic timing precision (2 frames instead of one frame).

This first proposed code pattern has the following features:

- An overall number K of 6+N ON/OFF periods, determining a code length L of (6+N)*2 frames. Hence the length increases by one period of interval I = 2 frames per additional participant;

- A constant second number k2 of 3 OFF periods within the 6+N periods, therefore irrespective of the number N of participants. As a result, the probability of timing errors, which is conditioned among others by the probability of a competitor to enter the field of view of the photofinish camera in an OFF position, depends directly on this ratio between this second number k2 of OFF periods and the overall number K of ONN/OFF periods of the cycle, decreases with N.

- The provided intrinsic timing precision, depending on the number subsequent OFF periods in the code, is equal to ±1 frame, and is thus the best possible.

For this first code, detection can be obtained merely by thresholding and calculating integral values, and the participant is identified by position of the 0 in the code after the 1010 sequence of the flag block.

This simple and short code with a small constant number of 0 periods is applicable in environments with low noise, where the code is expected to be well discriminated from mainly dark background. In other words, this code will preferably be employed for indoor competitions, or competitions taking place in any kind of environments with a static lighting conditions where very little noise is provided, and hence only very basic image processing is required to recognize the codes. More efficient processing techniques such as cross-correlation, discussed later further in view of figures 8 & 9, could also be applied, but at the expense of increased computational costs. For competitions where the noise conditions are expected to be worse, e.g. due to sun reflections over water, another type of code pattern may be employed, such as the second code detailed below, in which the participant is now identified by an ON period within a high frequency code.

In this second proposed code pattern, the flag block 4 is composed of the "00" sequence, whereas the position block 5 is 1 X1 X1 X1 , where the number of X is equal to the number N of competitors and all X are 0 except for the n-th X = 1 to identify the n-th participant.

For a number N of 4 competitors, the second code is then:

001 1 1010101 - for the 1 st participant

00101 1 10101 - for the 2nd participant

0010101 1 101 - for the 3rd participant

001010101 1 1 - for the 4th participant

Again it can be appreciated that the sequence of the flag block 4 cannot be found in the position block 5; other flag blocks, such as e.g. 01 1 1 10, could be taken in order not to have two consecutive OFF periods and thus improve the intrinsic timing precision; yet such a flag block 4 would also simultaneously increase the overall length of the code.

This second type of code pattern has the following characteristics:

- A code length L equal to (3+2N)*2 frames, due to the overall number K of (3+2N) ON and OFF periods. As a result, the Length L of this code increases by 2 intervals I, i.e. 4 frames per additional participant;

- It is a high frequency signal by alternating 1 and 0 periods, hence more robust against background noise;

- since the number of OFF periods increases with the number N of participants, the probability of timing errors remains approximately equal when the total number N of participants to a race increases; - the provided intrinsic timing precision, depending on the number subsequent OFF periods in the code, is equal to ±2 frame, and is thus slightly worse than for the first code previously introduced.

For this second code pattern, detection can also be obtained by thresholding and calculating integral values, but preferably Fourier analysis and cross-correlation will be employed in order to face noisy conditions, for which simple processing approaches are less robust. The competitors are identified by the position of the 1 1 1 sequence in the code after the 00 sequence of the flag block 4.

This high frequency code, better discriminable from strong background noise, is especially useful when used with high speed photo finish cameras with frame rates F of 10000 fps or better, since the frequencies in the code signal are then much higher than the frequencies of background noise. Then by Fourier analysis of the photofinish image in time the code signal can be distinguished from background due to Fourier coefficients of large magnitude at high frequencies, which are not present when only background signal is observed. An example of such background noise is sunlight reflected from the water surface in boat races. Due to the changes in the surface reflections occur only for a short time with strong intensity. Nevertheless, their signal remains constant for a time interval much longer than the short ON and OFF intervals of the markers' signals. In the photo finish image their appearance is thus similar to IR-LED markers that would cross the finish line in ON mode only. The above mentioned second code in combination with a 10000fps camera system still remains clearly discriminable from the noise background signal due to its high frequency nature, as illustrated on figure 4, showing a real photofinish image that uses the second code version, i.e a high frequency code with alternating ON and OFF intervals of two frames only, where the camera is pointed directly to the area of water in direction of the sun. The code C can clearly be discriminated from the reflections R of the sun. Figures. 5-7 show real photofinish images 3 using the second version of the code, i,e the high frequency code discussed previously and that was also briefly introduced on figure 3. The thicker black stripes make up the flag block 4, while the widest white block corresponds to the "1 1 1 " sequence identifying the participant within the position block 5. Hence figure 5 shows a first optical code C1 corresponding to a first competitor, since the wider white block is the first after the flag block 4, figure 6 shows a second optical code C2 corresponding to a second competitor, since the wider white block is the second white stripe after the flag block 4, and eventually figure 7 shows a third optical code C3 corresponding to a third competitor, since the wider white block is now the third white stripe after the flag block 4.

In the following, we discuss a preferred detection process for the codes C carried out by the detection module 13, as well as a preferred method for computing the finish time of participants. Due to the fact that the optical codes C used in the frame of the preferred embodiment for the present invention are periodic signals with a period length equal to the length L of a code cycle, a code C corresponding to any of the competitors can be detected as soon as a full code sequence is available in the recordings, regardless of where the recorded signal begins in the code's sequence. In other words, it is sufficient to consider the last pixel values on a sliding window of length L of the code C to determine if a valid code is currently observed and its participant number.

At the top left of figure 8, the function g(t) corresponds to a prototype code function, showing a first position block 5 of 1 1 10101 01 , and then a reference block 00. It can be appreciated that it actually matches the first code C1 of fig. 5 using to the second preferred high frequency code pattern, i.e. according to which the position of a competitor is determined by an "ON" position within the high frequency code sequence. The function g(t) is referred to as a prototype function code function because it starts at the beginning of a block, here the position block 5. However, when recording a code sequence at the finish or intermediate line 2 level (see fig. 1 ), the recorded sequence of pixels may start at any position in the code's cycle, as shown e.g. by the signal f(t) at the bottom left of Fig. 8. In fact, f(t) is the prototype function circularly translated by 6 time steps. By concatenation, i.e. appending the first values that are cut off the signal at the beginning at its end, the exact prototype function g(t) is found back.

In practice, the concatenation is avoided by techniques from the field of signal processing, which are established on the assumption of signal periodicity and thus are perfectly suited for the present task of detecting valid periodic codes and determining corresponding participant numbers. In fact, the prominent discrete Fourier transform (DFT) is an obvious choice to process the present data. It is able to extract signal spectrum features that are invariant to circular translation, i.e. these spectrum features do not change for any circularly translated version of the signal. This alleviates additional computational effort to analyze each translated version of the current signal, and allows to derive a code value instantly no matter how the signal is translated. In Fig. 8 a prototype code function and a translated version on the left are presented together with their absolute spectra computed by DFT on the right, and clearly, the absolute spectra are identical and could be compared instantly to evaluate similarity of prototype and with any recorded signal.

Furthermore, the convolution theorem linked to the DFT allows efficient computation of circular convolution, which corresponds to linear filtering. This leads to the widely employed approach of matching a template - here the prototype function - with a recorded signal by computing their cross-correlation. Here, circular cross-correlation can be used to compute a correlation coefficient for the signal and each circularly translated version of the prototype function. The maximum value yields a score of similarity. If the signal is circularly translated, cross-correlation values are translated as well. However, for code detection only the score, i.e. the maximum correlation coefficient, is of interest, which stays the same and thus is invariant to circular translation. An example of cross-correlation functions is given in Fig. 9 where two versions of circularly translated prototypes, i.e. f1 (t) and f2(t), shifted of 4 and 6 time steps respectively, are correlated with the original prototype g(t). Clearly, cross-correlation functions are identical but only translated circularly of two time steps, the maximum being at t=4 for convolution function h1 (t) = f1 (t)*g(t), whereas it is at t=6 for convolution function h2(t) = f2(t)*g(t). Note that for use in code detection, normalization of correlation coefficients is required.

As a result, irrespective on how scores are computed, the presence of a valid code is indicated when the similarity score of signal and a prototype function exceeds or falls below a predefined threshold. In this example, in view of figure 9 we could take the threshold at 10, the maximum value being 12. In any case, high scores correspond to high similarity.

Now once a code has been recognized, and hence the corresponding participant has been identified, a finish time still needs to be assigned for him. One problem that may arise in practice is that it is quite likely that optical markers are out of focus with respect to the photofinish camera, which leads to nearly isotropic diffusion of IR light in the obtained line scan image. For this reason, since this also is true for the emitted light of IR markers, the time when a marker's light just enters the image cannot be considered as finish time because the marker might not yet have actually entered the field of view of the photofinish camera. Therefore, according to a preferred embodiment for the present invention, the finish time (or crossing time for an intermediate line) is chosen to be equal to the average time of begin and time of end of the code. This yields a good estimate of the center of each marker irrespective of how distant the optical marker is from the focal plane of the camera. Yet in order to make sense from a timing perspective and not only be meant to avoid out of focus problems, it should be noted that this proposed mean finish time definition assumes a constant speed of the marker when crossing the line. This assumption is yet totally justified by the small width of the field of view (just one pixel) and the short time during which the optical markers actually travel through this field of view. Details of this preferred finish time computation method are given by the schema of figure 10, showing the detection and time computation principles of a second code C2 associated to a second participant of a race, just like the one illustrated on figure 6.

The second code C2, illustrated on the top part of figure 6, can be detected, as explained above, by computation of similarity scores in a time window of the size of code's length L. But then the similarity scores - explained earlier in view of figure 9 - above a predetermined threshold and that allow to identify a given participant are assembled in a score image S, shown on the bottom part of figure 10. This score image, consisting in connected sets of high score pixels in time and space can be obtained by standard image processing techniques, and it helps determine the time that an optical marker was visible in the field of view of the photofinish camera. Indeed, the first appearance of a high similarity score, here e.g. the first pixel value P1 of the score image S, determines a start time t_s, i.e. the first time that a full length code cycle, i.e. a sequence of the whole code's length L, is visible in the field of view of the photofinish camera. Similarly, the end time t_e corresponding to the last moment when a full code cycle is available in the field of view can directly be derived from the time of the last high score. It can be appreciated that the end time t_e actually indicates the time when the optical marker starts to leave the field of view of the camera, because from this moment on, there are no more image pixels recordings. In contrast, the first time t_0 of first appearance of a code within the field of view is the time when a first image pixel corresponding to a code sequence is detected; this time is given by the formula t_0 = t_s - L+1 , where L is the code cycle length. It can indeed be appreciated in view of the top dotted rectangle of the top half of figure 10 that from this moment on, only pixels corresponding to the code sequence will be recorded. Then, the finish time t_f is simply yielded by the average of time of first appearance of an image pixel corresponding to the detected code and the end time, i.e. t_f = ½ (t_0+t_e)

Of course, such choosing such an average value does not necessarily yield an integral value, but rounding to the next lower or upper value will not affect the timing precision that is already, in the best case, of +/- one frame.

For the sake of completeness, a second pixel value P2 and third pixel value P3 are illustrated on figure 10 to respectively show the first time when a full prototype code sequence is completed, i.e. the first time when a full flag block 4 followed by a full position block 5 are detected, and a time when the optical is not is the field of view of the camera any more.

Although other computing methods could be considered in the frame of the present invention, such as e.g. taking the sole first time t_0 of first appearance of the code in the field of view. Yet taking the time corresponding to the second pixel P2 of fig. 10, i.e. the time of end of a full prototype code sequence should preferably be avoided, because it would not confer a fair timing between competitors due to the lack of synchronization between every code cycle starts, which could hence be shifted of a maximal time value of around L/2, assuming all codes of each participants have and equal length L. According to the preferred embodiment computing the finish time by averaging as described here above, only the lack of synchronization of the clocks of each optical marker affects the timing fairness between participants, but the shift is at most equal to one frame, and hence of the same order of magnitude as the intrinsic timing precision of the timing system and method using the preferred first and second code patterns (+/- one frame for the first code pattern using OFF period for position detection, and +/- two frames for the second code pattern using ON period for position detection).

It can be appreciated that the proposed timing system and timing measurement method can provide a timing precision that is now close to the one guaranteed by photofinish systems, with now a fully automated detection and also an acceptable timing fairness between participants. Even when adding the potential timing errors (+/- 2 frames in the worst case for the second proposed code pattern) and fairness inaccuracy (+/- 1 frame, irrespective of the coding pattern chosen), the timing precision remains theoretically above 1 /1000 of second, which meets by far the standards of timing precision for international events.

The timing system and time measurement method introduced in the frame of the present invention is primarily intended for providing intermediate times, as well due to its slightly inferior precision as compared to regular manual photofinish apparatuses, as the limited capital and operational expenditures it would imply as compared to regular photofinish systems (absence of backup requirement for timing and occlusions on such lines, as opposed to finish lines). It will preferably be used in competitions where lanes are available in order to identify each participant. In this case, the position block 5 will correspond to a lane identifier and the camera viewpoint will be chosen depending on the height of the objects crossing the line - such as boats - in order to make sure that the different codes corresponding to each participant cannot overlap, and that the resulting line scan image of several participants crossing the line almost simultaneously looks like the one depicted on figure 3, i.e. with some vertical spacing between each code.

Since the timing system is based on a photofinish camera apparatus, usual masking problems between competitors crossing the line almost simultaneously could also occur. Therefore, the photofinish camera viewpoint may thus be shifted upwards in order to solve this problem. Since this issue is critical for finish times, the photofinish camera can e.g. be placed above the finish line and look down at all participants of the race. Another way to address this issue could be to have e.g. a redundant photofinish camera placed on the opposite side of the finish or intermediate line with respect to a first regular photofinish camera. Reference list

0 OFF signal of the LED

1 ON signal of the LED

2 Finish line

3 Line scan image

4 Flag

5 Position block

10 Timing system

11 Photofinish camera

12 Control unit

13 Detection module

14 Infra-red Sensor

15 Camera's field of view

16 Infra-red LED

A First detection step

B Second detection step

B1 First substep

B2 Second substep

C Optical Code

C1 First optical code of a first competitor

C2 Second optical code of a second competitor

C3 Third optical code of a third competitor

D Direction of travel of competitors/participants of a race

DFT Discret Fourier transform

F Frame rate of photofinish camera 1 Time interval for ON and OFF sending (I = 2*1)

k Overall number of ON/OFF intervals in a code C

k1 First number of ON intervals in a code C

k2 Second number of OFF intervals in a code C

L Code length in terms of number of frames (2*k*T)

M Optical marker

M1 Optical marker of a first competitor

M2 Optical marker of a second competitor

N Number of participants

P Pixel value

P1 First pixel value of the score S

P2 Second pixel value of the score S

P3 Third pixel value - not in the score S

R Sun reflection

S Score image of a recorded code

S1 Situation 1

S2 Situation 2

S3 Situation 3

S4 Situation 4

T Frame time acquisition

t_0 First time of first appearance of a code within camera field of view

t_s Start of high score

t_e End of high score

t_f Finish time = timing result

X Variable pixel value (0 or 1 , depending on position number)

Claims

CLAI M S
1 . Timing system (10) for automatically measuring split and/or finish times of competitors during a race, comprising a photofinish camera (1 1 ) centered on a finish or intermediate line (2) and a control unit (12) for triggering the capture of line scan images (3) by said photofinish camera (1 1 ), wherein optical markers (M) are provided and respectively associated with each competitor, characterized in that said timing system (10) further comprises a detection module (13) for automatically detecting said optical markers (M), whereby said optical markers (M) are not only intended to identify each competitor, but also to help assign a timing result (t_f) to each said competitor.
2. Timing system according to claim 1 , wherein each competitor is fitted with an active optical marker (M) sending an optical code (C) identifying said competitor, wherein said detection module (13) is arranged to read said optical code (C) on said line scan image (3) captured by said photofinish camera (1 1 ).
3. Timing system (10) according to claim 2, wherein said optical markers (M) are infra-red LEDs (16) and wherein said photofinish camera (1 1 ) further comprises an infra-red sensor (14).
4. Time measurement method using the timing system (1 0) of any of claims 1 to 3, comprising a first step (A) of triggering the capture of a line scan image (3) by a photofinish camera (1 1 ), characterized in that it comprises a second step (B) of automatically detecting optical codes (C) respectively associated to each competitor on said line scan image (3), whereby said optical codes (C) are not only intended to identify each said competitor, but also to help assign a timing result (t_f) to each said competitor.
5. Time measurement method according to claim 4, wherein said second step (B) of automatically detecting optical codes (C) consists of a first substep (B1 ) of identifying a competitor followed by a second substep (B2) of computing a timing result (t_f).
6. Time measurement method according to claim 5, wherein said optical codes (C) consist of a predetermined sequence of ON and OFF period signals sent by LEDs at fixed time intervals (I), and are cyclically repeated back-to-back without interruption.
7. Time measurement method according to claim 6, wherein said first substep (B1 ) of identifying a competitor involves further signal processing of said optical code (C), such as cross-correlation and/or Fourier analysis.
8. Time measurement method according to claim 6 or 7, wherein said optical codes (C) are made up by a flag block (4) and a position block (5).
9. Time measurement method according to any of claims 6 to 8, wherein said optical codes (C) have an equal length (L) for all competitors, and wherein said length (L) is chosen such that at least a whole optical code (C) is visible in the field of view (15) of said photofinish camera (1 1 ) when competitors cross the finish or intermediate line (2).
10. Time measurement method according to any of claims 6 to 9, wherein said time interval (I) for said ON and OFF signals is set to be equal to at least twice the acquisition time (T) of one frame by said photofinish camera (1 1 ).
1 1 . Time measurement method according to any of claims 6 to 10, wherein a first number (k1 ) of ON periods and a second number (k2) of OFF periods are equal for each of said optical codes (C).
12. Time measurement method according to any of claims 6 to 1 1 , wherein there are at most two subsequent OFF periods within one said optical code (C).
13. Time measurement method according to any of claims 8 to 12, wherein a competitor is identified by the position of an OFF period within a predetermined sequence of said position block (5) containing only ON periods otherwise.
14. Time measurement method according to any of claims 8 to 12, wherein a competitor is identified by the position of a block containing three consecutive ON periods within a predetermined sequence of said position block (5) containing only an alternation of ON-OFF periods otherwise.
15. Time measurement method according to any of claims 5 to 14, wherein said timing result (t_f) is obtained by averaging between a first time (t_0) of first appearance of a optical marker (M) within said field of view (15) of said photofinish camera (1 1 ), and a second end time (t_e) when said optical marker (M) leaves said field of view (15) of said photofinish camera (1 1 ).
PCT/EP2015/061137 2014-06-17 2015-05-20 Automated camera-based time measurement system and method WO2015193049A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP14172705.7 2014-06-17
EP14172705 2014-06-17

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
EP20150724261 EP3158538A1 (en) 2014-06-17 2015-05-20 Automated camera-based time measurement system and method

Publications (1)

Publication Number Publication Date
WO2015193049A1 true true WO2015193049A1 (en) 2015-12-23

Family

ID=50972525

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2015/061137 WO2015193049A1 (en) 2014-06-17 2015-05-20 Automated camera-based time measurement system and method

Country Status (2)

Country Link
EP (1) EP3158538A1 (en)
WO (1) WO2015193049A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018087369A1 (en) * 2016-11-14 2018-05-17 Osram Oled Gmbh Device, reference object for a device, and method for operating a device in order to ascertain specified information of an object along a transport route

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO1992015969A1 (en) * 1991-03-11 1992-09-17 Kjell Pettersen Device for time taking
US6545705B1 (en) 1998-04-10 2003-04-08 Lynx System Developers, Inc. Camera with object recognition/data output
JP2006033329A (en) 2004-07-15 2006-02-02 Advanced Telecommunication Research Institute International Optical marker system

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO1992015969A1 (en) * 1991-03-11 1992-09-17 Kjell Pettersen Device for time taking
US6545705B1 (en) 1998-04-10 2003-04-08 Lynx System Developers, Inc. Camera with object recognition/data output
JP2006033329A (en) 2004-07-15 2006-02-02 Advanced Telecommunication Research Institute International Optical marker system

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018087369A1 (en) * 2016-11-14 2018-05-17 Osram Oled Gmbh Device, reference object for a device, and method for operating a device in order to ascertain specified information of an object along a transport route

Also Published As

Publication number Publication date Type
EP3158538A1 (en) 2017-04-26 application

Similar Documents

Publication Publication Date Title
US4074114A (en) Bar code and method and apparatus for interpreting the same
US4567609A (en) Automatic character recognition system
US6405925B2 (en) Autodiscrimination and line drawing techniques for code readers
US20110043624A1 (en) Method and device for processing recorded image information from a vehicle
US6526160B1 (en) Iris information acquisition apparatus and iris identification apparatus
US5627915A (en) Pattern recognition system employing unlike templates to detect objects having distinctive features in a video field
US20060043191A1 (en) System and method for aiming an optical code scanning device
US20100252635A1 (en) Exposure control for multi-imaging scanner
US8090194B2 (en) 3D geometric modeling and motion capture using both single and dual imaging
Rossi et al. Tracking and counting moving people
US6922478B1 (en) Method for verifying the authenticity of an image recorded in a person identifying process
US20100078477A1 (en) Method and apparatus for operating indicia reading terminal including parameter determination
US20030067538A1 (en) System and method for three-dimensional data acquisition
Kalal et al. Forward-backward error: Automatic detection of tracking failures
WO2012066501A1 (en) Depth mapping using time-coded illumination
US5052044A (en) Correlated masking process for deskewing, filtering and recognition of vertically segmented characters
JP2001264033A (en) Three-dimensional shape-measuring apparatus and its method, three-dimensional modeling device and its method, and program providing medium
US20100019042A1 (en) Efficient multi-image bar code reader
US20100290668A1 (en) Long distance multimodal biometric system and method
US20120281879A1 (en) Method and System for 2D Detection of Localized Light Contributions
US6941026B1 (en) Method and apparatus using intensity gradients for visual identification of 2D matrix symbols
US6616044B1 (en) Methods and apparatus for obtaining imaging information in bar code scanners
US7303131B2 (en) Automatic focusing system for imaging-based bar code reader
US20060118632A1 (en) Barcode scanner decoding
Park et al. A real-time focusing algorithm for iris recognition camera

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 15724261

Country of ref document: EP

Kind code of ref document: A1

DPE1 Request for preliminary examination filed after expiration of 19th month from priority date (pct application filed from 20040101)
REEP

Ref document number: 2015724261

Country of ref document: EP

NENP Non-entry into the national phase in:

Ref country code: DE