GB2191361A - Enchancing the image of retro-reflective projectiles - Google Patents

Abstract

A colour television camera 10 has a zoom lens 11 and an angle dichroic mirror 15 behind the lens. A source of e.g. infra-red light 14 is located by the camera and directed towards the field of view of the lens 11. A projectile such as a squash or tennis ball has retro-reflectors on its surface to reflect back the infrared light from the source (14). Filter 15 passes visible light from the field of view direct to camera 10 but reflects infra-red to one side to a silvered mirror 18 and thence to a pick-up device 21 from which an enhanced image of a required colour, size and form is produced at the appropriate location in the field of view and mixed with the image produced by the camera to create an image with the projectile trajectory correspondingly enhanced. Other embodiments are disclosed in which stationary retro-reflectors are placed on the court to define the boundaries and the image received to the pick-up device is processed to determine whether the ball bounces in or out of court. <IMAGE>

Description

SPECIFICATION Trajectory analysis and image processing of retroreflective projectiles The televising of many ball (or puck) based sports events is marred by the current inability to televise the trajectory of the ball in a satisfactory way. Briefly, as the apparent velocity ofthe ball across the television screen increases, its contrast with the background diminishes. In consequence television cameras are positioned and operated in such a way as to renderthe ball as stationary as possible in relation to thefield of view as seen on the monitor screen.

In golfthe rapidly moving ball cannot be seen when the camera is stationary and the lens in a wide angle position. It can be seen by setting the zoom lens to a long focal length and panning and tilting the camera to render the ball relatively stationary to the field of viewofthe camera. This method, however, makes it impossible to get a satisfactory view ofthe surroundings, setting and context in which the ball is being played. The ball is made visible atthe cost of almost eliminating everything else.

Simiiar problems exist in ice-hockey. tennis, squash, cricket, baseball and soon. In all cases the camera must "followthe ball" in orderto reduce the apparent ball velocity in relation to the field of view and thereby clarifythe ball. Thetelevising of these games imposes severe constraints upon possible camera positions, camera movements and lens settings. Were it not for these constraints, were directors freed from these limitations, then the most artistically desirable presentation ofthese ball games might be very different from the conventions of today.This invention is not concerned with determining precisely what constitutes the best presentation of ball games but in giving directors precise control over the image ofthe ball and image ofthetrajectory ofthe ball while simultaneously allowing camera positions and movements to befreedfrom the constraints that existed before.

Atelevision sequence of a ball's trajectory consists of a succession offrames or fields in which the trajectory is defined as a ribbon shaped area which will be referred to hereinafter as the "trajectory footprint". Thefootprint, in the case of a ball as opposed to a more complicated shape such as an ice hockey puck, shows most contrast to the background at the centre ofthe footprint and decreases to zero contrast atthe sides. An elementary analysis ofthe contrast in brightness between the trajectoryfootprint and its surroundings follows, and is essentially the same for visible light, red, green or blue, infra red or infactfor any range of wavelengths.

t = exposuretimeofthefieldorframe I = average brightness ofthe central portion of the image ofthe trajectory footprint (for one field) v = velocityoftheactual ball d = diameteroftheactualball a = angle between the line joining the camera and the ball and the trajectory ofthe ball b = brightness ofthe stationary ball g = brightness ofthe background (g = gn + gi) gn= natural background brightness gi = background brightness produced by a projectile illuminator k = constant of proportionality reated to the imaging system c = contrast ratio between trajectory footprint image and background image (I-kg)/kg When vt (sin a)- d is positive kbd + kg [vt(sina)]-kgd 1= vt (sin a) k(b-g)d+kg [vt (sin a)] c = (b-g)d/[gvt(sina)] The constraints imposed upon the conventional televising of ball games are due to the necessity of making the term vt(sin a) as small as possible.

In response to these difficulties, particularly in relationtothesportofsquash, I have devised a retro-reflective method of televising that sport. My International PatentApplication No. PCT/G B84/00240 describes howthetraiectory of a ball can be televised more clearly by using a retro-reflective ball and a light (projectile illuminator) nearthe camera. Johnson US Patent No. 3944738 describes a retro-reflective ice hockey puck used in a similarway. This method increases the contrast (c) by increasing the brightness ofthe ball (b). As can be seen from the formula referred to above, an increase in (b) results in an increase in (c) Provided that (b) is largerthan (g) at the outset. This is of course not the case for. ice hockey.

The method oftelevising squash has proved to be successful and is now widely used. In spite of this, the method does have severe shortcomings which can be listed as follows:- 1. It is not practical to use lights (projectile illuminators) powerful enough to have a useful effect on games such as tennis, ice hockey or golf. Even in squash, where my system has been used in many national and international tournaments, the results arefarfrom ideal.

2. The system is subjectto contrast (c) diminution with increasing apparent two-dimensional ball velocity(vt(sin a)).

3. The ball can only be made lighter or brighter, not darker, blacker or browner. This is the opposite of what is required in games televised against a light background such as ice hockey.

4. Strong lights next to the cameras can distract the players.

5. Multiple camera positions are difficult as light from one camera can interfere with the picturefrom the other camera.

6. In squash, the lights produce undesirable reflections in any glass walis ofthe court.

Thetrajectorycontrast (c) can be made greater by increasing the power ofthe projectile illuminator as the retro-reflected light is directly proportional to this.

There is a practical limit to this and also a theoretical limit. Eventuallythe projectile illuminator becomes the dominant illumination and increases the background brightness (g) in the same proportion as the trajectory(l), producing no further improvement to the contrast (c). In otherwordsthe background brightness produced by projectile illuminator (gi) becomes greater than the natural background bright ness (gn).

The contrast can be further improved by ensuring thattheangle between the light source, the ball and the aperture ofthe camera to less than 0.4 degrees.

Forthe balls currently in use retro-reflection falls off rapidly outside this angle. If the aperture ofthe light source is large, making this difficult, it can be reflected offa mirror with a hole in it, through which the camera lens is exposed. When this is aligned correctly, from the direction in which the camera points, looking back at the camera, one sees what looks like the light source with a camera lens at its centre. This minimises the above-mentioned angles between the light source, ball and camera. An arrangement similarto this is shown in PCT/GB84/ 00240.

There is a theorectical possibility of making the trajectory footprint of a retro-reflective ball darker, as opposed to lighter, bythe useofa brightness separation overlay. If the footprint is much brighter than the remainderofthe picture then this difference could be used to key or gate outa portion ofthe picture,thetrajectoryfootprint, and fill in the hole in the picture with black or brown or any other colour.

This was tried and the ball did appear black, but so did other light areas ofthe picture. In practice the squash court is brightly illuminated, as it must be, and the footprint in not the brightest area of the picture. This experiment is mentioned as it introduces the next stage of development being the essence ofthe present invention, according to one aspect, of using a variety of lightfiltration methods to remove light other than that retro-reflected from the ball from an additional image or images in registration with the cameratp produce a keying signal which is used to insert a newly generated image of the trajectory footprint into the colour picture.

Thus the invention provides an apparatus for producing an image of a retro-reflective projectile comprising: (a) Meansto illuminate a projection in afield of view in which the projectile is moving with light of a predetermined wavelength (b) optical meansto receive the lightfromthefield of view including light retro-reflected bytheprojectile from said illuminating means to form an image of the projectiletrajectory.

(c) and means to filter from the lighter received by the optical means light otherthan that produced by the illuminating means to enhance the definition of the projectile trajectory in said image.

The keyed area can have any colour, making the ball appearto be that colour, or it can be com plimentaryto the background orflash in any sequ- ence of colours. The edge can be hard orsoftandthe footprint area can be mixed with the picture of the background scene in any proportion. Most important of all the keying signal can be digitized and memorised and used as thefirst stage in the production of a computer generated image ofthe ball under total software control.Many ofthe analyses used in computer vision can be applied to the trajectory footprints as detected, and used to clean up the footprint signal, and these signals used as inputs to a computer program to generate a newtrajectory footprintto be inserted into the television picture and seen bytheviewer.

The size ofthe ball can be changed but more generally, the red, green and blue images of the picture can also be stored and general image processing performed. The limits to this process would appearto be the limits of the imagination of the software writer. Spurious keying signals could be filtered out by the detection of movement and other software devices. will give two examples ofthe sort of pictures that can be produced with the system. In golf, real time, wide angle pictures ofthe complete trajectory of the ball, from being struck to coming to a halt can be produced.In ice hockey, in addition to the puck having anyappearancethatwe please itto have, the position ofthefootprintcould initiate the production of a brightened area in the picture of the scene surrounding the footprint The puck and its surrounds would thus always appear two be in a spotlight.

The digitised footprint can be used to analyse the trajectory ofthe ball and a computer can be programmed to determine, for example, if a retro-reflective tennis ball bounced in or out of court. From the two functions of image processing and trajectory analysis I have named my invention the Trajectory Analysis and Image Processing (TAIP) camera or attachment, if it attaches to a conventional camera.

The contrast (c) of the external keying can be improved by four means that can be used simultaneously or in any combination: 1. The use of monochromatic lightwhich term is intended to include ultra-violet and infra-red wavelengths and narrow band-pass filters or of broad band infra red or ultra violet light and suitable filters.

2. The use of shutters or shutter means to shorten the exposure time.

3. The use of very fast shutters combined with a synchronised, pulsed projectile illuminator, such as a pulsed laser.

4. The use of polarizing filters. This iast method can also employ two additional images and a different keying system based on enhancement by means of subtraction of the signals.

We have seenthatthe production of a usable internal keying signal was not practical because sufficient contrast (c) was not possible owing to the brightness of the background (g). We can reduce the background brightness "visible" to an external keying system to minute levels bythe use of monochromatic projectile illuminators and com plimentary filters in front ofthe CCD array or tube which will only allow that wavelength to pass, thus filtering out most of the background light Similarly one could use a broad spectrum infra red or ultra violet projectile illuminator and suitable filters, in an environmentfrom which those wavelengthswere removed or not present to begin with. Broad spectrum infra red is suitable in indoor arenas lit by fluorescent tubes or high pressure mercury lamps both of which produce very little infrared radiation.

The most successful TAIP pictures recorded to date were produced in such an environement. Visible monochromatic light can produce a high contrast keying signal without being visible to the red, green or blue tubes of the television camera. This is because it does not need to be very strong, and the specific narrow band in question is filtered from the colour camera bya narrow band reflector.

A shutter or shutter means improves trajectory contrast in inverse proportion to shutter time, as we can seefromtheformula. Using highershutter speeds is the classical way to photograph rapidly moving objects. The footprint length is correspon dingly shortened by this means and thus discon tinuous. Normal television trajectories are con tinuous, heel to toe, and this appearsto be the most satisfactory form. Recently I produced footprints that were double this length, using my system. This made the ball appearcigarshaped when viewed at normal speed. In a normal shuttered movie camera, of 1/50 second or so, trajectories are discontinuous and less satisfactory. A rapid shutterwould produce an even worse appearance.The rapid shutter speed, however, produces a more well defined keying signal and using this it is a comparatively simple task to generate continuous footprints, or any other form one wishes, using a computer image generation system.

A ball travelling at 200 miles per hour moves a distance of approximately .35 inches in 1/10,000 second. Usingshutterspeedofl/10,O00second,for all ball games, vt (sin a) - d is negative and c = (b g)/g. Atfirst glance it might appearthatthere is no advantageto be obtained byfastershutterspeeds.

Consider, however, the case of using a pulsed laser as the projectile illuminator, or a pulsed broad spectrum illuminator such as an array of infra red light emitting diodes. Laser pulses can be extremely short duration of even lessthan a picosecond. 100 to 200 nanosecond pulses are routinely used. A laser with a modest average overall power output (powerex- pressed as a continuous wave) can have an extremely high peak power. The shorter the pulse the greater the ratio between peak power and overall average power. Consider a shutter synchronised with the laser pulse. The brightness ofthe ball (b) is prop ortionaltothe peak laser power whereas the bright ess ofthe background (g) is constant (as long as gi is small with respectto gn).Until this limit is reached, and at shutter speeds that render the ball stationary, or greater, the contrast between the ball and the general background, for a pulsed projectile illuminator of any given overall average power, synchronous with the shutter, is directly proportional to the shutter speed (and inversely proportional to shutter and pulse time).

Lasers such as YAG (Yttrium/Aluminium/Garnet) laser are bulky but could be used as a projectile illuminator. More promising are the semi-conducting diode lasers.

These are smaller, more robust, cheaper, simplerto operate than other type of gas and solid-state lasers.

They are a refinement ofthe light-emitting diode (semi-conductor) and are sometimes considered to havethe same relationshipto the more conventional larger lasers as thetransistorhasto the vacuum tube.

Output wavelengths range from 770 to 1,550 nm.

They are also naturally divergent, which is an advantage for a projectile illuminator. A typical off-the-shelf high power laser diode array can have a peak output of 100 watts, a pulse width of 200 nano-seconds, wavelength of 900 nanometers and a maximum pulse repetition frequency of 5000 pulses per second.

In general the projectile illuminatorwill coverthe same field ofview as the camera. Where exceptional power of illumination is required, such as in golfwith the ball at a distance, the beam can be narrowed and concentrated, and the ball which will appearto move slowly, tracked manually, if necessary, by the beam.

Furthermore, a secondary projectile illuminator, situated up to about a metre or so from the camera, with a narrow beam, can be used to boost the signal from a golf ball from a distance.

Shuttered cameras fall into two main groups, mechanically shuttered cameras or electronically shuttered, usually by using an image intensifierto gate the incoming light or by using a CCD sensor that permits variable integration time. Mechanically shuttered cameras have minimum exposuretimestypi- cally of 100 microseconds while the electronically shuttered cameras can achieve exposures of less than one microsecond.

Consider a pulsed laserwith a pulse length of less than a microsecond and a pulse repetition frequency (prf) of 5,000 pulses per second. Ov.er the period of a television frame, 1/50 second,thiswill produce 100 images ofthe ball and a continuous trajectory footprint, produced by the overlapping individual exposures, whilst retaining the advantages of shuttering. Widely spaced shuttered footprints will not always sample enough information to determine if a ball passes in front of, or behind, a player and therefore if gaps or discontinuities need to be iserted in the train of generated trajectory footprints.

Light can be reflected in two ways specularly or diffuselyorin a combination ofthese. Specular reflections retain the polarity of polarized light.

Retroreflective balls retain the polarity of all polarized light, linear (plane) circular and elliptical. Upon retroreflection the sense of circularly and elliptically polarized light is reversed. Many, if not all ofthe subjects in the scene tht reflect excessive light from the projectile illuminator, usually because they are nearthe camera, reflect diffusely. If the projectile illuminator is linearly polarized then these objects produce a depolarized reflection. A linear polarizer positioned as shown in Figure 3 referred to below and adjusted so asto pass all the polarized light from the trajectory footprintwould filter outabout half the polarized light reflected from these foreground diffusely reflecting objects. This method improves the contrast ratio, with respect to projectile illuminator light, by about 100%.

If a second image detecting system were introduced, in all respects similarto the first additional system, as shown in Figure 4 but with its polarising filter at right angles to thefirstfilter, then this would filter out the trajectory footprint image byt leave the diffusely reflecting foreground unaltered. By subtracting the two signals one obtains a strong signal of the trajectory footprint and eliminates the signal produced by diffusely reflecting objects. This subtracted signal can be used for keying.

The dichroic mirrorthat deflects the projectile illuminatorwavelengths, and is positioned at45 degrees to the main lens components rotated about a horizontal axis, can have a detrimental effect on the polarized reflected image. Experiments showed that using polarized light whose plane is either vertical or horizontal minimisesthis. Reducing the angle of incidence ofthe light to the dichroic mirror by changing the angle ofthe mirror,to a similar angle to the dichroic surface of the prism block in Figure 6, also helps.

Any form of polarized light could be used asthe projectile illuminator. Any form of polarized light can be transformed to any other by the suitable use of retarders. It fol I ows that some optical systems embodying the invention that also include retarders, may take the polarized light reflected from the ball and transform itto some other form prior to filtration and detection.Whateveritsform, however, every form of polarized light has its orthogonal form, for which a pair of mutually excluding filters can be constructed. The two polarising filters in the invention, in the general case filter out each of an orthogonal pair of polarized forms, and in the particular case described are plane polarizing filters at rightanglesto each other.

The ability ofthe present invention to isolatethe trajectory outline from the rest ofthe picture, either before orafterdigitisation, allows the automatic calculation of trajectory featu res that are of interest in various sports. The output from such a system is information and a determination derived from the trajectory analysis, such as whether or notatennis ball bounced in or out of court, ratherthan a pictorial representation. The colour camera can be dispensed with and a simplified system using one image will suffice. Here too an additional image can be used to improve the definition of the trajectory footprint.The essence of thins system is the definition of the position ofthetrajectory of a retro-reflective ball, by the use of previously described techniques, within an image, which also contains a sufficient number of markers, preferably retro reflective, to which the trajectory is related in space and from which calculations will be made.

if two cameras, angled to each other, each contain ing the same scene with sufficient retro-reflective markers, a minimum ofthree,then a complete trajectory analysis can be obtained. With a little ingenuity, however, one camera can often givethe necessary information as I will illustrate by describing a system that determines if a tennis ball bounces in or outofcourt.

Consider a tennis court with small retro-reflective markers (patches) atthe intersection points of all the lines on the court. These markers can be isolated by the invention and a computer can join the markers in a graphic display, illustrating that the computer contains information of the layout of the court. The prespective projection ofthe trajectoryfootprint is also clearly related to the court layout.

Atennis ball in flight is subject two small steady forces such as gravity and aerodynamic forces and thus undergoes similar steady acceleration. The perspective projection (image) ofthe trajectory retainsthis characteristic. Given three successive positions, or points, of the ball in flight one can predict the fourth with some accuracy, simply from the two dimensional coordinates ofthe ball. We assumethatthefourth point lies on a circle (or mathematically similar) defined by thefirstthree points. Its position on the arcofthe circle can be predicted by similar considerations.When the ball bounces or is struck it is subjectto a large force of short duration and undergoes a rapid acceleration and its position no longer lies within the predicted limits.

A computer program was written to test the ability to identify bounce points in a one camera two dimension situation. Assuming that the foursucces- sive ball coordinates satisfy the two parabolical equations, expressed as a function of time (T), in the following from: x=aT2+ bT+candy=aT2+ bT+c The given three successive ball coordinates (x1, y1), (x2, y2) and (x3, y3) it can be shown thatthe predicted coordinates ofthe fourth position are (x1 3x2 + 3x3,y1 3y2 + 3y3).

The coordinates ofthe "toes" of the trajectory footprints of successive frames from a recorded tennis match, using normal non-retro-reflective balls, between two players were plotted manually on graph paperandfed into a computer. The computer was programmed to identifytheframefollowingthe bounce point by drawing a small circle around the plotted point representing the ball position in that frame. The computer identified the bounce points from the information from a single camera, as predicted.

When a game such as tennis is simultaneously being televised and refereed with retro-reflective balls it is necessarythatthe camera should not identify the retro-reflective markers as bal Is. This can be avoided bythefollowing methods: 1) Place the televising camera in a different position to the refereeing camera and construct the retroreflective markers so that they only retroreflectto the refereeing camera. Making them more like specular reflectorthan general retrpreflectorswill do this.

2) Have different wavelength projectile illuminators for televising and refereeing cameras and placing appropriate filters over the markers.

3) By masking out the markers electronically in the keying system.

4) By identifying the markers by software control.

They are stationary and have a geometrical relationship to each other. This is sufficientto write a programme to identify them, just as a bounce point of a ball was identified by its mathematical peculiarities.

The following is a description of somespecific embodiments of the invention, reference being made totheaccompanying drawings in which: Figure lisa block diagram illustrating how additional images are used to precess a television image in various camera arrangements; Figure 2 is a diagrammaticview of a colour television camera having a supplemental television unit for delecting a trajectory footprint of a retro reflective projectile; Figure 3 is a block diagram representing the essential components of the set-up of Figure2; Figures 4to 7 are diagrammatic views of further systems embodying differentforms ofthe invention; Figure 8 is a diagrammatic view ofatennis court with a ball and its trajectory footprint superimposed thereon;; Figure 9 is a diagrammatic view of an optical system for "refereeing" a ball game embodying the present invention; and Figures 1 Oa, 1 Ob and 1 Oc are computer print-outs illustrating the path of a ball in play in a tennis game obtainable from the apparatus of Figure 9.

Referring firstly to the block diagram of Figure 1, there are illustrated the essential elements of the optical arrangements to be described in greater detail below which comprise a colour picture and a filtered, in wavelength, time and polarization, image of the trajectoryfootprint (A) plus, in some cases, an identical second image to (A) with its trajectory footprintdesignaled (B) filtered out. (B) contains residual unfiltered and unwanted light from (A). From these signals a new image is generated and mixed with the colour picture from the camera.

When two images (A) and (B) are used, these must firstly be subtracted in order to remove the unwanted components from (A). When only one sigal is being used, then this can go directly into a keying system.

Alternatively one could by-pass the simple analogue keying system and go directly into a digital system and perform the same operations and more complicated operations digitally.

Using either method an electrically generated image of a trajectory footprint is produced and inserted in the correct position bythe mixer. A colouriser is necessary to a simple keying system to give the keyed area the necessary colours one wishes the ball to have.

Other input could be the match score or other graphic information, but more interestingly, is squash, for example, spectators could be televised with aseparatecamera and inlaid intowhatwould otherwise be a blank wall or other uninteresting part ofthe picture. In Squash, the spectators see the players on the court through the particularly opaque glass walls defining the sides and ends ofthe court.

The players and the T.V. audience seethe courtonly.

Bythis meanstheT.V. audience could seethe players and spectators as in tennis.

Digitization and image storage are necessaryfor more complicated effects.

Referring now to Figure 2 ofthe drawings, there is shown a diagrammaticview of a colourtelevision a camera 10 of a type suitablefortelevising a ball game such a squash ortennis.The camera has a zoom lens 11 directed art a field lens 12 and a relay lens 13 in the optical path to the camera. The ball to be used in the game is provided with retro-reflective elements in its surface as described in International PatentApplication No. PCT/GB84/00240. A "monochromatic" or narrow band pulsed infra-red projectile illuminator, for example a laser source 14is disposed alongside the zoom lens and faces generally along the optical axis ofthe lens and closely parallel thereto. The laser has a plorizing filter 23.Lightfrom the laser isthus retro-reflected from the ball to the zoom lens to form a real image atthe field lens 12. The visible wavelengths from the general scene also pass through the lens 11, relay lens 12 and dichloric mirror 15 into the television camera to form part of the general scene televised by the camera 10.

The dichroic mirror 1 which reflects infra-red wavelengths but transmits the visible spectrum (i.e. a hot mirror) is disposed in the optical axis ofthe camera between the field lens and relay lens at 45 degrees to the axis. Infra red wavelengths are reflected from the dichrnicmirrnrthrnugh a narrow band pass filter 16 disposed to one side of the optical axis of the television camera and into a supplemental television unit indicated generally at 17.The sup plemental television unit contains a front silvered mirror 18 angled at45 degrees to the path from the filter 16to direct light from the filter along a path through a relay lens pair 19 separated by a polarizing filter 20 and into an infra red sensitive camera 21 or a pickup device either ofwhich can be coupled to an image intensifier which can be gated electronically to give exposure times of less than a micro-second.The essential element ofthe above system are illustrated in block diagram form in Figure 3.

The above system may be modified in a number of ways. Forexamplethe illuminator may be a source of visible monochomatic light, pulsed or non-pulsed in which case the dichroic or hot mirror 15 is replaced by a dichroic mirror reflecting the appropriate narrow band of visible radiation.

Further instead of gating the intensifier, a shutter device 22 may be disposed adjacent to filter 16 to shorten and clarify the trajectoryfootprint as received in camera 21.

Figure 4 is a diagrammatic view of a development ofthe apparatus as shown in Figure 2 and like parts have been allotted the same reference numerals. In the arrangement of Figure 4, the supplemental television unit embodies two television cameras or pickup devices either of which can be coupled to an image intensifier with our without gating.Infra-red wavelengths reflected by the dichroic mirror 15 towards the supplemental television unit pass through a field lens pair 30 having a narrow bandpass filter 16 between them and thence into the sup plementaltelevision unitl7.Abeamsplitter3l is mounted in the television unit to direct one beam through a relay lens pair 19 spaced apart by a polarizing filter20 into a first camera 21 a and infra-red passing through the beam splitter are reflected by a silvered mirror 18 through a second relay lens pair 19 and into a second camera 21 b. The polarizing filters 23 and 20a are so arranged that infra-red retro-reflected from the ball passes through 20a. Polarizing filter 20b is so arranged that it stops completely the light retro-reflected from the ball. The function of the two camera system is set out above in relation to Figure 1 and in the preamble.

Thetwo4elevision supplement unit of Figure 4 can, equally, be applied to the system of Figure 2 and, likewise the single camera system of Figure 2 can be applied to the arrangement of Figure 4.

FigureS is a generally similar arrangement to that ofFigure4exceptthatthe dichroic mirror iSis moved forwards into the zoom lens in front of some of the stationary zoom elements and those zoom elements behind the mirror are repeated in the transverse optical path at 42.

Two convenient places to locate the dich roic mirror in suitable zoom lenses are the positions occupied by the mirrorofa built-in pattern projector, or the space occupied by a built-in extender. The monochromatic polarized projectile illuminator 14, illuminates the ball, as before. The zoom has all of its movable elements 40 (focusing, variator and compensator groups) before the dichroic reflector 15, and at least some of its stationary optical elements (the relay group 41), behind the dichroic mirror. The optical element42, is equivalent to the stationary elements behindthedichroicmirror(rearrelaycomponent) and focuses a real image atthe field lens 12. The reminder ofthe system, 17, can be either as shown in Figure 2 or Figure 4.

Figure6 isa diagrammaticviewofa colour television camera with a four-way beam splitting prism block53,oneimageofwhich isusedforthe detection of the ball trajectory footprint and the others of which are used for the normal red, green and blue images of a colourtelevision camera picture. The monochromatic polarized projectile illuminator, (preferably a pulsed laser source 14), is retro-reflected from the ball to the zoom lens 11 to the prism block. The dichroic reflector 15, separates the visible from the infra red wavelengths.Dichroic filters 50, separate the red, green and blue, which are trimmed by filters S1.The narrow band-passfilter and polarizing filter and shutter are situated at 52, befog the infra red pick-up device or infra red pick-up device coupled to an image intensifier.

Figure7 isa diagrammaticviewofa colour television camera 10with its own lens optically linked in registration to a camera or pick-up device sensitive to infra red wavelengths 60, via dichroic mirror 61 thar reflects infra-red wavelengths. The infra red lamp, 62, is reflected from the mirror 63. The bandpass filter 16 in this instance has a broad pass band to correspond to the output of the projectile illuminator.

Figure 8 is a diagrammaticview ofthe image of the retro-reflective ball trajectory and pick-up device and its relationship to a tennis court. (Not drawn to scale).

The image is shown for oneframe ortwo fields. The sizeoftheimage of a stationary ball 1 is related to the theoretical footprint of the same thickness as the ball diameter 2. The actual electronically detected foot- print3maybe narrowerthanthetheoreticalfoot- print, forexample, when a certain threshold illumina tion strength is required before electronic detection.

It can also be wider, for example, in the case of a CCD detector exhibiting the phenomenon of halation.

When a shutter is employed the footprint is much shorter 5. Whatever the thickness ofthe detected footprint, it is, nevertheless, a function ofthe ball's actual size and perspective factors, and can be used as input datato determinethe size ofthe generated ball image, which can initially be selected by the user, and which can thereafter be madeto show realistic changes in size as distance from the camera varies.

The thickness ofthe trajectory footprint generated in the processed television image determinestheview- ers subjective impression of the ball's size. When a shutter is used the trajectory footprint generated will need to be made longerthan the footprint detected for a satisfactorysubjectiveeffect Figure9 isa diagrammatic view ofan optical system suitable for determining if a tennis ball bounces in or out of court. Lightfromthe monochromatic projectile illuminator, preferably a pulsed infra red laser source 70 goes through polarizing filter 71, is retro-reflected from the ball, anc passes th rough narrow pass band filter 72 and polarizing filter 73, to the zoom lens 74.The image of the trajectory and markers is detected by a pick-up device coupled to an image intensifier, synchronously gated to the pulses of the laser Figure 10(a) is a computer print-out illustrating the tennis refereeing program in action. The first printout is ofthe "x" and "y" coordinates ofthe "toes" trajectory footprints of 74 successive frames from a recorded tennis match. When the ball is returned the apparent greatest height, before the bounce forms a sharp point, similar in appearance, but inverted to a bounce print. Figure 10(b) is a print-out illustrating the computer identifying the frame afterthe bounce point and illustrating this by drawing a circle around this point. Figure 10(c) is a print-outofa graph ofthe Y coordinates plotted against frame number. The "sharp point", when plotted in time, is seen not to be a bounce point but a smooth curve which the computer identifies as such.

As indicated earlierthe various systems described above may utilize broad spectrum light in eitherthe infra-red or ultra violet regions of the spectrum, narrowspectrum light in the infra-red, visible or ultra-violet regions all of which may be polarized or unpolarized, pulsed or unpulsed, and the supplemental television can be either shuttered or unshuttered. There is however no advantage in using pulsed illunator in an unshuttered system.

Claims (26)

1. An apparatusfor producing an image a retro-reflective projectile comprising: (a) means to illuminate a projectile in a field of view in which the projectile is moving with light of a predetermined wavelength (b) optical means to receive the light from the field of view including light retro-reflected by the projectile from said illuminating means to form an imageofthe projectile trajectory (c) and means to filter from the light received by the optical means light other than that produced by the illuminating means to enhance the definition of the projectiletrajectoryin said image.

2. An apparatus as claimed in Claim 1 wherein, the illuminating means produces monochromatic light in the infra-red, visible or ultra-violet regions of the electromagnetic spectrum and said filter means is adapted to pass that light and to exclude other light.

3. An apparatus as claimed in Claim 2 wherein the monochromatic light is visible light and said filter means is a narrow band pass filter to pass only visible lightofthewavelength produced bythe illuminating means.

4. An apparatus as claimed in Claim 2, wherein the monochromatic light produced by the illuminat- ing means is infra-red or ultra-violet light and the filter means is a broad infra-red or ultra-violet passband filter.

5. An apparatus as claimed in any of Claims 1 to 4 wherein the filter means includes shutter means to shorten the exposure time ofthe light passed by the optical means.

6. An apparatus as claimed in Claim 5 wherein the projectile illuminating means produced pulsed light and the shutter means is a high speed shutter synchronized with the illuminating means to pass the light pulses produced by the illuminating means and retro-reflected by the projectile.

7. An apparatus as claimed in any of Claims 2 to 6 wherein the illuminating means comprises a laser.

8. An apparatus as claimed in any ofthe preced ingclaimswhereintheilluminating means produces a polarized light and the filter means is a polarized filteradaptedto pass onlythe polarized light retroreflected from the projectile.

9. An apparatus as claimed in any of the preceding claims wherein two optical means are provided one having a polarizing filter means to restrict lightto that oriented in one direction and another optical means having a polarizing further means to restrict lightto that oriented orthogonallyto said one direction and means are provided for subtracting the resulting signals to produce an enhanced image of the projectile trajectory.

10. An apparatus as claimed in anyofthe preceding claims wherein means are provided to synthesise a trajectory footprint of any required size, colour orform derived from thetrajectoryfootprint information received in said optical means and located in afield ofviewin accordancewiththe position of the trajectory footprint in the field of view ofthe optical means.

11. An apparatus as claimed in any of the preceding claims wherein said optical means in cludes at least one image pick-up device for converting the said optical trajectory footprints into electronic information.

12. An apparatus as claimed in Claim 11 wherein means are provided for synthesisting electronically an image of a required colour and size and form corresponding in location to the trajectory footprint received in the field of view ofthe pick-up device.

13. An apparatus as claimed in Claim 11 or Claim 12 wherein a colourtelevision camera is provided for observing the field of view concurrently with the pick-up device or devices and means are provided for mixing the output ofthe pick-up device or devices of the projectile trajectory with the field of view ofthe colourtelevision camera to create an image ofthe field of view with an enhanced projectile trajectory.

14. An apparatus as claimed in Claim 13wherein means are provided for varying the colour of the projectile trajectory footprint to contrast with the background ofthefield of view as the projectile moves around the field of view..

15.. An apparatus ascaimed in Claim l3orClaim 14wherein said television camera has an optical axis to be aligned with the field of view and an optical device in said optical path for passing light from the field of view into the camera and also reflecting light to one side of the optical path, said pick-up device or devices being disposed to one side of the optical path to receive reflected light.

16. An apparatus as claimed in Claim 15 and in the case where the illuminating means produces monochromatic visible lightwherein the optical device is a dichroic mirror adapted to reflect that wave range of visible light.

17. An apparatus as claimed in Claim 15 and in the case where the light is infra-red or ultra-violet wherein the optical device transmits visible light directly into the camera and reflects infra-red or ultra-violet lightto one side of the said optical path into the pick-up device or devices.

18. An apparatus as claimed in any of Claims 1 to 9wherein stationary reflectors are provided at spaced positions around boundary lines of a courtonwhich a game is to be played and facing said illuminating means and a said optical means receives both light from the projectile and lightfrom the stationary reflections and has a pick-up device incorporating means to interpret rapid accelerations as occurs during bouncing of the projectile atthe boundaries of the court determined by the stationary reflections and to determine whether changes of acceleration at the boundaries indicatewhetherthe projectile also bounced in or out of court.

19. An apparatus as claimed in Claim 18 wherein means are provided for producing an outputfrom the pick-up device depicting the trajectory ofthe projectile and highlighting any bounce in the path ofthe projectile in relation to the boundaries ofthe court and indicating whetherthe bounce is in or out ofthe court.

20. An apparatus as claimed in Claim 19wherein the pick-up device includes means for plotting the path ofthe projectile.

21. An apparatus as claimed in any of Claims 18to 20 wherein means are provided for storing a plurality of consecutive positions of a projectile and predicting a subsequent position and for indicating that a bounce has occurred ifthe subsequent actual position does not conform to the predicted subsequent position.

22. A method of televising a game using apparatus as claimed in anyof Claims 13to 17.

23. A method of refereeing a game using apparatus as claimed in any of Claims 18 to 21.

24. An apparatus for televising a game substantially as hereinbefore described with reference to and as illustrated in the accompanying drawings.

25. A method oftelevising a game substantially as hereinbefore described.

26. A method of refereeing a game substantially as hereinbefore described.