AU2006100154A4 - Recording contacts during sporting activities etc - Google Patents

Description

1 01/83/2886 22:30 81-3-98981337 PATENT ATTORNEY SERV PAGE 03/29 This invention relat location) of a contact such The invention has b bounce point of a tennis bal objective determination wh of tennis or whether the poi invention, however, is not L invention can be used in cri cricket bat before being cau batsman has been dismissed useable with other sporting, parameters of a contact can such activities and such con 0 in the sport o opponent's body may consti in the sport o allowable parts of the body and other parts of the opponi boxing; in the sport o is required for a legitimate st in the sport ol ball bounces, the natu-e of ec s to determining the existence and/or a property (such as the s a balI bounce in a sporting activity or the like.

-en particularly developed for deterimining the location of the in relation. to the court line markings so as to enable ether the preceding tennis stroke has complied with the rules it of bounce is outside of the relevant court lines. The mited to this particular field of application. For example the ,ket to determine if the ball has contacted the edge of the ght by a fieldsman and hence to determine objectively if the under the rules of cricket. The invention may also be game, recreational, or entertainment activities where ,e relevant to some aspect of the activity. Examples only of acts may include: F basketball where contact of a player with a part of an ute a foul under the rules of basketball; "boxing a contact involving the gloved fist of a boxer against f the opponent may score points for the boxer, or contacts ,nt's body may incur penalties or warnings under the rules of volleyball, simultaneous contact of the ball with two hands rike of the ball under the rules of volleyball; tennis, apart from legality of a shot depending on where the ntact of the ball with the racket strings, e.g. to impart COMS ID No: SBMI-02837423 Received by IP Australia: Time 22:33 Date 2006-03-01 01/03/2006 22:30 61-3-98901337 PATENT ATTORNEY SERV PAGE 04/29 particular spin to the ball, T player; in the recrea gymnastics, the locations o desirably monitored particu in motor car of the vehicle tyres with the indicative of the perfonnan4 monitoring be useful partici training or practice of the dT Although the inventi to detennrmining parameters o it is to be understood that th, and, as outlined above, can 1 entertainment activities.

The determination of line markings or boundaries legal or is illegal or "out" ac is a skilled exercise and in pi persons located around the tc the shots legal or "out". The concerning the validity of cal To help alleviate suet used in the past. These have lay be desirably monitored particularly in coaching of a tennis tion or entertainment of dancing, or in the sport of Sfoot falls on the arena surface of the participants may be larly in training or coaching of such participants; or motor cycle racing, the nature and location of the contact road or track surface may be desirably monitored as e of the vehicle or behaviour of the driver/rider and this may larly in adjusting the technical features of the vehicle or in iver/rider.

an will be particularly described herein primarily in relation rthe bounce of a tennis ball occurring within a tennis game, invention is not limited to this particular field of application 'c applicable to other sporting, game, recreational, and the point at which a tennis ball bounces in relation to the :f the tennis court determines whether the stroke is called :ording to the rules of tennis. Such calling of bounce points ofessional tournaments a central umpire and many lines nnis court continuously monitor the bounce points to call -e is much dissension evident in tennis tournaments Is made by the umpire and lines persons.

dissension some electronic aids have been proposed or 3een based on one of three principles: 2 COMS ID No: SBMI-02837423 Received by IP Australia: Time 22:33 Date 2006-03-01 01/03/2006 22:30 61-3-98901337 PATENT ATTORNEV SERV PAGE 05/29 Impact sense the ground, the sensors at ti monitored to determine obji court boundary lines.

Light beams in the vicinity of the bound interrupts the beams, the eX interrupted beams and hene( is within or outside of the cc used in some tournaments a Images of the located at known positions a images including images of trajectories of the tennis ball within the computed trajecto including location of any adi viewer of the display simulat outside the court boundary li in tournaments and is known These systems can ha systems and installations thel calibrating or testing and ope applicability throughout a ten many impacts on the court su apparatus and, unless accurate ,rs placed within the court surface so that when a ball strikes at location record the impact and the resulting signals can be tively whether the bounce point was within or outside the have been projected along and transverse to the tennis co-urt ry lines so that when the tennis ball strikes the surface and i.ct location of the bounce point can be determined from thile San objective detennrmination made whether the bounce point urt boundary lines an apparatus based on this principle is id is known by the trade mark "Cyclops".

tennis court and surrounds are captured by multiple cameras round the court and these cameras continuously capture he ball in flight which are then analysed to determnnine enabling computation of bounce points by interpolation ries. A simulation of the ball bouncing on the court surface acent court boundary lines can then be displayed for a ion to determine whether the bounce point was within or oes. A system based on this general principle has been used as the "Hawk Eye" system.

ve one or more limitations such as expense of the hardware eof, the difficulty and precision required in providing rating of such systems at a tennis venue; limitations on nis game since there can be many movemnents of bodies and ,face during a tennis game that can be registered by such ly discriminated, could lead to false indications for 3 COMS ID No: SBMI-02837423 Received by IP Australia: Time 22:33 Date 2006-03-01 01/03/2006 22:30 61-3-98901337 PATENT ATTORNEY SERV PAGE 06/29 example the "Cyclops" sys a tennis player's serve and: It is an object of the determining parameters of least a useful alternative to functions.

According to a first deternining parameters of a particularly within a sportin including the steps of: directing a digital im em is generally used only for determining the bounce point of lot subsequent tennis shots.

present invention to provide methods and apparatus for ontact occurring within an activity and which can provide at )ther systems aimed generally at perfonring similar ispect of the present invention there is provided a method of contact between bodies occurring within an activity, game, recreational or entertainment activity, the method age capture apparatus, such as a digital camera, towards a target area during the course of the activity and capturing thereby image frames of the target area including image comparing image da reference datum so as to rec incident frame, such localise generated by a contact whicl heat, such as by friction betv involved in the contact, analysing the image least one parameter of the co recording the image c selectively displaying including the image data in. tl the scene captured for the inc lata in the infra red region of the spectrum, a for multiple temporally spaced frames with image data of a rd any localised infra red emission appearing in at least one i infra red emission being indicative of heat having been has occurred in the target area and which has generated een the bodies or deformation of at least one of the bodies ata for at least one of the incident frames to determine at itact, ita for the at least one incident frame, and the at least one incident frame by visual display means e IR. spectrum by presenting IR data as a visible display of dent frame so as to enable visual inspection of the activity 4 COMS ID No: SBMI-02837423 Received by IP Australia: Time 22:33 Date 2006-03-01 01/03/2006 22:38 61-3-98901337 PATENT ATTORNEY SERV PAGE 07/29 in the target area at the tim< contact producing the in fira Preferably the imag generate image data for the infra red spectrumn, the imas for the at least one incident incident frame(s) comprises spectrum so that the display target area in the visible par infra red emission is superir part of the spectrum so that visible scene of the target ar for the visible spectrum con visible part of the spectrum, each pixel an infra red craniss infra red region of the spectr localised infra red emission: comprises recording any sigi emission within at least one The step of comparin reference datum may compri for preceding frames of capti data of the reference datum r contact generating the infra r of the at least one incident frame and that gave rise to the ed emission.

capture apparatus, such as the digital camera, is operative to risible region of the spectrum in addition to image data in the e data in the visible part of the spectrum being also recorded iame, and wherein the step of selectively displaying the displaying the image data from the visible part of the ;d image includes a more naturally recognisable scene of the of the spectrum and the image data relating to the localised iposed on the scene in a contrasting display in the visible ,vidence of the contact is readily visible superimposed on the In this embodiment preferably the image data captured Prises for each pixel of each frame net light intensity in the 3uch as a grey scale measure or luminance measure, and for on intensity is separately recorded as the image data in the im, whereby the step of comparing data so as to record any adicative of heat having been generated by a contact ificant increase in the intensity value of the infra red orTesponding pixel of successive frames.

Simage data for multiple frames with image data for a ie comparing the image data with corresponding image data red image data for the same target area so that the image lates to the target area immediately or shortly before the d emission.

COMS ID No: SBMI-02837423 Received by IP Australia: Time 22:33 Date 2006-03-01 01/03/2006 22:30 61-3-98901337 PATENT ATTORNEY SERV PAGE 08/29 The analysing step as being indicative of a leg by analysing parameters of including at least one of: analysing the shape infra red part of the spectra frame, analysing an infra rc as the infra red spectral emi temperature change, S analysing the duratic contiguous pixels, and analysing the locatio red emissions within the tarj In the preferred emb( one of the incident frame cot preceding the incident frame number of successive incidei preceded by displaying irnag frames leading up to and inel In one possible embo by visual display means is ca command.

referably includes verification of the or each incident frame timate event in the activity giving rise to a contact of interest the infi-a red emission as recorded in the infra red image data and/or size of the contact area exhibiting emissions in the 0 and represented by contiguous pixels of the captured image d signature indicative of a contact of the type of interest such ision characteristics indicative of a pre-detnnmined n of decay of the infra red emission in the cluster of Sof the cluster of contiguous pixels having significant infra et area of the activity.

idiment the step of recording the image data for the at least nprises recording the image data for several frames or preceding each incident frame or preceding the first of a it frames, the step of displaying the incident frame(s) being :s from the preceding recorded frames so that a sequence of ading the at least one incident frame is displayed to the user.

liment, the step of displaying the at least one incident frame Tied out in response to user initiation of a display 6 COMS ID No: SBMI-02837423 Received by IP Australia: Time 22:33 Date 2006-03-01 01/03/2006 22:30 61-3-99901337 PATENT ATTORNEY SERV PAGE 09/29 The step of analysii contact may be used to init parameter of the contact.

In a case where the part of the tennis court witi the contact causing a locali the tennis court surface and parameter of the impact of 1 existence of an impact of th determination of the locatio surface in relation to the ten play.

In another case wher comprises at least part of tilt causing Iocalised infra red a bat used by the batsman and parameter of the contact oft] existence of a contact of the According to a secon of determining parameters o1 particularly within a sporting including the steps of: directing a digital still spectrum recording functiona and capturing thereby a singh ig the image data to determine at least one parameter of the iate generation of a signal indicative of a predetermined activity is a tennis game, the target area can comprises at least ,in boundaries defined by outer sidelines and base lines and ;ed infra red emission comprises impact of the tennis ball with the step of analysing the image data to determine at least one be tennis ball comprises at least determination of the tennis ball with the court surface and preferably also i of the contact or impact of the tennis ball with the court nis court line markings delineating the legitimate zones of the activity comprises a cricket game, the target area cricket pitch where a batsman is stationed and the contact nission comprises contact of the cricket ball with tle cricket the step of analysing the image data to determine at least one ie cricket ball comprises at least determination of the ,ricket ball with the cricket bat.

I aspect of the present invention there is provided a method a contact between bodies occurring within an activity, gamne, recreational or entertainment activity, the method camera having both visible spectrum and infra red ity towards a target area during the course of the activity image frame of the target area including both visible 7 COMS ID No: SBMI-02837423 Received by IP Australia: Time 22:33 Date 2006-03-01 01/03/2006 22:30 61-3-98901337 PATENT ATTORNEY SERV PAGE 10/29 spectrum image data. and ii still camera being calibrate emission indicative of heat occurred a short time befor heat by friction between thi in the contact, recording the visibh displaying the fran spectrum by presenting IR superimposed on the visible activity in the target area at producing the infra red emi, According to a third for determining parameters particularly within a sportinj including: a digital image captui during the course of the actin including image data in the i means for comparing image data of a reference dat appearing in at least one incit indicative of heat having beei area and which has generated of at least one of the bodies ir nage data in the infi-a red region of the spectrum, the digital d to record the existence and location of a localised infra red having been generated by a contact between bodies which has Sthe image capture in the target area and which has generated bodies or deformation of at least one of the bodies involved and infra red image data for the frame, and by visual display means including the image data in the IR lata as a visible display of the scene captured for the frame spectrum display so as to enable visual inspection of the :he time of the frame and that gave rise to the contact sion.

ispect of the present invention there is provided an apparatus fa contact between bodies occurring within an activity, game, recreational or entertainment activity, the apparatus e apparatus, such as a digital camera, towards a target area ity and for capturing thereby image frames of the target area ifra red region of the spectrum, image data for multiple temporally spaced frames with im so as to record any localised infra red emission .ent frame, such localised infra red emission being i generated by a contact which has occurred in the target heat, such as by friction between the bodies or deformation volved in the contact, 8 COMS ID No: SBMI-02837423 Received by IP Australia: Time 22:33 Date 2006-03-01 01/03/2006 22:30 61-3-98901337 PATENT ATTORNEY SERV PAGE 11/29 means for analysint determine at least one para means for recordin visual display mean including the image data in the scene captured for the ii in the target area at the time contact producing the infra Possible and preferre with particular reference to that the features illustrated i construed as limiting on the Fig 1 is a schematic present invention.

Fig 2 is a flow chart present invention.

Fig 3 is a schematic, areas of a tennis court.

Fit 4 is a schematic il operation of the present invei Fig 5 is a schematic il invention can be used.

The apparatus in Fig I towards a target area such as where an activity is to'be und Sthie image data for at least one of the incident frames to neter of the contact, the image data for the at least one incident frame, and s for selectively displaying the at least one incident frame the 1R spectrum by presenting IR data as a visible display of icident frame so as to enable visual inspection of the activity of the at least one incident frame and that gave rise to the "ed emissiond features of the present invention will now be described he accompanying drawings. However it is to be understood i and described with reference to the drawings are not to be scope of the invention. In the drawings: icw of an apparatus according to an embodiment of the f possible data processing operations in an apparatus of the iew of cameras placed to capture images of selected target ustration of tennis ball bounces used to illustrate the ition.

[ustration of an incident in a cricket game where the present includes one or more digital cameras 15 directed in use sporting, game, recreational, or entertainment area, or area taken. Camera 15 need not be fixed in location but could 9 COMS ID No: SBMI-02837423 Received by IP Australia: Time 22:33 Date 2006-03-01 01/03/2006 22:30 61-3-98901337 PATENT ATTORNEY SERV PAGE 12/29 be movable by an operator, ai operations of the invention cE area when the particular cont digital "still" camera operate contact of interest occurs. H invention will involve at leas known location and is direct( activity so as to capture ima present invention captures in captured image frames therel data for infra red image detei infra red data may include st locations within the target a may include infra red emissi the camera may be construct predetermined part of the in: from contacts of the kinds o the detected radiation in thai fotbr each image frame, Preferably the earner spectrum so tl at for each pi: but also visible spectrum da or at least, net light intensit) measure or luminance meas apparatus using the visible s d indeed could be hand held since the process and n be performed provided the camera is directed to the target ct incident of interest occurs. The camera could be a Sto capture an image of thie target area shortly after the owever for most purposes the implementation of the Sone digital video camera which in use is mounted in a d towards a respective target area during the course of the e frames of the target area. The camera(s) 15 used in the tage data in the infra red region of the spectrum. The bore comprise sets of pixels each of which includes image -ted by the camera arising from within the target area. The ectral data indicative of the temperature of respective ea giving rise to the infra red emissions (or reflections), or n intensity data, or may include both. If desired however ed or calibrated to record only infra red radiation within a ra red spectrum selected so as to record infra red emissions interest (further discussed below) with only the intensity of part of the T.R. spectrum being recorded as the infra red data as 15 also record image data in the visible part of the el of each image frame there is not only T.R. data recorded .a recorded. The visible spectrum data may comprise only, in the visible part of the spectrum, such as a gey scale ire, so that by generating an image on a visual display pectrum intensity data, a monochrome or grey scale image of COMS ID No: SBMI-02837423 Received by IP Australia: Time 22:33 Date 2006-03-01 01/03/2006 22:30 61-3-98901337 PATENT ATTORNEY SERV PAGE 13/29 the target area is produced fo will be readily understandabl area by viewing the visual di digital video camera capturit image frame captured (in ad( generally conventional and r The image data captu which can be a general purp( data capture, storage, proces to memory 18 where the image for processing and output to with the data processor 17 c: or multiple such display unii keyboard, mouse, joy stick, can include a visual display activity is taking place, eg a the display unit and in parti enable viewing of the tennis The umpire or referee may: captured and stored by the a display 20 can be provided i view and review the recorde Another possibility is that th at a venue of the activity, eg tennis match so that the audi r each recorded image frame. This kind of display of course e and will enable the user to naturally visualise the target splay. However if desired, each camera may be colour g colour data such as RGB values for each pixel of each lition to the IR data). The colour data capture may be eed not be further described.

red by the cameral 15 is fed to the data processing means 17 ,se computer loaded with the required software for image ;ing etc. Associated with the data processing means 17 is data can be stored and from which the data can be recalled displays and the like. A visual display means 20 associated in comprise a conventional visual display unit of a computer s. Other conventional peripherals such as 1/O control 21 (eg will be provided for obvious purposes. The display irovided for example adjacent the target area where the Ijacent a tennis court where an umpire or referee can view ular can view replayed image frames of the target area to ball bounce or other contact of interest in the target area.

iave a control unit 21 to instigate replay of image frames 5paratus for repeated viewing and assessment. Likewise the a a coaching or training situation for a coach or players to i image frames for learning or other analysis purposes.

e display 20 comprises a large scale visual display provided a large scale display provided to an audience viewing a ence can see a replay of image frames of the target area

I

COMS ID No: SBMI-02837423 Received by IP Australia: Time 22:33 Date 2006-03-01 01/03/2006 22:30 61-3-98901337 PATENT ATTORNEY SERV PAGE 14/29 from a preceding incident in he tennis game. Again another type of display 20 can comprise external displays to which the recorded image frames can be transmitted eg by television transmission or ot er transmission medium to reach external displays such as television sets of an audience away from the sporting venue.

Referring to Fig 2, th. functions of the processor 17 can include those illustrated in the flow chart. These processes comprise: Capturing, eg by means of conventional frame grabber means multiple successive image frames fron the outputs of cameras 15, this data including image data in the infra red region of the spectrum and preferably also from the visible spectrum, and storing this data for each framne in memory 18 for later processing. Conventional software can be used for cap iring and storing image data, including if desired some preliminary processing such as filtering and perhaps data enhancement such as magnification oflR data, wl ether universally or when multiple adjacent pixels have enhanced IR data relative to a datum such as a mean TR intensity level.

(26) The image data particularly the IR data of the pixels of successive captured image frames is compared ith a reference datum. The reference datum provides a base enabling detection of any localised infra red emission. Therefore the reference datum may for example comprise: corresponding pi els of an irmmediately preceding image frame, image data from a frame recorded several frames before the subject frame, am average of several preceding frames, or a single calibration frame or an average of several calibration frames captured for the target area captured rior to the "live" recording of image frames during the course of the sporting activity 12 COMS ID No: SBMI-02837423 Received by IP Australia: Time 22:33 Date 2006-03-01 01/03/2006 22:30 61-3-98901337 PATENT ATTORNEY SERV PAGE 15/29 (27) The detection of datum is recorded in the resp there has been a possible or z been generated by a contact' generated heat such as by fii tennis ball contacting the coi involved in the contact.

(28) Preferably there frame as an "incident frame'" associated with the data set f data such as frame number, I readily identify the incident the incident frame.

(29) The image data parameter of the contact.

The most basic parar recorded a contact of the kin verfication of the incident f parameters of the recorded ii eg the size and shape oft] all recording simultaneously frictional contact, the size of tennis ball (which can be ein shape of the imprint being st the long axis in the general c any significant TR emission compared to the reference active frame and is noted as an "incident frame" in which ctual localised infra red emission indicative of heat having vhich has occurred in the target area and which has ;tion between the bodies engaged in the contact (eg the .t surface) or by deformation of at least one of the bodies is generated a record or a marker to indicate the relevant such record or marker comprising for example a data tag or the image frame or perhaps recording in a register other Ine of incident, etc. The record or marker is useable to ?rae. This is to enable later selective recall and display of for the incident frame is analysed to determine at least one ieter to determine is whether the incident frame has indeed I of interest, namely a tennis ball bounce. To enable ame as having recorded a legitimate contact of interest, rfra red emission can be analysed, such parameters including le contact as represented by the contiguous cluster of pixels the notable increase in the temperature indicative of a the contact needing to be cormrnensurate with the size of a pirically determined by test runs of the apparatus) and the bstantially as expected, particularly generally elliptical with irection of travel of the tennis ball. Another parameter of 13 COMS ID No: SBMI-02837423 Received by IP Australia: Time 22:33 Date 2006-03-01 01/03/2006 22:30 61-3-98901337 PATENT ATTORNEY SERV PAGE 16/29 the IR emission enabling ver contact, such signature inclu< area being consistent with a I empirically determined by re be (iii) the duration of decay the infra red emission from a with a hyperbolic IR intensit entailing analysis of multiple the same point are detected.

(29)(continued) The analyse in most fields ofapp' contact. In particular, in the of bounce in relation to the c and uses of the invention, the eg by an umpire viewing the displayed by the unit 20. Th the bounce point in relation t can also provide automated a the exact location of the cont therein, or stored in look-up 1 captured to determine the spi parameters, particularly tenni captured data to discriminate colour to the major part of th analysis software to discrimi fication may be (ii) the infra red signature of the frictional ling the change in temperature at the point within the target Sctional contact of interest (again something which can be jeated tests of the apparatus). Another signature that may of the IR emission for example it may be determined that contact of interest can decay over one or several seconds i decay pattern this type of verification necessarily successive image frames in which infra red emission from 3ther parameter of the contact that will be important to ication of the invention comprises the location of the analysis of the tennis ball bounce, the location of the point )urt boundary lines can be important. In some embodiments location of the contact point can be determined by the user, display 20 when at least one of the incident frames is tt is, the umpire can analyse and determine the location of the boundary lines of the court. However the invention nalysis of the incident frame or frames in determination of ict. For example, the processor 17 may have programmed ables, or may be able to actively analyse image data tial relationship of the contact point to other spatial s court lines. For example, the processor may analyse the tennis line markings which are provided in contrasting Stennis court surface such as by using conventional image Late transitions between areas of different colour or 14 COMS ID No: SBMI-02837423 Received by IP Australia: Time 22:33 Date 2006-03-01 01/03/2006 22:30 61-3-98901337 PATENT ATTORNEY SERV PAGE 17/29 reflectivity (court surface ver discriminated colour transitio scuffed or dirty etc. and henc line markings. This "virtual with the derived location of t whether the bounce point fall bounce point is outside the li outside the boundary lines, t] of that particular parameter c generation of an audible and performed. This can substiti The system can have undergone the verifical and verification so that the i: precision by operators of the (31) The system pro display means 20 to enable rise to the contact producinl image data in the infra red s scene captured for the incid incident by the display 20 b presented by the display. A the visible part of the specti display a naturally recognis relating to the localised infr us marked boundary lines) and to interpolate between is in cases where the line markings may be damaged or derived mathematically accurate delineations of the court mage" of the court line markings can then be superimposed )e bounce point and a determination automatically made s legally on or within the boundary lines or whether the aes, In response to determination of a bounce point falling ie system may be operable to generate the signal indicative fthe contact. The signal may be used to initiate the or visible alarm in the area where the activity is being ite for the call made by a lines person at a tennis tournament.

and desirably does, record for those incident frames which ion and analysis processes the further results of the analysis icident frames can be flagged and selectively recalled with system..

vides for selectively displaying incident frames by the visual visual inspection of the activity in the target area that gave the inf-a red emission. The display will include display of )ectrum which will be presented as a visible display of the nt frame. That is, the IR data will be presented as a visible y generating a false colour or false image intensity to be s mentioned above, the cameras 15 preferably record data in urn and this visible spectrum data is used to generate on the ible scene of the target area.. With this system the image data 1 red emission are superimposed on the natural scene in a COMS ID No: SBMI-02837423 Received by IP Australia: Time 22:33 Date 2006-03-01 01/03/2006 22:30 61-3-98901337 PATENT ATTORNEY SERV PAGE 18/29 contrasting display in the visi colour or enhanced intensity) the natural visible scene of t control of the processor 17 a succeeding the start of the de this will mean a display of fr bouncing thereon leaving the the bounce point. Such a dis image merely also showing t o1 superimposed so as to be vis associated with the display recorded image frames with the case of a disputed line cE displays of replays of the inc via television transmissions From the preceding 3 and 4 can be readily unde desired capture video image processed and recorded. As surface 41 involves some fi hence infra red radiation. T emission characteristics (sp therefore verifiable signatur 42 in relation to the court bc that the bounce point was b, ble part of the spectrum (eg in false colour or enhanced so that the contact event is readily visible superimposed on ie target area. The display 20 may generate under the sequence of several recorded frames preceding and tected TR producing contact. In the case of a ball bounce unes showing the ball moving towards the court surface, IR "imprint", and continuing to travel upwardly away from play is not an artificial simulation of the bounce, it is a real 3e heat imprint of the bounce point enhanced and ble on the natural scene. If desired, the control 21 0 can be operated by a user to play or replay the sequence of :he enhanced infra red bounce imprint only on demand, eg in 11 of a lines person at a tennis tournament. Of course such ident can be made to the audience at the sporting venue, or For enhanced entertainment.

escription the possible use of the invention depicted in Figs .tood. The cameras 15 continuously or intermittently as of target areas of the court, the signals from which are shown in Fig 4, the bounce of the tennis ball 40 on the court ction at the bounce point 42. The friction generates heat and ie area 43 will have a characteristic size, shape and IR ctral and temporal) which provide an analysable and Sof the tennis ball bounce. The location of the bounce point undary lines 45 following a tennis player's serve will show yond the service line and hence the serve was a fault. The 16 COMS ID No: SBMI-02837423 Received by IP Australia: Time 22:33 Date 2006-03-01 01/03/2006 22:30 61-3-98901337 PATENT ATTORNEY SERV PAGE 19/29 bounce point 42a from tennis is outside the sideline. Recor command at display 20 eg by can be automatically analysec In Fig 5 the contact in with edge of the bat 51 and tl 51 can be difficult for an umi there is uncertainty whether I cricket. However with the pi of the bat 51 will be a frictio process and apparatus of the the field of play eg in a studi ready verification of the exis umpire can signal according it will be seen that th means for detecting, and in contacts within sporting and points in relation to the line of real events, not simulatio alternative to known system It is to be unde additions may be made to th invention as herein describe invention.

ball 40a is legal. The bounce point 42b of tennis ball lings of these tennis ball bounces can be replayed on an umpire for manual assessment and call of the legality, or I and the legality signalled automatically.

cident of interest comprises contact of the cricket ball ie point 52. Such a glancing contact with the edge of the bat lire to detect so that if the ball is caught by the fielder he batsman 53 should be dismissed under the laws of esent invention, the glancing contact at point 52 at the edge 1 event causing an infra red emission detectable using the invention. Selective replay by the umpire (who can be off Sas is known for higher level televised cricket) can enable tence of the contact 52 with the bat 51 and that off-field y to the on-field officials.

Sprocess and apparatus of the invention provide a useful ne embodiment automatically analysing and signalling, similar activities, particularly location of tennis ball bounce markings. The system is objective and generates real images is. The system can therefore provide a most useful 3 for similar purposes.

rstood that various alterations, modifications and/or e features of the possible and preferred embodiment(s) of the i without departing from the spirit and scope of the 17 COMS ID No: SBMI-02837423 Received by IP Australia: Time 22:33 Date 2006-03-01

Claims (12)

1. A method of d occurring within an activity, entertainment activity, the me directing a digital ima target area during the course target area including image d comparing image dat, reference datun so as to recc to incident frame, such localise generated by a. contact whict beat, such as by friction betv involved in the contact, analysing the image least one parameter of the cc recording the image selectively displayin including the image data in the scene captured for the in in the target area at the time contact producing the infra

2. A method as such as the digital camera, spectrum in addition to ima part of the spectrum being ion are as follows: ermining parameters of a contact between bodies articularly within a sporting, game, recreational or thod including the steps of: e capture apparatus, such as a digital camera, towards a f the activity and capturing thereby image frames of the ata in the infra red region of the spectrum, Sfor multiple temporally spaced frames with image data of a rd any localised infra red emission appearing in at least one I infra red emission being indicative of heat having been has occurred in the target area and which has generated 'een the bodies or deformation of at least one of the bodies iata for at least one of the incident frames to determine at intact, Iata for the at least one incident frame, and g the at least one incident frame by visual display means :he IR spectrum by presenting IR data as a visible display of cident frame so as to enable visual inspection of the activity of the at least one incident frame and that gave rise to the •ed emission. claimed in claim 1 wherein the image capture apparatus, s operative to generate image data for the visible region of the ge data in the infra red spectrum, the image data in the visible Iso recorded for the at least one incident frame, and wherein 18 COMS ID No: SBMI-02837423 Received by IP Australia: Time 22:33 Date 2006-03-01 01/03/20OG 22:30 61-3-98901337 PATENT ATTORNEY SERV PAGE 21/29 the step of selectively display data from the visible part of t naturally recognisable scene image data relating to the loc contrasting display in the visi readily visible superimposed

3. A method as c visible spectrum comprises f part of the spectrum, such as pixel an infra red emission ir red region of the spectrmn, v localised infra red emission i comprises recording any sigi emission within at least one

4. A method as of comparing image data for comprises comparing the im of captured image data for tI datum relates to the target at infra red emission.

A method as analysing step includes veril legitimate event in the activi of the infra red emission as i ing the incident frame(s) comprises displaying the image ie spectrum so that the displayed image includes a more >fthc target area in the visible part of the spectrum and the ilised infra red emission is superimposed on the scene in a ble part of the spectrum so that evidence of the contact is on the visible scene of the target area. laimed in claim 2 wherein the image data. captured for the )r each pixel of each frame net light intensity in the visible a grey scale measure or luminance measure, and for each tensity is separately recorded as the image data in the infra hereby the step of comparing data so as to record any ndicative of heat having been generated by a contact ificant increase in the intensity value of the infra red orresponding pixel of successive frames. :laimed in any one of the preceding claims wherein the step multiple frames with image data for a reference datum ige data with corresponding image data for preceding frames e same target area so that the image data of the reference ea immediately or shortly before the contact generating the :laimed in any one of the preceding claims wherein the ication of the or each incident frame as being indicative of a Ly giving rise to a contact of interest by analysing parameters ecorded in the infra red image data including at least one of: 19 COMS ID No: SBMI-02837423 Received by IP Australia: Time 22:33 Date 2006-03-01 I 01/03/2006 22:30 61-3-98901337 PATENT ATTORNEY SERV PAGE 22/29 S analysing the the infra red part of the spect: image frame, S analysing an i interest such as the infra red determined temperature char S analysing the contiguous pixels, and analysing the to significant infra red emissior

6. A method as of recording the image data I the image data for several fr: frame or preceding the first displaying the incident fram recorded frames so that a sec incident frame is displayed t

7. A method as of displaying the at least one response to user initiation of

8. A method as of analysing the image data 1 initiate generation of a signa

9. A method as activity is a tennis game, the hape and/or size of the contact area exhibiting emissions in urn and represented by contiguous pixels of the captured nfra red signature indicative of a contact of the type of spectral emission characteristics indicative of a pre- ge, duration of decay of the infra red emission in the cluster of location of the cluster of contiguous pixels having Is within the target area of the activity. :laimed in any one of the preceding claims wherein the step or the at least one of the incident frame comprises recording Lines preceding the incident frame or preceding each incident if a number of successive incident frames, the step of being preceded by displaying images from the preceding uence of frames leading up to and including the at least one 5 the user. :laimed in any one of the preceding claims wherein the step incident frame by visual display means is carried out in a display command. ;laimed in any one of the preceding claims wherein the step o determine at least one parameter of the contact is used to Sindicative of a predetermined parameter of the contact. laimed in any one of the preceding claims wherein the target area comprises at least part of the tennis court within COMS ID No: SBMI-02837423 Received by IP Australia: Time 22:33 Date 2006-03-01 01/03/20OG 22:30 61-3-98901337 PATENT ATTORNEY SERV PAGE 23/29 boundaries defined by outer q localised infi-a red emission c surface and wherein the step parameter of the impact of th existence of an impact of the determination of the location surface in relation to the tenn play.

A method as 1o comprises a cricket game, thi where a batsman is stationed comprises contact of the cric wherein the step of analysing contact of the cricket ball coi the cricket ball with the crick

11. A method of occurring within an activity, entertainment activity, the m directing a digital stil spectrum recording functionm and capturing thereby a sing] spectrum image data and im still camera being calibrated emission indicative of heat h occurred a short time before idelines and base lines and wherein the contact causing a mprises impact of the tennis ball with the tennis court ,f analysing the image data to determine at least one b tennis ball comprises at least determination of the tennis ball with the court surface and preferably also of the contact or impact of the tennis ball with the court is court line markings delineating the legitimate zones of laimed in any one of claims 1 to 8 wherein the activity target area comprises at least part of the cricket pitch and wherein the contact causing localised infra red emission cet ball with the cricket bat used by the batsman and the image data to determine at least one parameter of the nprises at least determination of the existence of a contact of et bat. letermining parameters of a contact between bodies particularly within a sporting, game, recreational or thod including the steps of: Scamera having both visible spectrum and infra red lity towards a target area during the course of the activity e image frame of the target area including both visible ge data in the infra red region of the spectrum, the digital to record the existence and location of a localised infra red wing been generated by a contact between bodies which has hle image capture in the target area and which has generated 21 COMS ID No: SBMI-02837423 Received by IP Australia: Time 22:33 Date 2006-03-01 01/03/2006 22:30 61-3-98901337 PATENT ATTORNEY SERV PAGE 24/29 O O O In 0 0 heat by friction between the I in the contact, recording the visible displaying the frame spectrum by presenting IR d superimposed on the visible activity in the target area at t producing the infra red emis

12. Apparatus for occurring within an activity, entertainment activity, the ai a digital image captu during the course of the acti' including image data in the i means for comparinI image data of a reference da appearing in at least one inc indicative of heat having bei area and which has generate of at least one of the bodies means for analysing determine at least one paran means for recording visual display mean, including the image data in odies or deformation of at least one of the bodies involved md infra red image data for the frame, and )y visual display means including the image data in the IR ta as a visible display of the scene captured for the frame ipectrum display so as to enable visual inspection of the ie time of the frame and that gave rise to the contact lion. determining parameters of a contact between bodies particularly within a sporting, game, recreational or iparatus including: *e apparatus, such as a digital camera, towards a target area lity and for capturing thereby image frames of the target area nfra red region of the spectrum, Simage data for multiple temporally spaced frames with :um so as to record any localised infra red emission dent frame, such localised infra red emission being nm generated by a contact which has occurred in the target I heat, such as by friction between the bodies or defonnation involved in the contact, the image data for at least one of the incident frames to ietcr of the contact, the image data for the at least one incident frame, and for selectively displaying the at least one incident frame :he IR spectrum by presenting IR data as a visible display of 22 COMS ID No: SBMI-02837423 Received by IP Australia: Time 22:33 Date 2006-03-01 I 01/03/2006 22:30 61-3-99901337 PATENT ATTORNEY SERV PAGE 25/29 the scene captured for the inc in the target area at the time contact producing the infra n ident frame so as to enable visual inspection of the activity f the at least one incident frame and that gave rise to the d emission. Dated this 1 st day of March f006 PATENT ATTORNEY SER Attorneys for BRENNAN BROADCAST VICES aROUP PTY LTD 23 COMS ID No: SBMI-02837423 Received by IP Australia: Time 22:33 Date 2006-03-01