AU2015101938A4 - Assessment of sporting incidents - Google Patents

Abstract

The method and apparatus enables accurate determination of the exact location of an incident in a sporting event, particularly Australian Rules Football, relative to a boundary 5 relevant to the application of the rules of the sport. Video image frames in a target zone of interest which includes the boundary are captured and displayed. The presented image for at least one of the image frames is modified in a manner that portions of the image depicting areas to one side of the boundary are visually differentiated from areas to the other side of the boundary. The video camera can be mounted by a goal post so that the images captured include 0 the goal line and the opposite goal post enabling accurate determination of the side of the boundary line where an incident of interest occurs. - 15 Fig. I

Description

ASSESSMENT OF SPORTING INCIDENTS

This invention relates to the presentation of images of sporting events.

In a number of different sports the rules of the sport contain provisions applicable to incidents which occur outside of a boundary. Adjudication by sporting officials during the sporting event sometimes requires difficult assessments whether the incident occurred beyond the boundary or not. The decisions made by officials in these circumstances can have major consequences for outcomes in the sporting event, sometimes even determining the result of the sporting contest. In major sporting events, broadcasters capturing video images of the sporting event can sometimes replay in slow motion the captured images of the incidents enabling commentators and viewers to assess whether the incident occurred beyond the boundary or not. Occasionally the officiating persons or organisations have an agreement with the broadcasters to enable review by the officials of such replays within a short time after the incident thereby enabling the determination whether a particular rule of the sport concerning incidents near the boundary applies or not.

,5 One example of the importance of determining whether an incident occurred beyond a boundary is a situation encountered in basketball. If the ball is travelling through the air beyond the court boundary line, a player can knock the ball back into the field of play provided that player’s foot has not contacted the ground outside of the peripheral boundary line. However if that player’s foot contacts the ground while the player is touching the ball in the action of knocking it back into the field of play, the ball is deemed to be “dead” and is given to a player of the side other than the one which last touched the ball. Therefore the accurate determination of a player’s foot contacting the ground in relation to the basketball court’s boundary while that player is in the act of knocking a ball whilst in the air can have a significant bearing on the game. Another sport in which the location of an incident in relation to a boundary occurs is athletics, particularly some relay events. A runner handing the baton to another must release the baton into the receiver’s hand before a boundary line marked on the ground. Therefore the moment of transfer of the baton to the receiving runner in relation to the boundary of the permissible zone for the baton transfer can be critical to determine the validity of the transfer under the relevant rules of the sport. If the baton is not transferred before the end of the permissible zone delineated by the line, the relay team can be disqualified.

Another sporting event in which the location of an incident in relation to a boundary occurs is soccer. There is a boundary line internal to the field of play or pitch demarcating the “penalty box”. If a foul is committed against a defender inside that boundary, a penalty shot is awarded so the determination of the precise location of the foul in relation the boundary can be critical.

2015101938 14 Dec 2015

Another sporting event where determining the location of an incident in relation to a boundary occurs in Australian Rules Football. The boundary line for the field of play between the goal posts is a straight line marked on the ground between the bases of the posts. If the football passes across this goal line between the goal posts, a goal is not necessarily scored. If 5 the ball has been touched (whether by a player of the attacking side involving contact above the knee or by a player of the defending side involving contact with any part of the body) before it crosses the goal line, a goal (scoring 6 points) is not awarded under the rules of the sport, but merely a “behind” (scoring 1 point). Therefore determining whether the ball has been touched before or after crossing the goal line can determines the applicable score to be awarded under the 0 rules of the sport. Such determinations can be difficult, not only for the goal umpire positioned to make such determinations, but also even during televised broadcasts following review of slow motion replays. The present invention has been particularly developed for use in making determinations of such incidents in Australian Rules Football and it will be convenient to describe the invention in relation to this particular field of use. However the invention is not ,5 limited exclusively to Australian Rules Football uses.

It is an object of the present invention to provide a method and apparatus for enabling accurate determinations of the locations of incidents in sporting events in relation to boundaries affecting the application of rules of the sport.

According to one aspect of the present invention there is provided a method of presenting images of a sporting event enabling determination of the location of an incident in the sporting event in relation to a boundary, the method including the steps of:

(a) capturing video image frames of activities occurring in the sporting event in a target zone of interest which includes a boundary relevant to application of a rule of the sport, (b) presenting by a display the captured images, (c) modifying the presented image for at least one of the image frames depicting the incident in a manner such that portions of the image depicting areas within the target zone to one side of the boundary are visually differentiated from areas to the other side of the boundary, whereby the visual differentiation enables visual assessment of the location of the incident in relation to the boundary.

The step of modifying the presented images may comprise modifying pixel parameters or data for pixels representing parts of the target zone located to one side of the boundary. The

2015101938 14 Dec 2015 modification of pixel parameters or data is preferably a uniformly applied modification for all the pixels representing the parts of the target zone located to one side of the boundary.

The modification of the pixel parameters or data in one possibility comprises applying or altering a transparency value of the pixel data whereby the modified parts of the resulting displayed image visually resembles a curtain or semi-transparent overlay or the like while still enabling visibility of the part of the target zone being presented by those modified pixels.

The modification of the pixel parameters or data in another possibility comprises adjusting at least one colour value for all pixels or at least a substantial number of pixels representing the part of the target zone located to one side of the boundary, e.g. by addition to the red value for 0 the pixels to the selected side of the boundary line whereby the presented images have a red tinge for parts of the image to one side of the boundary line.

Instead of modifying pixel data for a sizable area of the field of view, it is also possible in another embodiment to modify pixels for a line (of discernible width) so as to visually highlight the boundary line in the image frames.

,5 A particularly preferred field of use for a game of Australian Rules Football and the boundary comprises the goal line extending between the two goal posts, and wherein the incident of interest involves the question of whether the football has crossed the goal line.

Preferably in one embodiment, the video image frames are captured by a video camera located in a plane extending vertically and containing the boundary. When the sporting event is 0 a game of Australian Rules Football and a camera is preferably located in or on a goal post (or two cameras are located one on each goal post) so that its field of view includes the other goal post and a major proportion of the goal line extending in a straight line along the ground between the two goal posts, whereby the presented images from that camera depict the goal line constituting the boundary line of which the opposite goal post is a visual continuation and 25 whereby the area to one side of that line is within the field of play and to the other side of that line is outside of the field of play. If this is not possible, a camera located to the side of the field of play, but substantially in the plane defined by the goal line and the two goal posts (or as close as possible to that plane) and extrapolated beyond the field of play to the location of the camera.

Alternatively, or preferably in addition, video image frames are captured by a camera which 30 is not located in a vertical plane in which the boundary line lies but is displaced from that plane so that the camera is directed towards that plane at a shallow acute angle, preferably less than about 20°-30°, and most preferably less than 10°. Preferably the camera displaced at a shallow acute angle from the boundary line plane is operated simultaneously with a camera capturing images of the target zone from a point within the vertical plane containing the boundary line

2015101938 14 Dec 2015 whereby the modified images from the camera in the boundary line plane enable accurate differentiation of the location of the incident relative to the boundary line plane and wherein the modified images of the sporting activity in the target zone displayed from the images captured by the displaced camera enable inspection of the incident itself, and wherein the method further 5 includes modifying the displayed images for at least one frame depicting, or very near to, the incident of interest and preferably either modifying the displayed images for image frames leading up to that incident frame or modifying the displayed images for image frames depicting the activity following the incident frame, whereby the timing of the incident of interest in relation to movements occurring within the sporting activity relative to the boundary can be 0 determined with the visual aid of the modification of the displayed images from the displaced camera.

The method of the invention preferably includes the further step of performing a calibration to predetermine the areas within the captured images that are depicting locations to one side of the boundary line and which is therefore to be visually differentiated, the calibration step ,5 including identifying points of the imaged target zone that define limits of the area lying to one side of the boundary, and storing those identification points to be retrieved and used in performing the modification of the presented images of an actual incident in the sporting event. The calibration step may include capturing and displaying a still image of the target zone (which may be a single frame from a captured video sequence) and which includes at least part of the 0 boundary, and wherein the step of identifying points comprises identifying on the displayed still image points demarcating the boundary and the vertical plane in which the boundary lies and storing those points whereby a line connecting those points demarcates the side of the image representing the field of play and the other side outside of the field of play.

The calibration step may include the step of erecting a screen or other visible aid at the target zone in a calibration operation performed before the sporting activity commences, such as a screen or curtain erected between two goal posts and extending vertically above the goal line extending between the goal posts, the calibration step further including capturing an image of the screen by a camera located at the point where the live camera later capturing image frames for the sporting event is located, followed by recording data for the location of the screen within the captured image, the data being selected from an image of the screen, the lines in the captured image are depicting peripheral edges of the screen, and points within the captured image depicting key points of the edges of the screen, whereby during the subsequent sporting activity the presented images derived from the camera located at the same point as the camera in the calibration step can be modified by superimposing an image of the calibration screen.

2015101938 14 Dec 2015

For attractive and helpful presentation of the image frames to viewers when the method of the invention is in use, the step of modifying the presented image may not be performed instantly but may be a progressive modification of the affected part of the displayed image to simulate a screen or curtain or colour tint progressively fading into (or out of) or moving into the area affected. This can be effected by a suitable image processing software, e.g. to progressively introduce a change in transparency value of the affected pixels, or a progressive change in a colour value of the affected pixels.

According to another aspect of the present invention there is provided a method of adjudicating in a sporting event to make a ruling whether an incident during the sporting event 0 occurred to one side of a boundary line relevant to the application of a rule of the sport, the adjudicating method including inspecting the modified presented images produced by a method according to the first aspect of the invention and, at least partially on the basis of that inspection, providing a ruling concerning the applicable rule of the sport in relation to the incident.

According to another aspect of the present invention there is provided a data set or recording ,5 of picture data for a sequence of video image frames depicting activities occurring in a sporting event and which have been captured and modified by the capturing and modifying steps according to the first aspect of the invention.

According to another aspect of the present invention there is provided a single image depicting an incident in a sporting event and the relationship of that incident to a boundary, the 0 image having been produced by the capturing step (a) and the modifying step (c) according to the first aspect of the invention, and wherein the image is depicting the particular image frame, or a closely adjacent frame, most clearly presenting the time of the incident and the incident’s relationship to the boundary, and that image includes the image modifications to the areas located to one side of the boundary.

Possible and preferred features of the present invention will now be described with particular reference to the accompanying drawings. However it is to be understood that the features illustrated in and described with reference to the drawings are not to be construed as limiting on the scope of the invention. In the drawings:

Fig. 1 depicts an image frame of an incident in a game of Australian Rules football,

Fig. 2 depicts an image frame at the critical incident of the incident generated using a video camera mounted by one of the goal posts,

Fig. 3 depicts an image frame at the same instant as Fig. 2 using a camera displaced from the vertical plane containing the goal line,

2015101938 14 Dec 2015

Fig. 4 schematically illustrates a sequence of frames of an incident being assessed in accordance with the invention.

Fig. 5 schematically illustrates a work station for an operator monitoring the image frames in slow motion replay and initiating modification of the images presented.

Fig. 1 of the drawings represents the presentation of an image frame 10 out of a sequence of image frames captured during a sporting event namely a game of Australian Rules Football. The single image frame can be the frame at the critical instant which, in this case, will be the instant at which the ball 11 contacts the upraised hand 16 of the player 15 who is located between the goal posts 21, 22 and close to the goal line 25. This frame has been captured by a video camera located at an elevated position displaced from the vertical plane containing the goal line 25 which is a conventional location for a televiser’s camera so the image frame 10 is a conventional type that would be familiar to viewers. The determination whether the ball 11 has crossed the vertical plane between the goal posts 21, 22 and containing the goal line 25 before being touched by the player’s hand will determine whether a goal (six points) has been scored and whether a ,5 behind (1 point) is awarded. Judging the exact instant of contact of the ball 11 with the player’s hand is difficult using this image frame and preceding video frames and succeeding video frames because of parallax effects in judging whether the ball has crossed the plane.

Fig. 2 depicts the same instant in the televised football game as Fig. 1 but this image frame 30 is captured by a camera 31 located or mounted in or on goal post 21. Such cameras 31, 32 are 0 schematically illustrated in Fig. 1 mounted by the two posts 21, 22 and directed to have a relatively wide field of view (as shown by the broken lines for the nearer camera) extending from close to the base of the goal post by which each camera is mounted and encompassing preferably at least the height of the opposite goal post. Thus the cameras 31, 32 in the goal posts 21, 22 in Fig. 1 which are capturing video image frames such as shown in Fig. 2, are capturing images in the target zone of interest, namely the vicinity on both sides of the vertical plane defined between the goal posts 21, 22 and containing the goal line 25.

The video images captured and stored by each goal post camera 31, 32 can be later replayed as a slow motion replay of the incident. Frames preceding the one shown in Fig. 2 will include frames in which the ball 11 is moving from the side of the field of play including the ball totally 30 within the field of play, and in the process of crossing the plane (such as shown in image (a) of Fig. 4) and after having crossed the plane (images (b), (c), (d) of Fig. 4).

In viewing the replay of recorded video images from the goal post camera 31, an operator can watch the successive frames presented by a display 42 in slow motion replay and determine when the ball 11 has passed the vertical plane containing the boundary or goal line 25 and (if a

2015101938 14 Dec 2015 clear view is provided from the particular camera angle), the instant at which the ball 11 and player’s hand 16 touch. This is the critical instant of the touch at which the determination is made whether the ball 11 has crossed the plane and therefore whether a goal is scored or not. Upon determining the instant that the touch occurs, the operator monitoring the display 42 initiates a process of modification of the presented image so that portions of the image depicting areas within the target zone to one side of the goal line 25 are visually differentiated from areas to the other side. Fig. 4(b) depicts the frame at which the touch of the ball 11 first occurs so that the operator determines that this frame depicts the critical instant. Fig. 4(c) depicts the same frame as Fig. 4(b) but now is incorporating the modification of the presented image thereby visually differentiating, in this illustrated example, the area to the side of the plane containing the goal line 25 that is within the field of play, e.g. by altering visually that area of the image.

The determination of the critical instant and the modification of the presented image for each frame of a sequence starting (or if desired ending) at that instant can be automatically applied if desired to presented images from other cameras than the goal post camera(s) 31, 32. ,5 For example, as shown in Fig. 3, the captured images from the camera located more conventionally to capture images such as shown in Fig. 1 can also have the image frames presented modified at the same instant by the operator. As shown in Fig. 3, the vertical plane 36 containing the goal line 25 can be identified in the displayed image by modifying the transparency value, or by modifying a colour value, of pixel data for image areas depicting or 0 representing the plane defined between the goal posts 21, 22 and containing the goal line 25.

The sequence of image frames from a goal post camera 31 and the sequence of image frames from the conventional camera displaced from the plane 36 can be presented simultaneously and synchronously by a split screen presentation for viewing by the television audience, by commentators, and by officials, all of whom can be reviewing the incident for entertainment, for 25 broadcast commentary, and for official adjudication purposes. The frame of Fig. 2 will in most cases provide the most certainty in determining the critical instant of the ball touch although it may happen that the video frames from a camera displaced from the goal line plane will provide clearer images for determining the instant of the touch. For example, if several football players are located around the incident, they may obscure at least partially the instant of the ball being 30 touched from the cameras 31, 32 mounted by the goal posts 21, 22 whereas a displaced camera from outside of that vertical plane 36 may provide a clearer view enabling determination of the critical instant of the touch. In this case, the operator can initiate the modification of the presented images, including the presented images from the goal post cameras 31, 32 by viewing frames from a camera displaced from the goal line plane 36. Assessment therefore of whether

2015101938 14 Dec 2015 the ball 11 had crossed the goal line 25 may then be more readily determined from a sequence of video frames other than the sequence used to determine the instant of ball touch, such as assessment using frames from a goal post camera 31, 32 using the instant of modification of the images to determine the frame at which the judgement must be made concerning crossing by the 5 ball of the goal line.

Fig. 4(d) depicts a hypothetical incident in which the touching of the football 11 by the player 15 is followed by the ball 11 being knocked back towards, and later into, the field of play. During the live action of the football game in which such a hypothetical incident occurs, it may be very difficult to determine (i) whether the first touch occurred before the ball 11 crossed the 0 plane yet proceeded to fully cross the goal line plane, yielding a score of behind (1 point), (ii) whether there was no contact until the ball 11 had crossed the plane but was subsequently pushed back into the field of play, yielding a goal score (6 points), or (iii) whether the ball 11 never fully crossed the plane but was contacted before it had crossed and was pushed back into the field of play, yielding no score and continuation of the football game.

,5 Fig. 5 schematically illustrates a work station for the operator implementing the method of the present invention. In Fig. 5 the images 10 are presented by the display 42 and using suitable, such as conventional, peripheral input devices 43, 44 the operator can control processing system 45 which is generating the replay of the stored and captured video image frames from selected cameras operating during the sporting event. By monitoring the images 10 displayed, the operator can modify the presented images upon determining the critical instant of the incident in the play, such as by a particular keyboard stroke, whereupon the processor 45 modifies the presented images 10 at the selected frame and for subsequent frames (or if desired modifies the frames up to the instant and thereupon discontinues modification). The modification of the presented images performed by the processor 25 may comprise modifying the pixel parameters or data for pixels representing parts of the target zone to one side of the boundary as shown in Fig. 2. The modification of the pixel parameters or data may be uniformly applied for all pixels representing the selected part of the target zone. In one possible embodiment, the modification may comprise applying or altering a transparency value of the pixel data. Apart from conventional RGB values captured and stored for each pixel using conventional video camera technology additionally the pixel data for each pixel can include what is termed an “alpha channel” which specifies a transparency measure for that pixel. Another term used for this technique is “image layering”. By allocating a particular alpha value for pixels representing the area of the image to be modified, the image presented for that area appears when presented to have a curtain or mist or the like overlayed thereon but still enabling visibility of the part of the

2015101938 14 Dec 2015 target zone being presented by those modified pixels. The technique is equivalent or similar to overlaying two images with the strength of the two images being determined by their respective alpha values. For example, if one pixel for one picture has an alpha value of say 200 and the corresponding pixel for the other picture has the alpha value of 100, the picture with the alpha 5 value of 200 will provide two-thirds of the “strength” of the picture at that pixel and the entire image having that uniform alpha value will become a “foreground” image with the other picture having the alpha value of 100 becomes a weaker or background image. In the present case, the background can be a plain white colour giving the appearance to the final modified part of the image of a translucent curtain.

In use of the method and apparatus of the invention, the modification or, visually, the superimposition can be performed progressively over a period of say a second or two to give the visual impression of a curtain being drawn across or up or down in the area to be modified. For example, the replay in slow motion of the frames leading up to the initial instant can be displayed normally, the replay action can be stopped at the critical frame and the curtain drawn ,5 across for dramatic visual effect, and the action resumed with the curtain in place. Of course, the modification of the image area can be an instant change at the frame depicting the critical incident so that during uninterrupted replay in slow motion of the incident the modification occurs instantly at the critical instant. Variations include a curtain being faded into the predetermined area, being moved out or faded out of the area if events after the critical incident are better seen without the curtain, or the areas other than the vertical plane containing the boundary if that enhances the visual discrimination or aesthetics of the incident.

To demarcate the area of the presented image to be modified, preferably a calibration is performed before the real time use of the invention in a sporting event. In one possible calibration, an operator may view image frames captured by a particular camera and may select 25 and store points of the image which will define boundaries or limits of the area to be modified.

For example, using an image frame from a goalpost camera 31 as depicted in Fig. 2 (without any players or sporting activity occurring) the operator can, by a point and click operation using a cursor on the displayed image, store point A indicating the bottom point of the opposition goal post 32, point B being the highest point of the opposite goal post 32, and point C being the nearest point on the goal line 25 in the camera’s field of view. A suitable algorithm may then be operated to connect point A to point B (extrapolating above the top of the goal post if visible, such as shown in Fig. 3) and to join point A to point C thereby defining an edge of the area to be modified in its transparency or colour and the rest of the image, to the left thereof in Fig. 2, from that edge to be subject to the colour or transparency modification.

2015101938 14 Dec 2015

A similar technique of marking by a point and click operation can be carried out in a captured frame as shown in Fig. 3 from a camera displaced from the goal line plane. In this case points A, B, C and D can be marked by a point and click procedure of the operator, line A to B is then extended to the top of the frame 10, B is joined to C, and C to D and continued to the top of the frame 10 thereby defining the vertical plane of the goal line 25 which define areas as shown which are to be colour or transparency modified (or defining the complementary area of the frame to be modified to exclude that delimited area), these points and/or area delimited thereby being stored for later use and real time image capture and slow motion replay.

As an alternative to this calibration process by selecting and storing particular points in the captured image presented, it is possible to store an image of the target area with a curtain or the like erected in the plane of the boundary, with that image being stored for superimposition on the real time image frames, e.g. by assigning a suitable alpha channel value to the earlier stored image frame. In the particular illustrated embodiment, the screen can be erected between the goal posts. This can be very carefully located and fixed in position without approximations that ,5 may be required if an operator can only define the boundaries of the relevant area by viewing captured images and a point and click operation to select the comers. For example, if the rules of football require that it is the edge of the goal line away from the field of play that defines a line which the ball must fully cross then the screen can be erected accurately on that line. Likewise if the back edge of the cross section or shape of the goal post defines the relevant line, the screen can be accurately erected at that rear most point of each goal post. Captured images of the erected screen can then be used to capture and store reference points for the comers of the image of the curtain to be superimposed on real time images. The comers of the erected calibration screen can be visually marked on the actual image of the calibration screen facilitating later capture by the operator of the reference points.

Although described herein with the critical instant of the incident being identified and the image modification being effected at that frame, another possibility is to modify the parts of the captured images in the target area depicting the areas to one side of the boundary throughout the entire replay sequence. This can still enable improved discrimination of the critical instant of the incident by enhancing visibility of the boundary. For example, the images captured and displayed from a goal post camera 31 (e.g. the frame in Fig. 2) can have a “permanent ” pixel data modification for one side of the boundary which will improve discrimination of the critical instant of the ball 11 crossing the line 25.

It will be seen that the method and apparatus of the present invention can provide a means for accurately determining the exact location of an incident in the sporting event relevant to the

2015101938 14 Dec 2015 boundary. This can enhance the entertainment value of a telecast of the sporting event when significant incidents occur. The commentators involved in a broadcast of a sporting event will later reply the event and, have the opportunity to make entertaining and informative comments on the incident. Officiating persons required to make the decisions under the rules of the sporting activity can utilize the method, the apparatus and images to make rulings concerning the incident in the sporting activity which can be more accurate and therefore can reduce controversy.

Claims (18)

1. Claims

   1. A method of presenting images of a sporting event enabling determination of the location of an incident in the sporting event in relation to a boundary, the method including the steps of:

   (a) capturing video image frames of activities occurring in the sporting event in a target

   5 zone of interest which includes a boundary relevant to application of a rule of the sport, (b) presenting by a display the captured images, (c) modifying the presented image for at least one of the image frames depicting the incident in a manner such that portions of the image depicting areas within the target zone to one side of the boundary are visually differentiated from areas to the other side

   0 of the boundary, whereby the visual differentiation enables visual assessment of the location of the incident in relation to the boundary.
2. 2. A method as claimed in claim 1 wherein the step of modifying the presented images comprises modifying pixel parameters or data for pixels representing parts of the target zone ,5 located to one side of the boundary.
3. 3. A method as claimed in claim 2 wherein the modification of pixel parameters or data is a uniformly applied modification for all the pixels representing the parts of the target zone located to one side of the boundary.
4. 4. A method as claimed in claim 2 or 3 wherein the modification of the pixel parameters or

   0 data comprises applying or altering a transparency value of the pixel data whereby the modified parts of the resulting displayed image visually resembles a curtain or semi-transparent overlay or the like while still enabling visibility of the part of the target zone being presented by those modified pixels.
5. 5. A method as claimed in claim 2 or 3 wherein the modification of the pixel parameters or

   25 data comprises adjusting at least one colour value for all pixels or at least a substantial number of pixels representing the part of the target zone located to one side of the boundary, e.g. by addition to the red value for the pixels to the selected side of the boundary line whereby the presented images have a red tinge for parts of the image to one side of the boundary line.
6. 6. A method as claimed in any one of the preceding claims wherein the sporting event is a

   30 game of Australian Rules Football and the boundary comprises the goal line extending between the two goal posts, and wherein the incident of interest involves the question of whether the football has crossed the goal line.
7. 7. A method as claimed in any one of the preceding claims wherein the video image frames are captured by a video camera located in a plane extending vertically and containing the boundary.

   2015101938 14 Dec 2015
8. 8. A method as claimed in claim 7 as appended to claim 6 wherein the camera is located in or on a goal post so that its field of view includes the other goal post and a major proportion of the goal line extending in a straight line along the ground between the two goal posts, whereby the presented images from that camera depict the goal line constituting the boundary line of which

   5 the opposite goal post is a visual continuation and whereby the area to one side of that line is within the field of play and to the other side of that line is outside of the field of play.
9. 9. A method as claimed in any one of claims 1 to 6 wherein video image frames are captured by a video camera which is not located in a vertical plane in which the boundary line lies but is displaced from that plane so that the camera is directed towards that plane at a shallow acute

   0 angle, preferably less than about 20°-30°, and most preferably less than 10°.
10. 10. A method as claimed in claim 9 wherein the camera displaced at a shallow acute angle from the boundary line plane is operated simultaneously with a camera capturing images of the target zone from a point within the vertical plane containing the boundary line whereby the modified images from the camera in the boundary line plane enable accurate differentiation of the location ,5 of the incident relative to the boundary line plane and wherein the modified images of the sporting activity in the target zone displayed from the images captured by the displaced camera enable inspection of the incident itself, and wherein the method further includes modifying the displayed images for at least one frame depicting, or very near to, the incident of interest and preferably either modifying the displayed images for image frames leading up to that incident 0 frame or modifying the displayed images for image frames depicting the activity following the incident frame, whereby the timing of the incident of interest in relation to movements occurring within the sporting activity relative to the boundary can be determined with the visual aid of the modification of the displayed images from the displaced camera.
11. 11. A method as claimed in claimed in any one of the preceding claims and including the further 25 step of performing a calibration to predetermine the areas within the captured images that are depicting locations to one side of the boundary line and which is therefore to be visually differentiated, the calibration step including identifying points of the imaged target zone that define limits of the area lying to one side of the boundary, and storing those identification points to be retrieved and used in performing the modification of the presented images of an actual 30 incident in the sporting event.
12. 12. A method as claimed in claim 11 wherein the calibration step includes capturing and displaying a still image of the target zone (which may be a single frame from a captured video sequence) and which includes at least part of the boundary, and wherein the step of identifying points comprises identifying on the displayed still image points demarcating the boundary and

    2015101938 14 Dec 2015 the vertical plane in which the boundary lies and storing those points whereby a line connecting those points demarcates the side of the image representing the field of play and the other side outside of the field of play.
13. 13. A method as claimed in claim 11 wherein the calibration step includes the step of erecting a

    5 screen or other visible aid at the target zone in a calibration operation performed before the sporting activity commences, such as a screen or curtain erected between two goal posts and extending vertically above the goal line extending between the goal posts, the calibration step further including capturing an image of the screen by a camera located at the point where the live camera later capturing image frames for the sporting event is located, followed by recording data

    0 for the location of the screen within the captured image, the data being selected from an image of the screen, the lines in the captured image are depicting peripheral edges of the screen, and points within the captured image depicting key points of the edges of the screen, whereby during the subsequent sporting activity the presented images derived from the camera located at the same point as the camera in the calibration step can be modified by superimposing an image of ,5 the calibration screen.
14. 14. A method as claimed in any one of the preceding claims wherein the step of modifying the presented image is not performed instantly but is a progressive modification of the affected part of the displayed image to simulate a screen or curtain or colour tint progressively fading into (or fading out of) or moving into the area affected.

    0
15. 15. A method of adjudicating in a sporting event to make a ruling whether an incident during the sporting event occurred to one side of a boundary line relevant to the application of a rule of the sport, the adjudicating method including inspecting the modified presented images produced by a method as claimed in any one of the preceding claims and, at least partially on the basis of that inspection, providing a ruling concerning the applicable rule of the sport in relation to the

    25 incident.
16. 16. A data set or recording of picture data for a sequence of video image frames depicting activities occurring in a sporting event and which have been captured and modified by the capturing and modifying steps as defined in claim 1.
17. 17. A single image depicting an incident in a sporting event and the relationship of that incident

    30 to a boundary, the image having been produced by the capturing step (a) and the modifying step (c) as defined in claim 1, and wherein the image is depicting the particular image frame, or a closely adjacent frame, most clearly presenting the time of the incident and the incident’s relationship to the boundary, and that image includes the image modifications to the areas located to one side of the boundary.

    2015101938 14 Dec 2015
18. 18. Apparatus for enabling determination of the location of an incident in a sporting event in relation to a boundary, the apparatus including means for carrying out the steps of the method as claimed in any one of claims 1 to 14.