WO2002056254A2

WO2002056254A2 - Monitoring system

Info

Publication number: WO2002056254A2
Application number: PCT/GB2002/000093
Authority: WO
Inventors: James Matthew Orwell; Philip John Seeley
Original assignee: South Bank University Enterprises Ltd.
Priority date: 2001-01-10
Filing date: 2002-01-10
Publication date: 2002-07-18
Also published as: GB0100579D0; AU2002219357A1; WO2002056254A3

Abstract

A method for obtaining a record of the playing and performance characteristics of one or more players of a game is by video recording the game with spaced apart cameras, processing the images obtained using a visual recognition system to obtain a record of one or more playing or performance characteristic of any player or group of players. The record can be used as a coaching aid, to program a virtual player in a computer game or can be retransmitted.

Description

Monitoring System

The present invention relates to a method and equipment for monitoring and recording the movement of players playing a game and movement of ball etc. so the individual player's and team's performance and behaviour can be analysed.

When a team game such a football, rugby etc. is filmed or videoed a visual recording is made which can be re-played later and the performance of players analysed. However such recordings, of necessity, follow the action and the activity hvthe vicinity of the ball and no effort is usually made to follow any one player. Even if a camera is devoted to following one player, this means that this camera cannot follow other action or players on the field and no other player can be followed by this camera.

A system has been proposed for ice hockey in which the ice hockey players have a radio transmitter attached to them and there are receivers which can receive and record the individual signals so that the movement of each player can be monitored.

However such systems are active in the sense that players have to transmit a signal and these systems do not record the motion of the puck and can only record what is mainly two dimensional activity i.e. it can be difficult to record how a high a player may jump or movement of the puck through the air and would not be effective for a game such as football.

We have devised a system and apparatus which enables the activities of all the individual players and movement of the ball to be recorded which is passive and without the need for the players to transmit a signal.

According to the invention there is provided a method for recording the activities in a game or the like of individual players and movement of an object played with, which method comprises taking a video recording of the game using at least one camera, processing the images obtained through a visual recognition system and obtaining a record of the movement of individual players and/or the object played with.

Preferably more than one camera is used so that, if the view of a player is occluded by another player, another camera will have a clear view, also the use of one camera will suffer from the drawback that that the activities near the camera will be better recorded than those more distant from the camera.

When more than one camera is used, the cameras are preferably spaced apart e.g. on opposite sides of the field or spaced around the field etc. in a game such as football typically 4 to 10 e.g. 8 cameras are used spaced around the pitch.

As well as recording the movement of the players and the object played with the camera also records line markings positions of goals flags, posts etc.

The cameras can be mounted at any level above ground level e.g. on a gantry, or at player level and, by suitable mounting of a plurality of cameras, it is possible for the height and trajectory of the ball to be recorded so players reaction to ball movement can be analysed. The height should be at least sufficient to be able to see the markings etc.

The signal received by the video camera is processed using a visual recognition system; such systems are known and systems which could be used are described in Pfinder: Real-Time Tracking of the Human Body (1995); Christopher R. Wren, Ali Azarbayejani, Trevor Darrell, Alex Pentland IEEE Transactions on Pattern Analysis and Machine Intelligence, July 1997, vol 19, no 7, pp. 780-785. J. Orwell, P. Remagnino, G.A. Jones, "Multi-camera Colour Tracking " IEEE International Workshop on Visual Surveillance, Fort Collins, Colorado, pp. 14-21. (1999)

McKenna S J, Jabri S, Duric Z, Rosenfeld A, Wechsler H, "Tracking Groups of People", Computer Vision and Image Understanding, Vol 80, No 1, October 2000, 42-56.

These are incorporated by reference.

In these systems the movement of one individual amongst a number of people can be followed as the system can pick out and track any particular individual.

In the present invention it is possible, when more than one camera is used, to be able to select the images viewed, as all the action will have been recorded i.e. any particular player can be followed, any size of image selected etc.

The invention can use conventional video cameras and no special recording equipment is required.

With more than one camera it is possible to obtain images which can be manipulated to follow any one of selected players without using tracking cameras. In present systems a camera has to follow a player to enable that player to be selected and that camera cannot follow another player at the same time. Also a particular camera can only record one image at a time so that the size etc. of the image is that selected by the cameraman with the present invention

The invention can be used as a recording i.e. offline or it can be used in real time. When the invention is applied off line it can be used as a coaching aid. As the data obtained can be stored in digital form it can be easily processed by a computer to provide information in the form required by a coach so that, if a coach wishes to analyse a particular aspect of a player's performance, this aspect can be selected and presented in a suitable form. This selection and presentation greatly facilitates the analysis required by the coach compared to trying to analyse a recording of a match for individual features of a player's performance, as such recordings produce a large volume of unprocessed information,

The method of the invention enables all the movements of all the individual players to be analysed after a game, for example the speed of a player's movements on and off the ball, coverage of the field, how that player reacts to each situation, how he positions himself in relation to other players on both sides, how groups of players interact and the team performance etc. can all be played back and analysed after the game. This enables a coach to assess more easily the performance of each player, group of players or the team.

The method of the invention enables a database to be obtained of the performance of a player over a series of games so that his performance can be monitored over a season or part of a season so changes in fitness, skills, awareness etc. can be monitored and similar databases can be obtained for teams as well as for individuals.

The system can be used to obtain a range of data and statistics about a game, for example speed of a ball, trajectory of a ball, whether a player is an offside position, how far a ball misses the goal or the posts or crossbar etc.

The system can be used for different applications for example in analysing a player's and/or a team's performance as referred to above and for creating a game or similar representation. A database of the features of a player's performance can be obtained, based on actual performance in a game or games. Such a database incorporating features of a player's performance can be used to programme a "virtual player" for use in a computer game played on a computer or on a dedicated games machine. Such a virtual player can have all or selected characteristics of the real player and so would bring more realism to such a game. Similarly for a team, a virtual team can be programmed in with the playing characteristics of a particular team. In most games played on games machine and on computers, an operator can control the movement and action of one participant, for example in a football game an operator can move one player, the other players either are not moved or are moved by the program in a limited set of preordained moves e.g. horizontally in synchronisation with the ball. This is unrealistic as, in a real game, the players move off the ball in a wide variety of ways and such off the ball movement is a vital ingredient of the game. The present invention enables all the players to move in a way similar to how they move in an actual game and thus adding realism to such a game.

It is possible to have a virtual player based on a neural network which would learn sporting skills from one or more human exemplars with the data obtained as above being used to teach the virtual player.

For example, in a football game played with a team of such "virtual players", an operator can control one or more players and the other players would behave and perform similarly to how they have done so in actual games, thus bringing a great deal more realism to such games. When teams are selected, such a team would perform similarly to the real team and so operators can play against a chosen opposing team. It would also be possible to play teams of such virtual players against each other.

For individual games such as tennis, squash etc. an operator can choose to play against a specific virtual player and this player can be given particular capabilities of one or more players e.g. speed of serve, amount of spin that can be put on a ball etc. so as to provide a more realistic game. If playing another operator, players with different playing styles and capabilities can be chosen to play against each other, adding more interest to the game.

In an example of how the invention can be applied to an actual game such as a game of football, the digital video data (from static cameras) are transferred onto a recording medium such as hard discs. The image sequences are converted and compressed as necessary, then synchronised using both audio and visual cues. The cameras are calibrated, i.e. the relationship is established between the football pitch co-ordinates and the sets of image co-ordinates.

The people and the ball (Objects') are detected in the initial image, and then tracked through the sequence. Both detection and tracking are semi-automatic, i.e. an automatic process can provide an estimate, which can then be corrected through user inspection and intervention.

The automatic component comprises background models, used in conjunction with the inputs, to provide estimates of the relevant object properties. The models of these objects are updated, incorporating both this evidence and prior expectations.

The manual component is the user interface that allows an operator to interact with the above procedure. To correct the automatic detection, objects may be added, deleted, moved and relabeled. In correction of the automatic tracking, objects may be moved or relabeled. Additionally, an annotation facility is used to provide a formal commentary of the game, detailing the passes, tackles, interceptions and shots, together with changes of possession.

This information is saved to disc, then converted into a standard database format suitable for fast queries and aggregation. A second program is used to view the game analysis. Detailed descriptions of individual player statistics, team behaviours and match progression may be viewed, together with the original video observations. Training videos may be compiled with relevant captioning. Data from several matches can be integrated to provide a longer term perspective.

This system can be used for action replays with the ability to select a player or incident together with detailed analysis e.g. at half time or at the end of a game and will provide a more detailed analysis.

When the invention is applied as a real time tracking system the image from each camera is obtained and processed as above and transmitted to a viewing system.

The viewing system can display a representation of the sports event on a remote device for example over a low bandwidth e.g. up to 500kb by, for example, animating the record obtained.

For a team the movements of all the players can be recorded simultaneously and can be displayed as movements of a coloured image, if the individual numbering cannot be recorded, the players can carry identifiers which transmit a signal, provided that it would not interfere with the player, and movements of this signal transmitter recorded and displayed, and the images obtained correlated.

In a real time application the images from the cameras are recorded as above and then retransmitted to a watcher.

The interface for reporting can be any system, for example a mobile telephone network, so that an image can be accessed by mobile phone users or it can be transmitted as a video or television signal. There should be a minimum or no delay between the input and the output. In order to optimise performance i.e. minimise delay, optimisations such as only decompressing and processing parts of images in which there is expected to be evidence, using the most efficient decompressor suitable for the job, using the minimum acceptable frame-rate and resolution before the accuracy significantly degrades can be used.

The system may also include features such as being able to accommodate more than operator to allow for the different components of the report, e.g. player identification and match event recording.

As stated above, with more than one camera it is possible to obtain images which can be manipulated to follow any one of selected players without using tracking cameras. In present systems a particular camera can only record one image at a time so that the view, size etc. of the image is that selected by the cameraman so that, when watching a game on television, the watcher can only watch what the camera is showing, although it is possible, with interactive television, to select which camera's image is viewed, the watcher is still restricted to whatever view is chosen by that camera, although more than one camera's view can be displayed at one time on a "split screen".

In the present invention, as all the action will have been recorded and transmitted to a receiver, if that receiver has the facility for the viewer to be able to manipulate the image displayed, the viewer can choose what image he wishes to see, e.g. he is able to select any particular player, to control the view seen i.e. to view the image as would be seen from a particular vantage point, to select the magnification i.e. control the zoom etc. The system also enables activities such as the speed a ball moves etc. to be monitored. The viewing can be carried out in conjunction with a commentary as transmitted in existing broadcasts. In a real time viewing of a match an interactive watcher can select what he wants to watch and can act as his own director and is not limited to the images chosen by a particular cameraman and director.

Although described mainly as applied to football the system can be applied to any game, sporting event or similar activity.

The invention can be also be used in support of gaming for example, in matches such as football matches, bets are taken on which part of the pitch, as marked by squares, significant action takes place. The use of the present invention will enable this to be determined more accurately and the compilation and analysis of statistics over a series of games will enable more accurate predictions of such events.

Claims

1. A method for recording the activities of individual players in a game which method comprises taking a video recording of the game using at least one camera, processing the images obtained through a visual recognition system and obtaining a record of the movement of individual players and the ball.

2. A method as claimed in claim 1 in which there are a plurality of spaced apart cameras.

3. A method as claimed in claim 1 or 2 in which the signal received by the video camera is processed using a visual recognition system.

4. A method as claimed in any one of claims 1 to 3 in which the movements of individual players are analysed after a game.

5. A method as claimed in any one of claims 1 to 4 in which the movements of the object played with is recorded.

6. A method as claimed in any one of the preceding claims in which the recorded images are retransmitted.

7. A method as claimed in claim 6 in which the recorded images are retransmitted via a mobile telephone network.

8. A method as claimed in claim 6 in which the recorded images are retransmitted to a television receiver.

9. A method as claimed in claim 8 in which the television receiver is interactive and has the facility to enable a viewer to manipulate the image.

10. A method in which a representation of a sporting event is displayed on a remote device by animating the record obtained by the method of any one of claims 1 to 5.

11. A method as claimed in claim 10 in which the representation is displayed over a low bandwidth connection.

12. A method as claimed in any one of the preceding claims 1 to 5 in which the data recorded is stored on a database which database incorporates parameters of a player's performance and/or movement of an object played with.

13. A coaching system which uses data fRom the database as claimed in claim 12.

14. A computer game in which there is at least one virtual player who has playing characteristics obtained by the method of claim 12.

15. A computer game as claimed in claim 14 in which the virtual player is based on a neural network which learns sporting skills from one or more human exemplars with the data obtained as in any one of claims 1 to 5 being used to teach the virtual player.