EP3724851A1

EP3724851A1 - Apparatus and method for interpreting image sequences

Info

Publication number: EP3724851A1
Application number: EP18826977.3A
Authority: EP
Inventors: Vigen BADALYAN
Original assignee: Sc Ip Ltd
Current assignee: Sc Ip Ltd
Priority date: 2017-12-12
Filing date: 2018-12-10
Publication date: 2020-10-21
Also published as: GB201720652D0; WO2019115445A1

Abstract

A system to record information about events is disclosed, in which at least one image source is adapted to provide a series of photographic images of a field of view to a processing unit, the processing unit is adapted to identify and locate a marker within one or more photographic images provided thereto and provide a location of said marker relative to the field of view, and the processing unit is further adapted to derive and record a translation of said marker within the field of view between two or more images comprising the said series of images.

Description

Apparatus and method for interpreting image sequences

Field the invention

The present invention relates to the interpretation and/or analysis of sequences of images, especially in the field of live sports. The invention is especially useful relative to stadium sports such as tennis, soccer and the like but is not limited to this application. Indeed, the applicant believes that the underlying technology could be applied in a wide range of technical fields beyond that of live sports. Nonetheless, the invention will be described with a focus on live sports.

Background to the invention

Live sporting events may attract a significant following, and are often well-attended by people at the stadium itself as well as via live television broadcasts and subsequent edited highlights or review shows. In addition, a wide variety of statistics is collected during such sporting events. Such statistics may relate to, for example, the distance a particular athlete has run during a soccer match or the number of aces served during a tennis match. Some of the information collected during an event may be collated by automated means but some information requires manual recordal by a person in the stadium or watching a live feed. Such a person is sometimes referred to as a“scout”. However, in a fast-moving sporting event there may be multiple recordable events occurring in quick succession and a single scout may be insufficient to provide suitable recordal of desired information. Furthermore, it may be important for the information collected during an event to be provided as quickly as possible. This may arise, for example, if the information is to be relied upon by betting markets and the like. Such markets are fast-moving and allow betting positions to be traded. Prices in such markets may move in dependence on events taking place during a sporting event. Thus, it is important to obtain accurate information as quickly as possible.

There is therefore a need for the provision of an automated information recordal system to provide information about recordable events as soon as possible after they have taken place.

WO 01/41884 of Sherry and Hawkins discloses an arrangement which is implemented especially in cricket, which monitors the actual path of a ball and aims to extrapolate such a path with so as to predict a would-be future flight of the ball.

Summarv of the invention

Accordingly, the present invention is directed to a system to record information about events comprising at least one image source adapted to provide a series of photographic images of a field of view to a processing unit, the processing unit being adapted to identify and locate a marker within one or more photographic images provided thereto and provide a location of said marker relative to the field of view, and wherein the processing unit is further adapted to derive and record a translation of said marker within the field of view between two or more images comprising the said series of images.

Thus, the present invention enables movement of a marker to be tracked between images.

The two or more images may be substantially subsequent to each other within the said series. If the images are provided sequentially in time, movement of a marker may be tracked with respect to time.

Preferably, the field of view is known. More preferably, the field of view is pre-determined and substantially does not vary over time.

The image source may comprise a camera. The marker may represent a physical object located within the field of view of the camera. Thus, the system of the present invention may enable location and movement of a physical object within the field of view to be tracked over time.

The processing unit may be adapted to identify and locate more than one marker within each photographic image. In that case, the processing unit is preferably adapted to derive and record translations of each marker within the field of view between two or more images comprising the said series of images.

The image source may comprise more than one camera. Provision of more than one camera may enable location and movement information to be derived in more dimensions. For example, this may be achieved by triangulation.

The said field of view may alternatively or further be provided with a co-ordinate system such as axes or a grid. A co-ordinate system may supplement a single-camera arrangement or a plural-camera arrangement to provide improved location information.

The marker or markers may be relatively simple image artefacts forming part of an image provided by the camera or cameras. Alternatively, they may be formed of one or more complex arrangements of image artefacts. A correspondence between a marker and a physical object may be provided by the processing unit or by an external unit. Such a correspondence may be provided by a machine learning apparatus.

Examples of correspondences which may be useful in the present invention may be between a complex image artefact and an object forming part of a sporting event such as a ball or a player or a sports implement such as a raquet or bat. Correspondences may be derived, for example, by identification of a shape of an object or by a proxy such as an indicium displayed on an object. For example a player may have a number or other indicium displayed on his or her clothing, the indicium being capable of being recognised by the processing unit.

Thus, the system of the present invention may be able to track the movement of a physical object over time during a sporting event.

The processing unit may be adapted to provide a description of a derived translation of a marker. Such a description may be in a geometric or mathematical or computer-readable form, or in a linguistic form. For example, a geometric description may be a path along which a marker is travelling. Alternatively or in addition, a linguistic description may be provided which is understandable by humans.

Thus, the system of the present invention may be adapted to provide an output which describes in a suitable form the movement of an object within a field of view. A field of view may be the playing area of a sport, or some other location.

The system of the present invention may comprise more than one camera to provide more than one image of the field of view from differing points-of-view. Thus, the system may be able to derive and record a translation of a marker in more than one dimension. In that situation, a description of a derived translation may have components in more than one dimension. This may be useful for providing an indication of certain events, for example a ball going out of play in many sports or a player being off-side in soccer.

Preferably, the processing unit identifies a marker by recognising aspects of the shape or other features of the marker within one or more photographic images. While it is known from WO 01/41884 of Sherry and Hawkins for a system to identify pixels within an image relating to a ball, the present invention is able to identify and locate more complex markers by utilising machine learning.

In machine learning, a processing unit may be trained to identify aspects and elements of a particular marker by providing data sets to the processing unit including representations of the marker in question. In this situation the processing unit is adapted to identify common and varying aspects of the said marker so that when provided with a further image the processing unit has a reasonable chance of correctly identifying the said marker.

Furthermore, a processing unit may be further developed by an iterative process in which a marker or markers which was or were not correctly identified may be manually identified by a human operator to provide for a better chance of the processing unit correctly identifying a respective marker in a further image provided thereto.

In a further arrangement, the system of the present invention may comprise at least one microphone connected to the processing unit, the processing unit being adapted to identify a sound marker. By identifying a sound marker, the processing unit may be able to supplement or verify an identification and/or location of a marker in a photographic image.

The system of the present invention may further be adapted to provide an output suitable for a display. A display screen may be provided as part of the system or may be extrinsic - for example provided by a viewer connected remotely to the system of the present invention.

A possible further use of the present invention may be to provide a graphical representation of a recent movement path of the marker within the field of view. For example, as a marker moves within the field of view, such a movement path may be graphically represented as a trail on a display apparatus. In such a way, a display of an event in progress may be augmented. Thus, a viewer of an event in progress may be able to follow the event more fully. This may be especially useful in a fast-moving sport such as hockey, wherein the“puck” may be provided with a trail on a display screen to allow viewers to keep track of the game as it occurs.

Figure 1 shows an extract from a display image 100 showing a portion of a hockey pitch 102. In the extract 100, a puck 104 is moving across the pitch 102. As the puck 104 moves, the system adds a graphical trail 106 to the display image 100.

Claims

1. A system to record information about events comprising at least one image source adapted to provide a series of photographic images of a field of view to a processing unit, the processing unit being adapted to identify and locate a marker within one or more photographic images provided thereto and provide a location of said marker relative to the field of view, and wherein the processing unit is further adapted to derive and record a translation of said marker within the field of view between two or more images comprising the said series of images.

2. A system according to claim 1, wherein the two or more images are substantially subsequent to each other within the series.

3. A system according to claim 1 or claim 2, wherein the field of view is known.

4. A system according to any preceding claim, wherein the image source is a camera.

5. A system according to claim 4, wherein the marker represents a physical object located within the field of view of the camera.

6. A system according to any preceding claim, wherein the processing unit is adapted to identify and locate more than one marker within each photographic image.

7. A system according to claim 6, wherein the processing unit is adapted to derive and record translations of each marker within the field of view between two or more images comprising the said series of images.

8. A system according to any of claims 4 to 7, wherein the image source comprises more than one camera. Provision of more than one camera may enable location and movement information to be derived in more dimensions. For example, this may be achieved by triangulation.

9. A system according to any preceding claim, wherein the field of view is provided with a co-ordinate system.

10. A system according to any preceding claim, wherein the marker or markers are relatively simple image artefacts forming part of an image provided by the camera or cameras.

1 1. A system according to any preceding claim, wherein the marker or markers are formed of one or more complex arrangements of image artefacts.

12. A system according to any preceding claim, wherein a correspondence between a marker and a physical object is provided by the processing unit or by an external unit.

13. A system according to claim 12, wherein the correspondence is provided by a machine learning apparatus.

14. A system according to any preceding claim, wherein the processing unit is adapted to provide a description of a derived translation of a marker.