GB2589903A

GB2589903A - A device, computer program and method

Info

Publication number: GB2589903A
Application number: GB1918296.3A
Authority: GB
Inventors: Gadsby Howard
Original assignee: Sony Europe BV United Kingdom Branch; Sony Corp; Sony Europe BV
Current assignee: Sony Europe BV United Kingdom Branch; Sony Corp; Sony Europe BV
Priority date: 2019-12-12
Filing date: 2019-12-12
Publication date: 2021-06-16
Also published as: GB201918296D0

Abstract

A method of selecting at least one camera feed for display comprises: receiving a plurality of camera feeds, each camera feed being provided by a camera located at a different position in a sporting venue but capturing the same incident (e.g. goal, penalty, red card, foul, case of mistaken identity) in a sporting event (e.g. soccer match) held at the sporting venue; selecting the at least one camera feed based on the position of the camera relative to the position of the incident; and displaying the selected at least one camera feed. Camera feed selection or choice may be based on the camera field of view (FOV), or additionally based on incident type. The aim is to address the problems related to finding and efficiently displaying the most relevant camera streams, for example for use in a Video Assisted Referee (VAR) system.

Description

A DEVICE, COMPUTER PROGRAM AND METHOD

BACKGROUND

Field of the Disclosure

The present technique relates to a device, computer program and method.

Description of the Related Art

The "background" description provided herein is for the purpose of generally presenting the context of the disclosure. Work of the presently named inventors, to the extent it is described in the background section, as well as aspects of the description which may not otherwise qualify as prior art at the time of filing, are neither expressly or impliedly admitted as prior art against the present technique.

Video Assisted Referee (VAR) has been implemented in many sporting events. VAR assists referees in making the correct decision at a sporting event. For example, in a soccer match, VAR is sometimes used to help identify handballs, offside, goals, fouls and the like, in order to make a decision using VAR, many video streams are reviewed. These video streams show an incident from many different angles. In some instances, 60 different video streams showing an incident are shown on a display. The operator controlling the VAR system then has to review each stream to identify relevant streams. In some instances, there may be 2 or 3 operators performing this role. These relevant streams then have to be reviewed in detail by a referee in order to make the decision The process of reviewing the 30 or so video streams to identify; relevant streams is particularly time consuming. This is problematic as the spectators at the sporting event and any television viewers have to wait whilst all the video streams are reviewed. Moreover, in order to find the relevant video streams it is useful to have other video streams also showing the incident being displayed at the same time. This allows the operator to compare each video stream to find the most relevant video streams. However, where large numbers of video streams are shown at the same time, a very large display or displays will be required. This is expensive and inconvenient.

It is an aim of the present disclosure to address at least one of these problems.

SUMMARY

According to embodiments of the disclosure, there is provided a method of selecting at least one camera feed for display, comprising: receiving a plurality of camera feeds, each camera feed being provided by a camera located at a different position in a sporting venue but capturing the same incident in a sporting event held at the sporting venue; selecting the at least one camera feed based on the position of the camera relative to the position of the incident; and displaying the selected at least one camera feed The foregoing paragraphs have been provided by way of general introduction, and are not intended to limit the scope of the following claims. The described embodiments, together with further advantages, will be best understood by reference to the following detailed description taken in conjunction with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

A more complete appreciation of the disclosure and many of the attendant advantages thereof will be readily obtained as the same becomes better understood by reference to the following detailed description when considered in connection with the accompanying drawings, wherein: Figure lshows a plan view of a sporting event held at a sporting venue with cameras being located at different positions around the sporting venue; Figure 2 shows a data structure storing a camera identifier in association with the camera position and the camera type; Figure 3 shows an apparatus according to embodiments of the disclosure; Figure 4 shows a user interface according to embodiments of the disclosure and Figure 5 shows a flow chart according to embodiments of the disclosure.

DESCRIPTION OF THE EMBODIMENTS

Referring now to the drawings, wherein like reference numerals designate identical or corresponding parts throughout the several views.

Figure 1 shows a plan view of a sporting event held at a sporting venue with cameras being located at different positions around the sporting venue. In the embodiments of Figure I. the sporting venue is a soccer stadium and the sporting event is a soccer match. However, as would be appreciated, the disclosure is not so limited and any sporting event at any sporting venue is envisaged. For example, the sporting event may be a rugby match at a rugby stadium, an American football match at an American football stadium, a baseball match at a baseball stadium or the like.

In the sporting venue 100, the pitch has a number of cameras 105A-H around it. Each of the plurality of cameras 105A-H is placed in a different position around the pitch and captures a different view of the pitch. in other words, each of the plurality of camera is located at a different geographical or physical location around the pitch. In embodiments, the pitch is surrounded by a plurality of cameras. However, the disclosure is not so limited and the camera may be located around an area of the pitch that is smaller than the pitch.

Each camera 105A-H generates a camera feed that is an image or image stream of the field of view captured by the camera. These camera feeds may be broadcast video feeds which are sent to viewers at home watching the sporting event on television, or over the Internet or may be used to capture the event for use on in-stadium displays. The camera 105A-H may be specifically positioned to be used to assist the referee in a Video Assisted Referee (VAR) scenario. Of course, the cameras may have a mix of uses such as television cameras, and VAR cameras or the like Although not shown explicitly in Figure 1, the camera feeds generated by each of the plurality of cameras 105A-H which arc located at a different position in a sporting venue arc fed into an apparatus according to embodiments of the disclosure Such an apparatus will be explained with reference to Figure 3 of the present application.

As will be apparent to the skilled person, in the description of Figure 1, there are provided eight cameras located at different positions around the sporting venue. Of course, the disclosure is not so limited and any number of cameras is envisaged.

In the example of Figure 1, each camera 105A-H is located at a different position around the sporting venue and has a different viewing perspective of the sporting event being played on the pitch in the sporting venue. As each of the cameras 105A-H is located at a different position in the sporting venue, each camera has a different field of view of the sporting event. In embodiments, this means that each camera 105A-H has a different field of view of the sporting event. Further, each camera 105A-H may have a different focal length or different lens arrangement compared to the other cameras. This means that different cameras may capture different perspectives of incidents on the pitch. For example, camera 105D is, in embodiments, located in a broadcast gantry'. This means that camera 105D captures the entirety of the pitch in its field of view. However, due to the wide field of view, this means that it will be difficult for camera 105D to capture a detailed image of an incident that occurs in a penalty box on the pitch.

In embodiments, camera 105B is located behind the East goal. The camera 105B may be a camera having a narrow field of view, for example of the East penalty box, and a short focal distance. This means that camera 105B may capture a detailed image of an incident occurring in the East penalty box but may not capture an incident occurring in the West penalty box.

Accordingly, each camera 105A-H will have a different field of view of the pitch by virtue of the different position of each camera. This means that each camera 105A-H will have a different field of view of any incident that occurs on the pitch. Moreover, and as described with reference to Figure 2, each camera may have a different focal length or different lens arrangement which means that each camera will have a different view in respect of any incident. This different focal length or different lens arrangement are examples of different characteristics or types of camera.

Referring to Figure 1, the sporting event is a soccer match. A first team shown in solid lines are playing a second team shown in dotted lines. For case of explanation, only several players from each team are shown in Figure 1. Specifically, the first team includes players 110A-110H arid the second team includes players 115A-115C. Additionally, the ball 120 is shown in Figure I. In an embodiment of the disclosure, an incident has occurred in the sporting event shown in Figure 1. In embodiments of the disclosure, the incident represented in Figure 1 is a handball performed by player 110G of the first team. Of course, although the incident shown in Figure 1 is a handball, any kind of incident such as an offside, goal, foul on a player or the like is envisaged. Of course, in the event that the sporting event is not soccer, other incidents are envisaged. An incident therefore is defined as an event occurring that requires review of captured video footage by an operator. In embodiments, this is using the apparatus shown in Figure 3 which is, for example, a VAR apparatus.

In the example of Figure 1, the incident occurred near the East penalty box on the pitch.

In embodiments, the pitch may be divided into quadrants. These are shown in Figure 1 as quadrant 1-4. Specifically, the incident occurred in quadrant 2 of the pitch As will become apparent later, this may allow the approximate location on the pitch of an incident to be defined. Accordingly, the pitch may be divided in any manner and may include dividing the pitch into 2, 8, 16 or more areas. Typically, the more divisions are provided, the more accurate the location of the incident may be defined. Moreover, the shape of the divisions may vary. For example, in areas where there are a high number of incidents relative to other areas of the pitch (for example in the penalty area), this area may have more divisions compared to the rest of the pitch.

Turning now to Figure 2 that shows a data structure storing a camera identifier in association with the camera position and the camera type, in the embodiments of the disclosure, the data structure may be a database or the like. Of course, the disclosure is not so limited.

In the data structure 200 three camera identifiers are shown. These camera identifiers are shown in the camera identifier column 205. Associated with each camera identifier is the position of the camera.

Although this is described as being a location at the sporting event, the position of the camera may be a GPS co-ordinate or some other unique position that identifies the position of the camera at the sporting venue. This position is shown in column 210, in other words, the geographical position of the camera having the identifier shown in column 205 is shown in column 210.

Further, the data structure 200 has a camera type column 215. The camera type column 215 stores the type of camera having the camera identifier shown in column 205. In other words, the camera type column 215 shows the camera type associated with the camera identifier shown in column 205.

As noted above, in embodiments of the disclosure, the camera type relates to the field of view of the camera having the camera identifier in column 205. For example, the camera located on the "south broadcast gantry" is a wide angle camera type that shows the whole pitch. This is sometimes referred to as a camera number I or Main camera as this camera has a field of view covering the entire pitch.

In other examples, the camera identifier 105B shows a camera positioned behind the goal located on the east side of the sporting venue. This type of camera typically is a close camera meaning that it is following the movement of the ball or a certain player and its field of view includes the ball or a particular player.

In a third example, a camera identified with camera identifier 105A is located on the north-east corner of the sporting venue. This camera has a wider field of view and is designed to capture the north-eastern quadrant (quadrant 2 in Figure 1) of the sporting venue. This camera type is known as a landscape camera as noted in column 215.

Finally, a quadrant camera colunm 220 is shown. In the quadrant camera column 220, the quadrant which the field of view of die camera associated with the camera identifier 205 is noted. In other words, the camera 105D has a field of view covering all quadrants, camera 105B has a field of view covering quadrants 2 and 4 and camera 105A has a field of view covering quadrant 2.

Referring to Figure 3, an apparatus 400 according to embodiments of the disclosure is shown. The apparatus 400 includes processing circuitry 405. The processing circuitry 405 is any kind of circuitry that controls the apparatus 400 to perform a method according to embodiments of the disclosure. The processing circuitry 405 may therefore be an Application Specific Integrated Circuit (ASIC) or a microprocessor that operates under the control of software code.

The software code may be a computer program that is stored on storage 410 located within the apparatus 400 and connected to the processing circuitry 405. The storage 410 may be any kind of computer readable storage such as solid state storage, magnetically readable storage or optically readable storage.

The processing circuitry 405 is also connected to communication circuitry 415, The communication circuitry 415 may be any kind of circuitry that allows the apparatus 400 according to embodiments of the disclosure to communicate with the cameras located at the sporting venue. in other words, the communication circuitry 415 allows the apparatus 400 to communicate with the plurality of cameras 105A-H shown in the sporting venue of Figure 1. This communication may occur by a direct connection between the apparatus 400 and the cameras or via a network such as over the interaet or the like. The purpose of the communication circuitry 415 is to receive an image or camera feed (which may include a plurality of images) from the cameras provided at the sporting venue.

Figure 4 shows a user interface 300 according to embodiments of the disclosure. The user interface 300 includes a first camera feed 310 and a second camera feed 305. Each of these two camera feeds show live broadcast feeds. Typically, one feed is the main camera (camera 1 as noted above) and the other is the produced program feed being broadcast.

The user interface 300 also includes a list of cameras 365 located at the sporting event. The Fist of cameras 365 may include all cameras located at the sporting event or only cameras that are available for viewing at the sporting event.

The user interface 300 includes an incident checking area 320. The incident checking area 320 includes 4 buttons. These buttons instnict the apparatus 400 to show appropriate camera feeds for a goal, penalty, red card or mistaken identity (i.e, the referee applies a punishment such as a red card to the incorrect player).

There is also provided a camera feed area 315. The camera feed area 315 is a section of the user interface 300 upon which camera feeds showing the incident defined by the incident checking area 320 are located.

The camera feeds are synchronised so that each camera feed shows the same moment in time but from different camera angles and different fields of view. As will be explained later, the apparatus 300 automatically selects the most appropriate camera feeds to be placed on camera feed area 315. The operator may then view the camera feeds displayed on the camera feed area 315 and the referee (who may be the operator or may be alongside the operator) determines whether the incident did occur.

This selection by the operator is in contrast to placing all the available camera feeds onto the camera feed area 315. By automatically selecting the most appropriate camera feeds and displaying these, the size of each camera feed can be increased compared to the situation where all camera feeds are placed on the camera feed area 315. This makes more efficient use of screen real-estate and provides a larger version of the camera feed making it easier for the referee to adjudge whether the incident has occurred. Further, by reducing fewer camera feeds of the incident, the referee may determine whether the incident has occurred more quickly as irrelevant camera feeds are removed.

In addition, a workflow section 325 is shown. The workflow section 325 is designed for soccer VAR. In this case, when a foul incident is being reviewed, the referee (which may be an operator) needs to review the footage to detennine whether contact has been made, in order to do this, he or she will use a very short loop around the incident running at slow speed using only feeds having a tight camera angle to make a judgement whether contact had occurred or not. If there is contact, the intensity of that contact has to be established to determine whether the contact is enough to warrant penalising the player. In this instance, the operator or referee may use a short look around the incident using a wide camera angle running at hill-speed to determine the intensity of the contact.

For other incidents, such as offside decisions, calibrated angle views arc used so that the correct side of the pitch is being viewed.

Additionally provided is a toggle switch 360 switching on or off audio communication with referee on the pitch.

Also provided in the user interface 300 is a forward one frame button 335 and a rewind one frame button 350. These buttons move all camera feeds in the camera feed area 315 forward one frame or back one frame respectively. A quicker rewind button 355 is also provided that allows multiple frames to be skipped or the footage captured by the camera feeds to be rewound more quickly. A play button 340 is also provided that allows the camera feeds to continue playing in real time.

A time Fine 330 and a time bar 345 are shown. The time line 330 shows the time line of part or all of the entire match, in embodiments, a short period of time is shown such as 30 seconds. This is changeable by the operator. It is possible to include a record of incidents on the time line as they occur. For example, it is possible to mark on the time line each offside, foul or red or yellow card or the like. The time bar 345 indicates to the operator where along the time line the current footage is An example embodiment explaining the selection of one or more camera feeds for display in the camera feed area 315 is now provided with reference to Figure 1.

As can be seen from Figure 1, there is a possible handball incident inside the penalty area on the East side of the pitch. Therefore, the "penalty" incident button from the check incident buttons 320 is selected. As a penalty is to be checked, the apparatus 400 will need to select camera feeds that show different views of the penalty area as these camera feeds are more likely to contain relevant footage of the incident.

Accordingly, a determination needs to be made whether the incident occurred in the West penalty area or the East penalty area. This may be determined either by the operator, or using footage captured from camera 105D that has a field of view including all four quadrants on the pitch, in other words, the processing circuitry 405 within apparatus 400 may automatically review the camera feed from cameras capturing all four quadrants (or from camera feeds capturing just the two penalty areas). In particular, the processing circuitry 405 may review the footage to establish where most players or the ball is located. This will indicate which penalty area includes the incident.

Of course, the disclosure is not so limited to this mechanism. In embodiments, the direction in which the main camera (camera 1) is pointing may be used to determine which quadrant the play is based. In this embodiment, camera 1 is controlled by a camera operator who moves the camera to follow the play. The direction in which camera 1 is pointing will be known. For example, if camera 1 is calibrated prior to the start of the sporting event so that the direction where the camera is pointing is known, it is possible to determine in which quadrant the play is happening.

After it is determined in which quadrant the play is carried out, the choice of camera based on both the quadrant and the type of incident being checked (see below).

In the case of Figure I, the penalty incident occurred in the East penalty area.

As the incident occurred in the East penalty area, the incident occurred in quadrant number 2 and 4.

Therefore, the processing circuitry 405 will interrogate the data structure 200 to identify which cameras have a view of quadrant numbers 2 or 4. From the data structure of Figure 2, cameras 105D, 105A, 105B, 105C and 105H will have a field of view including quadrant number 2 Or 4. This means that the camera feeds from cameras 105G, 105F and 105E do not need to be included in the camera feed area 315. In other words, the processing circuitry 405 selects the at least one camera feed based on the position of the camera relative to the position of the incident. Accordingly, the number of camera feeds to be provided in the camera feed area 315 is reduced compared to including all camera feeds. This makes more efficient use of the screen real estate and shows only the most relevant camera feeds to the operator of the apparatus 400 which may be the referee or may be a person assisting the referee.

In embodiments the selection of the camera feed or feeds may be based on the type of camera capturing the images. in particular, in certain incidents, such as a foul, it is necessary to see the moment of impact (i.e. the moment the foul was committed) and the build up to the foul. This may mean that a camera having a narrow field of view of the incident may be selected to capture the moment of impact of the foul (such as camera 105B in Figure 1) and a camera having a wide field of view (such as camera 105D or 105H in Figure 1) to capture the build up to the foul. Moreover, in the event of a foul, a camera having a narrow field of view may be useful to determine the position of the foul accurately.

Referring to Figure 5, a flow chart 600 showing the operation of the processing circuitry 405 is described. The process starts at step 605 and then moves to step 610 where a plurality of camera feeds is received over the communication circuitry 415. The process moves to step 615 where the camera(s) is/are selected based on their position. The process then moves to step 620 where the camera feed(s) is/are displayed.

The process then ends at step 625.

Numerous modifications and variations of the present disclosure are possible in light of the above teachings. It is therefore to be understood that within the scope of the appended claims, the disclosure may be practiced otherwise than as specifically described herein.

Insofar as embodiments of the disclosure have been described as being implemented, at least in part, by software-controlled data processing apparatus, it will be appreciated that a non-transitory machine-readable medium carrying such software, such as an optical disk, a magnetic disk, semiconductor memory or the like, is also considered to represent an embodiment of the present disclosure.

It will be appreciated that the above description for clarity has described embodiments with reference to different functional units, circuitry and/or processors. However, it will be apparent that any suitable distribution of functionality between different functional units, circuitry and/or processors may be used without detracting from the embodiments.

Described embodiments may be implemented in any suitable form including hardware, software, firmware or any combination of these. Described embodiments may optionally be implemented at least partly as computer software running on one or more data processors and/or digital signal processors. The elements and components of any embodiment may be physically, functionally and logically implemented in any suitable way. Indeed the functionality may be implemented in a single unit, in a plurality of units or as part of other functional units. As such, the disclosed embodiments may be implemented in a single unit or may be physically and functionally distributed between different units, circuitry and/or processors.

Although the present disclosure has been described in connection with some embodiments, it is not intended to be limited to the specific form set forth herein. Additionally, although a feature may appear to be described in connection with particular embodiments, one skilled in the art would recognize that various features of the described embodiments may be combined in any manner suitable to implement the technique.

Embodiments of the present technique can generally described by the following numbered clauses: A method of selecting at least one camera feed for display, comprising: receiving a plurality of camera feeds, each camera feed being provided by a camera located at a different position in a sporting venue but capturing the same incident in a sporting event held at the sporting venue; selecting the at least one camera feed based on the position of the camera relative to the position of theincident; and displaying the selected at least one camera feed.

A method according to clause 1, wherein the step of selecting further comprises: selecting the at least one camera feed based on the field of view of the camera.

A method according to clause I, comprising: determining the type of incident and selecting the at least one camera feed based on the type of

incident and the field of view of the camera.

A method according to any preceding clause wherein the sporting event is a soccer match.

5. A computer program product comprising computer readable instmctions which, when loaded onto a computer, configures the computer to perform a method according to any one of clauses Ito 4.

6 A device for selecting at least one camera feed for display, comprising: communication circuitry configured to receive a plurality of camera feeds, each camera feed being provided by a camera located at a different position in a sporting venue but capturing the same incident in a sporting event held at the sporting venue; and processing circuitry configured to: select the at least one camera feed based on the position of the camera relative to the position of the incident; and provide to a display for displaying the selected at least one camera feed.

7 A device according to clause 6, wherein the step of selecting further comprises: selecting the at least one camera feed based on the field of view of the camera.

8. A device according to clause 6, wherein the processing circuitry is configured to: determine the type of incident and selecting the at least one camera feed based on the type of incident and the field of view of the camera.

A device according to any one of clauses 6, 7 or 8 wherein the sporting event is a soccer match.

Claims

CLAIMSA method of selecting at least one camera feed for display, comprising: receiving a plurality of camera feeds, each camera feed being provided by a camera located at a different position in a sporting venue but capturing the same incident in a sporting event held at the sporting venue; selecting the at least one camera feed based on the position of the camera relative to the position of the incident; and displaying the selected at least one camera feed.
A method according to claim I, wherein the step of selecting further comprises: selecting the at least one camera feed based on the field of view of the camera.
3. A method according to claim 1, comprising: determining the type of incident and selecting the at least one camera feed based on the type ofincident and the field of view of the camera.
A method according to claim 1 wherein the sporting event is a soccer match.
5. A computer program product comprising computer readable instmctions which, when loaded onto a computer, configures the computer to perform a method according to claim 1 A device for selecting at least one camera feed for display, comprising: communication circuitry configured to receive a plurality of camera feeds, each camera feed being provided by a camera located at a different position in a sporting venue but capturing the same incident in a sporting event held at the sporting venue, and processing circuitry configured to: select the at least one camera feed based on the position of the camera relative to the position of the incident; and provide to a display for displaying the selected at least one camera feed.A device according to claim 6, wherein the step of selecting further comprises: selecting the at least one camera feed based on the field of view of the camera.8. A device according to claim 6, wherein the processing circuitry is configured to: determine the type of incident and selecting the at least one camera feed based on the type ofincident and the field of view of the camera.A device according to claim 6, wherein the sporting event is a soccer match.