WO2019025833A1

WO2019025833A1 - A system and a method for automated filming

Info

Publication number: WO2019025833A1
Application number: PCT/IB2017/054723
Authority: WO
Inventors: Viktor KURAKIN; Karlis DIMZA; Harijs GRINBERGS; Liene CIRULE; Olegs KONCEVOJS; Krisjanis BALODIS; Aleksandrs SOROKINS; Jevgenijs KAIGORODOVS; Richard FOMRATS
Original assignee: Playgineering Systems, Sia
Priority date: 2017-08-02
Filing date: 2017-08-02
Publication date: 2019-02-07

Abstract

Invention relates to a system and a method for automated filming of sporting event. A system comprises one central tracking camera (1) positioned is a central area of a sporting venue (3), two side cameras (4) configured to capture a broadcastable video data of the sporting event, and a video data processing unit (5) operatively connected to the central tracking camera (1) and to both side cameras (4). A method for automated filming of a sporting event comprises the following steps: calibration of the sporting venue (3); receiving a video data stream from the central tracking camera (1) and from the two side cameras (4); identifying players (2) within the sporting venue (3); calculating an average central weight point (222) for group of identified players (2); and switching between cameras to provide a broadcast.

Description

A SYSTEM AND A METHOD FOR AUTOMATED FILMING Field of the Invention Invention relates to a system and a method for automated filming of sporting event. Background of the Invention

Recording and broadcasting sporting events is a very common practice at the professional level. Spectators are able to experience events, which they could not otherwise attend in person. This opens up the event to a much wider audience, increasing the potential viewership and benefitting both the team and league.

Typically, a quality broadcast of a sporting event requires a significant amount of equipment and manpower. Depending on the type of event, ten or more cameras may be spread out throughout the venue, many of which must be manually operated by a human cameraman to follow the game action. The feeds from each camera are received at a central control room where several operators select the appropriate feed to best portray the action unfolding in the game. The feed must be edited on the fly to produce a broadcastable product, which spectators can follow on their television, computer, or mobile device. The high cost of producing a quality broadcast makes such a practice unaffordable at lower levels of sport.

Several efforts have been made to develop the broadcasting of sporting events more accessible. Most of these efforts involve automating much of the process in order to reduce the amount of human labor required. Systems have been designed employing computer-controlled cameras to track players and automatically generate a broadcastable feed.

International patent application publication No. WO2015/134724 discloses a method, system, and apparatus for automated video tagging of one or more video streams from one or more video capture devices during team and individual player participation in a sporting event, training or testing activity.

United states patent application publication No. 2015/0221096 discloses a method for automatically tracking multiple objects from a sequence of video images that may extract raw data about participating elements in a sporting, or other event, in a way that does not interfere with the actual participating elements in the event. These data may be collected in real time and may include accounting for players moving fast and unpredictably, colliding with and occluding each other, and getting in and out of the playing field. The video sequence, captured by a suitable sensor, may be processed by a suitably programmed general purpose computing device.

United states patent publication No. 9,468,850 discloses a system and method optically track players and automatically capture broadcastable video of a sporting event within a venue. The system includes at least one fixed tracking camera, a remote-controlled broadcast camera, and a scoreboard camera, with each of the components being in communication with a processing system. The method involves dividing the venue into zones of interest, optically tracking individual players in the zones of interest, identifying player groups in the zones of interest, determining a zone of interest containing the most players, analysing player activity in the zone of interest with the most players, selecting a current gameplay scenario corresponding to the player activity from a set of predetermined gameplay scenarios, and adjusting the pan, tilt and zoom of a broadcast camera according to the current gameplay scenario.

Summary of the Invention

The aim of the invention is to develop budget friendly and operational costs free live streaming broadcasting system, which is also adaptive for different sports and arenas or sporting venues. The aim is reached by developing a system for automated filming of a sporting event, wherein the system comprises one central tracking camera, two broadcasting side cameras positioned on each side of the central tracking camera, and a video processing unit that is operatively connected to the central tracking camera and to both broadcast side cameras.

Moreover, the central tracking camera and both broadcast side cameras are fixedly mounted. Fixedly mounted cameras means less investment in complicated systems allowing to pan and tilt said cameras. In the meantime, it reduces ability of camera to pan and tilt in order to follow events on a sporting venue. The following invention overcomes said disadvantages creating a system with low costs of hardware and its operation, but in the meantime keeping all the advantages of adaptive filming - tracking necessary game elements (players, ball, puck, etc.) for providing fully-featured broadcasting. The system comprises one central tracking camera positioned is a central area of a sporting venue and configured to capture tracking video data of players on the sporting venue in a tracking mode and to capture a broadcastable video data of the sporting event on a sporting venue in a broadcast mode. The central tracking camera is configured to function in both modes simultaneously, the central tracking camera captures the tracking video data and the broadcastable video data from all the sporting venue. The central camera in one embodiment may be FullHD standard camera or in another embodiment a 4K UHD standard camera.

Each from two side cameras is configured to capture a broadcastable video data of the sporting event. Both broadcast side cameras are positioned on each side of the central tracking camera so that one side camera captures the broadcastable video data from one side of a sporting venue and another side camera captures the broadcastable video data from another side of the sporting venue. The system further comprises a video data processing unit operatively connected to the central tracking camera and to both side cameras. The video data processing unit is configured to calibrate the sporting venue by marking two zones of interest. The user of the system is able to mark a designated area of a sporting venue, therefore indicating a first zone of interest that covers a zone of the sporting venue to be captured by the two side cameras and the central tracking camera in the broadcast mode, and a second zone of interest that covers a zone of the sporting venue to be captured by the central tracking camera in the tracking mode. Both zones of interest are set by using video data received from the central tracking camera.

The video data processing unit is further configured to receive a video data stream from the central tracking camera and from the two side cameras and from said video data stream of the central tracking camera track players within the second zone of interest by determining respective positions of the individual players and by assigning a central weight point to each individual player and by calculating an average central weight point from all previously assigned central weight points of individual players. And based on a respective position of the average central weight point from all the previously assigned central weight points of individual players, the video date processing unit is configured to determine a location of the average central weight point within the sporting venue or within the second zone of interest that is filmed by the central tracking camera. In result of all aforementioned steps, the video data processing unit is configured to switch to and output to the broadcast the broadcastable video data of the camera in which field of capture the average central weight point is located and adjust digital pan, tilt or zoom of the side camera or of tracking camera in the broadcast mode. In result, the system provides a broadcasting or streaming of the sporting event where players and ball or puck can be followed in a manner sufficient for all featured commercial broadcast.

The video data processing unit further comprises a camera switch point module, a sensitivity module, a centre line setting module and a frame-skipping module.

The camera switch point module is configured to adjust a threshold on the sporting venue at which the output broadcast video data is switched from one camera to another camera, especially from the one side camera to another side camera, and from the side camera to the central tracking camera and vice versa in the broadcasting mode.

The sensitivity module is configured to adjust a sensitivity threshold creating a smaller or more adjusted second zone of interest. The players outside said adjusted second zone of interest are not taken into account upon calculation the average central weight point, therefore easing a calculation of said average central weight point.

The sensitivity module is further configured to adjust the second zone of interest by using a sensitivity threshold on a right side of the sporting venue and another sensitivity threshold on a left side of the sporting venue, therefore restricting the second zone of interest from both sides of the sporting venue, creating adjusted second zone of interest.

The object of the invention is also a method for automated filming of a sporting event. The method comprises the following main steps: calibration of sporting venue; receiving a video data stream from cameras; identifying players within said calibrated sporting venue, calculating average central weight point from identified players within zone of interest, and switching cameras, to where the identified average central weight point is identified, and broadcasting the video data from switched camera. Hence, providing seamless and all featured broadcasting or streaming of sporting event. The calibration step of said method includes two substeps. The first substep includes marking of a first zone of interest that covers a zone of the sporting venue to be captured by the two broadcast cameras and the tracking camera in the broadcast mode. The second substep includes marking of a second zone of interest that covers a zone of the sporting venue to be captured by the tracking camera in the tracking mode in order to identify and track players. Both substeps may be performed by a user of the method.

The next steps of the method includes receiving a video data stream from the tracking camera and from the two broadcast cameras. The received data are processed for identifying players within the second zone of interest from captured tracking video data by performing background subtraction, wherein the tracking video data is obtained from the tracking camera. When player are identified then a calculation of a central weight point for each identified player within the second zone of interest is performed. When all central weight points of each identified player are acquired, then a calculation of an average central weight point from calculated central weight points for each identified player is performed. By identifying a position of the average central weight point in the second zone of interest, a switching to the camera, in which designated area the central weight point is positioned, is performed. This is followed by the output of the video data from said camera to the broadcast. The following video data may include adjustments such as digital zoom, pan and/or tilt. In result of which the outputted video data, including digital zoom, pan and/or tilt, has the same filming quality, as it would be performed manually by skilled video operator. Moreover, zoom limit is adjustable, zoom pan speed is adjustable. It is not limited to only one fixed limit anymore. The aforementioned steps are performed using video data processing unit and its modules. The method further comprises the step of adjusting a threshold on the sporting venue at which the output broadcast video data is switched from one camera to another camera, especially from the one broadcast camera to the another broadcast camera, and from the broadcast camera to the tracking camera in the broadcasting mode and vice versa. The following step is performed by the camera switch point module of the video data processing unit.

The method further comprises the step of adjusting the second zone of interest in such a way that the players outside said adjusted second zone of interest are not taken into account upon calculation the average central weight point. The step of adjusting the second zone of interest further includes adjusting one sensitivity threshold on a right side of the sporting venue and another sensitivity threshold on a left side of the sporting venue, therefore restricting the second zone of interest from both sides of the sporting venue. The following steps are performed by the sensitivity module of the video data processing unit. Brief Description of the Drawings

These and other objects and advantages of the invention and a better understanding of the principles and details of the invention will be evident from the following description taken in connection with the following drawings in which:

Fig. 1 is a schematic of a system for automated filming of a sporting event.

Fig. 2 is a schematic of calibration process by creation of two zones of interest 6 and 7.

Fig. 3 is a schematics illustrating the assigning a central weight point 22 to each individual player 2 and determining a location of the average central weight point 222 within the sporting venue 3.

Fig. 4 is schematics illustrating the determining a location of the average central weight point 222 within the sporting venue 3 using a sensitivity thresholds 9.

Detailed Description of the Invention

Fig. 1 illustrates a system for automated filming of a sporting event, wherein the system comprises one fixedly mounted central tracking camera 1 positioned is a central area of a sporting venue 3 and configured to capture tracking video data of players on the sporting venue

3 in a tracking mode and to capture a broadcastable video data of the sporting event on a sporting venue 3 in a broadcast mode. The system further comprises two side cameras 4 configured to capture a broadcastable video data of the sporting event, wherein both side cameras 4 are positioned on each side of the central tracking camera 1 so that one side camera

4 captures the broadcastable video data from one side 3A of a sporting venue 3 and another side camera 4 captures the broadcastable video data from another side 3B of the sporting venue 3. The system is controlled by a video data processing unit 5 that is operatively connected to the central tracking camera 1 and to both side cameras 4. The video data processing unit 5 further comprises a camera switch point module configured to adjust a threshold 8 on the sporting venue 3 at which the output broadcast video data is switched from one camera 1 and 4 to another camera 1 and 4, especially from the one side camera 4 to another side camera 4, and from the side camera 4 to the central tracking camera 1 and vice versa in the broadcasting mode.

The video data processing unit 5 that is configured to: calibrate the sporting venue 3 by marking two zones of interest 7 and 8 (see Fig. 2). The first zone of interest 6 covers a zone of the sporting venue 3 to be captured by the two side cameras 4 and the central tracking camera 1 in the broadcast mode, and a second zone of interest 7 covers a zone of the sporting venue 3 to be captured by the central tracking camera 1 in the tracking mode. The video data processing unit 5 is further configured to receive a video data stream from the central tracking camera 1 and from the two side cameras 4; track players 2 within the second zone of interest 7 from captured tracking video data by determining respective positions of the individual players 2 and by assigning a central weight point 22 to each individual player 2 and by calculating an average central weight point 222 from all previously assigned central weight points 22 of individual players 2; and based on a respective position of the average central weight point (222) from all the previously assigned central weight points 22 of individual players 2, determine a location of the average central weight point 222 within the sporting venue 3 (see Figs. 3 and 4). At the end the data processing unit 5 is configured to switch to and output to the broadcast the broadcastable video data of the camera 2 and 4, in which field of capture the average central weight point 222 is located, and adjust digital pan, tilt or zoom of the side camera 4 or of tracking camera 1 in the broadcast mode.

The video data processing unit 5 further comprises a sensitivity module configured to adjust a sensitivity threshold 9 within the sporting venue 3 creating a second zone of interest 10, wherein the players 2 outside said adjusted second zone of interest 10 are not taken into account upon calculation the average central weight point 222 (see Fig. 4).

References in the Figures

1 - a central tracking camera

2 - a player;

22 - a central weight point of the player 2;

222 - an average central weight point;

3 - a sporting venue;

4 - a side camera; 5 - a video processing unit;

6 - a first zone of interest;

7 - a second zone of interest;

8 - a threshold;

9 - a sensitivity threshold; and

10 - an adjusted second zone of interest.

Claims

A system for automated filming of a sporting event, wherein the system comprises:

— one central tracking camera (1) positioned is a central area of a sporting venue (3) and configured to capture tracking video data of players (2) on the sporting venue (3) in a tracking mode and to capture a broadcastable video data of the sporting event on a sporting venue (3) in a broadcast mode, wherein the central tracking camera (1) is configured to function in both modes simultaneously, wherein the central tracking camera (1) captures the tracking video data and the broadcastable video data from all the sporting venue (3);

— two side cameras (4) configured to capture a broadcastable video data of the sporting event, wherein both side cameras (4) are positioned on each side of the central tracking camera (1) so that one side camera (4) captures the broadcastable video data from one side (3 A) of a sporting venue (3) and another side camera (4) captures the broadcastable video data from another side (3B) of the sporting venue

(3);

— a video data processing unit (5) operatively connected to the central tracking camera (1) and to both side cameras (4), wherein the video data processing unit (5) is configured to:

— calibrate the sporting venue (3) by marking two zones of interest (6, 7), wherein a first zone of interest (6) covers a zone of the sporting venue (3) to be captured by the two side cameras (4) and the central tracking camera (1) in the broadcast mode, and a second zone of interest (7) covers a zone of the sporting venue (3) to be captured by the central tracking camera (1) in the tracking mode;

— receive a video data stream from the central tracking camera (1) and from the two side cameras (4);

— track players (2) within the second zone of interest (7) from captured tracking video data by determining respective positions of the individual players (2) and by assigning a central weight point (22) to each individual player (2) and by calculating an average central weight point (222) from all previously assigned central weight points (22) of individual players (2);

— based on a respective position of the average central weight point (222) from all the previously assigned central weight points (22) of individual players (2), determine a location of the average central weight point (222) within the sporting venue (3); and

— switch to and output to the broadcast the broadcastable video data of the camera (2, 4) in which field of capture the average central weight point (222) is located and adjust digital pan, tilt or zoom of the side camera (4) or of tracking camera (1) in the broadcast mode.

The system for automated filming of a sporting event according to Claim 1, wherein the video data processing unit (5) further comprises a camera switch point module configured to adjust a threshold (8) on the sporting venue (3) at which the output broadcast video data is switched from one camera (1, 4) to another camera (1, 4), especially from the one side camera (4) to another side camera (4), and from the side camera (4) to the central tracking camera (1) and vice versa in the broadcasting mode.

The system for automated filming of a sporting event according to Claim 1 or 2, wherein the video data processing unit (5) further comprises a sensitivity module configured to adjust a sensitivity threshold (9) creating a second zone of interest (10), wherein the players (2) outside said adjusted second zone of interest (10) are not taken into account upon calculation the average central weight point (222).

The system for automated filming of a sporting event according to Claim 3, wherein the sensitivity module is further configured to adjust the second zone of interest (10) by using a sensitivity threshold (9) on a right side of the sporting venue and a sensitivity threshold

(9) on a left side of the sporting venue (3), therefore restricting the second zone of interest

(10) from both sides of the sporting venue (3).

The system for automated filming of a sporting event according to any of Claims, wherein the central tracking camera (1) is fixedly mounted.

The system for automated filming of a sporting event according to any of Claims, wherein the side camera (4) is fixedly mounted.

A method for automated filming of a sporting event, wherein the method comprises the following steps: a) calibration, wherein said calibration includes two substeps: aa) marking of a first zone of interest (6) that covers a zone of the sporting venue (3) to be captured by the two side cameras (4) and the central tracking camera (1) in the broadcast mode; and

aaa) marking of a second zone of interest (7) that covers a zone of the sporting venue (3) to be captured by the central tracking camera (1) in the tracking mode in order to identify and track players (2); b) receiving a video data stream from the central tracking camera (1) and from the two side cameras (4);

c) identifying players (2) within the second zone of interest (7) from captured tracking video data by performing background subtraction, wherein the tracking video data is obtained from the central tracking camera (1);

d) calculating a central weight point (22) for each identified player (2) within the second zone of interest (7);

e) calculating an average central weight point (222) from calculated central weight points (22) for each identified player (2);

f) switching to and output to the broadcast the broadcastable video data of the camera (1, 4) in which field of capture the average central weight point (222) is located and adjust digital zoom, pan and/or tilt of the side camera (4) or of the central tracking camera (1) in the broadcast mode.

The method for automated filming of a sporting event according to Claim 7, wherein the method further comprises the step of adjusting a threshold (8) on the sporting venue (3) at which the output broadcast video data is switched from one camera (1, 4) to another camera (1, 4), especially from the one side camera (4) to the another side camera (4), and from the side camera (4) to the tracking camera (1) in the broadcasting mode and vice versa.

The method for automated filming of a sporting event according to Claim 7 or 8, wherein method further comprises the step of adjusting the second zone of interest (7) in such a way that the players (2) outside said adjusted second zone of interest (7) are not taken into account upon calculation the average central weight point (222).

10. The method for automated filming of a sporting event according to Claim 9, wherein the step of adjusting the second zone of interest (7) further includes adjusting one sensitivity threshold (9) on a right side of the sporting venue (3) and another sensitivity threshold (9) on a left side of the sporting venue (3), therefore restricting the second zone of interest (7) from both sides of the sporting venue (3).

11. The method of automated filming of sport event according to Claim 7, wherein both substeps aa) and aaa) of the step a) are performed by a user of the method.