EP3891998A1

EP3891998A1 - Method and device for automatically evaluating and providing video signals of an event

Info

Publication number: EP3891998A1
Application number: EP19821249.0A
Authority: EP
Inventors: Philipp Lawo
Original assignee: Lawo Holding AG
Current assignee: Lawo Holding AG
Priority date: 2018-12-05
Filing date: 2019-12-04
Publication date: 2021-10-13
Also published as: WO2020114623A1; US11689691B2; DE102018009571A1; US20220103779A1

Abstract

The invention relates to a method and a device for evaluating and providing video signals of an event. A uniform time signal is assigned to recorded video signals of the event, to acquired camera parameters and to metadata, whereupon an automatic evaluation of the video signals is carried out as a function of a user input in a data processing device. In this way, a desired part of the video signals can be provided to the user as a function of the evaluation.

Description

Method and device for the automatic evaluation and provision of video signals of an event

The invention relates to a method and a device for the automatic evaluation and provision of video signals of an event.

Events in the sense of the invention are, for example

Sports events such as ball games, races and races or the like, in which cameras are used to record events at certain, also changing locations, such as, for example, ball positions, shots on goal, overtaking, target runs or the like. Several cameras are usually used to record what is happening at different locations, from different positions and from different angles. All video signals recorded by the cameras are recorded in the

Usually in a control room, as well as in a broadcast car, in which a responsible person selects the video signal to be transmitted to a TV station from the multitude of individual video signals from the cameras.

BESTAT1GUN5SK0PIE The video signals that are not sent are usually lost.

Interest in video signals that have not been sent has been growing for some time. For example, clubs can use these video signals, which show their players or opponents, to assess their individual performance. During races or races, the trainers or racing teams are also interested in all pictures of their runners, cyclists, racing cars etc. Therefore, every recorded video signal, especially during sporting events, can be relevant for a corresponding group of users. In order to filter out the relevant part of a video signal for various user groups, there is the option of manually checking and cutting all video signals, which, however, requires immense effort. In addition to the purely manual selection of video signals, there are also efforts to develop technical processes for their evaluation. These methods analyze the video signals using image processing algorithms to detect objects in the event, for example vehicles and / or players. However, these processes are complex, slow and expensive, so that they are currently only of limited economic viability.

It is therefore the object of the invention, while avoiding the disadvantages of the prior art, to provide an improved possibility for providing individual video material of an event.

The object of the invention is achieved by a generic method with the following steps: the video signals associated with the time signal, camera parameters, meta Data and / or the provided part of the video signals are stored on a data carrier.

Detecting at least one time signal;

Recording video signals by means of at least one camera and automatic assignment of the time signal or signals to the video signals;

Detection of local camera parameters at the location of the camera and automatic assignment of the time signal or signals to the camera parameters, the camera parameters being at least one parameter of the group: position of the camera, acceleration of the camera, orientation of the camera, camera angle kel, magnetic field, field of view, air pressure, volume, brightness, time, current power consumption; Acquisition of meta data by means of at least one meta sensor and automatic assignment of the time signal (s) to the meta data, the meta data being at least one parameter of the group: geographic data, object positions, shipment data, object-specific data , Statistics, databases, local volume, user-defined parameters;

Transmission of the video signals, camera parameters and meta data assigned to the time signals to a data processing device; Automatic evaluation of the video signals assigned to the time signals depending on the camera parameters assigned to the time signal, the meta data assigned to the time signal and by user input and provision of at least part of the video Signals depending on the evaluation. The stated object is further achieved by a device according to the gat, with data acquisition devices such as a camera for recording video signals, at least one camera sensor for recording local camera parameters, at least one meta sensor for recording meta data and a data processing device for receiving the video signals, camera parameters and meta data assigned to a time signal, for evaluating the video signals and for providing at least some of the video signals, the camera, the camera sensor and the meta sensor is connected to the data processing device and the camera sensor is connected to the camera.

The invention is based on the basic consideration that, as an alternative to the known image processing algorithms, the video signals are evaluated as a function of the camera parameters and as a function of the meta data. By means of the common time signals, the video signals, the camera parameters and the meta data can be clearly assigned to one another and synchronized. This in particular enables automatic evaluation of the video signals associated with the time signal on the basis of the camera parameters and the meta data, so that the part of the video signals of interest can be made available to a user.

The time signal in the sense of the invention is, for example, a time indication, in particular the time at the location of the camera, measured in milliseconds. Alternatively or additionally, the time signal can be designed as a stopwatch and have an incremental counter. The time signal is preferably designed such that each frame of the video Signal can be assigned a unique time value. In this respect, the time signal has the function of a clear time stamp.

Camera parameters in the sense of the invention are parameters which characterize properties of the assigned camera, for example the currently set camera angle, the inclination and the position of the camera, the latter being measurable by means of a GPS sensor. To record the camera parameters, at least one camera sensor is provided, which can be equipped, for example, to record the camera angle as a gyroscope and / or to record the camera orientation as an (electronic) compass. The camera parameters are preferably assigned to them simultaneously with the recording of the video signals.

For the purposes of the invention, meta data are, in particular, parameters of the event. A football game is, for example, the current position of the ball and / or the current positions of the players, which can be recorded using common tracking methods. Current scores are also meta data. In a race, meta data are, for example, the current positions of the drivers or vehicles and / or their current positions. The meta data is determined, for example, using common tracking methods, interfaces and / or using GPS sensors. However, metadata can also be broadcast data that provide information as to whether a certain part of the video signal was broadcast, for example, as part of a television broadcast. Meta data in the sense of the invention are also user-defined parameters such as players and / or vehicle names, individual statistics and other information about players and / or vehicles can be imported from databases, for example, the goal shot rate of a football player. Preferably, the volume information, measured by a microphone, in particular spatially separated from the camera, is meta-data in the sense of the invention. Thus, for example, volume information of a fan curve in a football stage can be recorded as meta data in the sense of the invention.

The invention includes that meta data _] *. - Also - can be captured as a camera via other devices or paths and that metadata, video and audio signals are processed by a central data processing device. It is therefore preferably a - central - data processing device independent of - local - recording devices, such as cameras, for processing metadata. The data of all data sources are thus connected to the common global time and accordingly to one another, blocked or locked relative to one another (locked). This ensures the temporal assignment or connection of all data, namely meta data and video and audio signals.

The video signals, the camera parameters and the meta data can be transmitted to the data processing device by means of cables and / or wirelessly, in which case the transmission can be carried out, for example, using WLAN, Bluetooth and / or radio.

A user input for the evaluation of the video signals according to the invention is, for example, a query for a combination of several criteria, for example the query at what time a particular vehicle was captured by a particular camera.

In one embodiment of the invention, the video signals are recorded using a single camera, the position and camera angle of which are changed during the event and recorded as camera data. For example, volume information of a spectator area is recorded as meta data by means of several external microphones, with a high volume indicating an important event, for example a shot on goal. If the special moments of an event - highlights - are requested by user input, the answer is provided to those parts of the video signal in which the volume of the meta data is significantly increased.

Preferably, the acquisition of the time signal, the acquisition of the video signals, the acquisition of the camera parameters and the acquisition of the meta data are synchronized in time, so that the assignment of the video signals, the camera parameters and the meta -Data for the time signal is simplified. For example, camera parameters and meta data can be recorded whenever a single image of the video signals is recorded. Alternatively or in addition, the video signals of several cameras can be recorded in synchronized time. In particular, the time signal can be assigned to the video signals simultaneously with the recording of the video signals. Analogously, this can apply to the assignment of the time signal to the camera parameters and / or to the meta data.

In a development of the invention, the time signal is recorded and the video signals and are recorded capturing the camera parameters over the entire duration of the event in order to be able to access the complete data record that was generated during the entire event during the automatic evaluation. In addition, the meta data are preferably additionally recorded over the entire duration of the event.

The various data sources preferably work with a global time source and thus a global time such as GPS time, NTP (Network Time Protocol) or PTP (Precision Time Protocol), so that the metadata with image, video or audio signals are central in terms of time can be connected without processing in a detection device. All data are provided with the common global time. The data from all data sources are thus connected to the common global part and accordingly to one another, blocked or locked relative to one another (locked). This ensures the temporal assignment or connection of all data, namely meta data and video and audio signals.

However, the meta data can only be acquired if a parameter of the meta data falls below and / or exceeds a user-defined limit. This avoids the accumulation of too much unused data. For example, a local volume can only be recorded if the sound level is above a user-defined limit. A high volume may indicate a significant event, such as a foul or overtaking. In a development of the invention, the steps of evaluating and providing the part of the video signals take place during the event, so that the part of the video signals desired by the user can be made available during the event, in particular continuously. In one example, there is a user request for the highlights of the event. The continuous evaluation during the event determines at what time signal an increased volume is detected in a fan area, which indicates a significant highlight. By determining the time signals of all highlights, the corresponding parts of the video signals can be made available to the user before the end of the event.

Further meta data are preferably generated when the video signals are evaluated. In this way, object or person-related statistics can be created or supplemented. This step can be carried out automatically, so that the newly created statistics can be available again as meta data when evaluating the video signals. The video signals, the camera parameters, the meta data and / or the part of the video signals provided can be stored on a data carrier, preferably in the form of a database, so that archiving and / or later evaluation is possible.

In the device according to the invention, the meta sensor can be provided spatially separated from the camera sensor. Furthermore, each camera is preferably assigned a camera sensor, which is integrated in particular with the camera assigned to it. In a development of the invention, at least one camera is arranged on a missile, in particular on a drone, so that the camera can be moved quickly and easily.

Further advantages and features of the invention result from the claims and from the following description, in which an embodiment of the invention is explained in detail with reference to the drawing. Show:

Fig. 1 The device according to the invention in a schematic representation and

Fig. 2 is a flowchart of the invention

Procedure.

Fig. 1 shows a schematic sketch of a running route 10, for example a running route 10 for a medium distance run, in which runners not shown in FIG. 1 run along the running route 10 while they are being run by two cameras 11, 12 Edge of the running track 10 are filmed. 1 shows a first camera 11 and a second camera 12, which are arranged at different positions on the edge of the running route 10. During the run, the first camera 11 records a first video signal 15 and the second camera 12 records a second video signal 16, which is outlined in the flowchart in FIG. 2.

Both cameras 11, 12 are each provided with an integrated camera sensor 13, 14, the first camera sensor 13 with the first camera 11 and the second camera Sensor 14 are connected to the second camera 12. The first camera sensor 13 detects local camera parameters of the first camera 11 during the run. In the example of FIG. 1, local camera parameters are the geographical position of the camera, its orientation and its camera angle. The geographic position of the camera is measured with a GPS sensor, the orientation with an electrical compass and the camera angle with an electrical gyroscope in combination with a software interface to the camera. The GPS sensor, the electrical compass, the electrical gyroscope and the software interface are integrally formed as the first camera sensor 13, which outputs the recorded camera parameters via a further interface. Like the first camera sensor 13, the second camera sensor 14 detects the local camera parameters of the second camera 12 during the run.

For example, in FIG. 1 the first camera 11 has a first camera angle 17, which in particular detects a curve 18 of the racetrack 10 and is larger than a second camera angle 19 of the second camera 12. The second camera angle 19 of the second camera 12 aligned to a target area 20 of the running route 10. In the example shown, the camera angle 17, 19 denotes the geographical area that is captured by the camera 11, 12. Since the second camera angle 19 is smaller than the first camera angle 17, an enlarged image is obtained in order to be able to better judge which of the runners is the first to cross the target area 20. The camera angles 17, 19 of the cameras 11, 12 can be changed over time and are continuously recorded as camera parameters by the cameras 11, 12 assigned to the cameras 11, 12, respectively. The runners, not shown in FIG. 1, are each provided with a meta sensor 21 in the form of a GPS sensor in order to record the geographic positions of the runners on the running route 10 at any time during the run. By way of example, a GPS sensor 21 is arranged in the right-hand area of FIG. 1. In addition, a second meta sensor 22 in the form of a microphone is arranged on the left-hand side of FIG. 1 in order to measure the volume of the spectators during the run.

The cameras 11, 12, the camera sensors 13, 14 and the metal sensors 21, 22 are each connected to a data processing device 23, the connections in FIG. 1 being represented by connecting lines 24, but alternatively also designed wirelessly could be.

The method according to the invention will be explained with reference to FIGS. 1 and 2, with FIG. 2 showing a schematic flow diagram.

During the run, a continuous time signal from a global time system such as GPS time, NTPO or PTP is recorded in every data acquisition device, such as every camera, in particular locally, from the start signal, which preferably indicates the time in milliseconds and consequently as uniform timestamp. This process step is represented by A in FIG. 2.

In addition, the two cameras 11, 12 of FIG. 1 each take continuous video signals 15 during the run,

16 on. Each frame of the video signals 15, 16 is automatically assigned the respective time signal. This process step is marked on the upper left side of FIG. 2 as B. At the same time as the video signals 15, 16 are recorded, meta data, such as, in particular, the positions, orientations and camera angles of the two cameras 11, 12 are recorded as camera parameters by the camera sensors 13, 14 and these automatically also receive the corresponding parameters global time signal assigned (section C).

At the same time, the GPS sensor 21 continuously detects the current position of the runner assigned to it and the microphone 22 detects the current volume of the audience. Both meta data are automatically assigned the current time signal when they are captured by the meta sensors 21, 22 (section D).

During the next process step E, the video signals, camera data and meta data associated with the time signal are transmitted to the data processing device 23 during the event. In the example shown in FIGS. 1 and 2, the runner's trainer is interested in the performance during the run and therefore makes a user input to the data processing device 23 in a next method step F by requesting video signals in which this particular runner is to see is. This user input is registered in the data processing device 23, so that the recorded video signals 15, 16 are analyzed as to whether the runner can be seen in the video signals 15, 16, see. Method step G of FIG. 2.

For the first camera 11, this is the case, for example, when the geographic position of the runner, which is continuously detected by the GPS sensor 21, from the first camera angle 17 is covered. In this case, the data processing device 23 only provides the part of the first video signal 15 in which the runner can be seen there. The procedure is analogous with the automatic evaluation of the second video signals 16 from the second camera 12. The evaluation of the video signals 15, 16 takes place during the event and simultaneously for all video signals 15, 16. In a last method step H of FIG. 2, the data processing device 23 provides the user with the desired parts of the video signals 15, 16 riding on which the runner can be seen.

In a second example, the user input is a request from a broadcaster for the highlights of the race. This user input is interpreted by the data processing device 23 in such a way that time signals are searched in which the microphone 22 at the edge of the running route 10 has detected significantly high volumes as meta data. This indicates a particularly significant event. After the data processing device 23 has determined the time signals in which high volumes were measured, the first video signals 15 of the first camera 11 assigned to the time signals are determined, since the first camera 11 is arranged closest to the microphone 22 . The rest of the evaluation and provision of the desired part of the video signals 15, 16 are carried out analogously to the previous example. In this way, who provides the user with the highlights of the event.

Claims

1. Method for the automatic evaluation and provision of video signals (15, 16) of an event, with the following steps:

- A: detection of at least one time signal;

- B: Recording video signals (15, 16) using

at least one camera (11, 12) and automatic assignment of the time signal or signals to the video signals (15, 16);

- C: Detection of local camera parameters (17, 19) at the location of the camera (11, 12) and automatic assignment of the time or signals to the camera parameters (17, 19), the camera parameters

(17, 19) at least one parameter of the group are: position of the camera, acceleration of the camera, orientation of the camera, camera angle, magnetic field, field of view, air pressure, volume, brightness, time, current power consumption;

- D: Acquisition of meta data using at least one

Meta sensors (21, 22) and automatic assignment of the time signal (s) to the meta data (28), the meta data being at least one parameter of the group: geographic data, object positions, shipment data, object-specific data, Statistics, databases, local volume, user-defined parameters;

- E: transmission of the one or more time signals assigned video signals (15, 16), camera parameters (17, 19) and meta data to a data processing device (23); - F: automatic evaluation of the time or

Video signals (15, 16) assigned to signals as a function of the camera parameters (17, 19) assigned to the time signal, of the meta data assigned to the time signal and of a user input and

- G: Provision of at least some of the video signals (15, 16) depending on the evaluation.

2. The method according to claim 1, characterized in that such data, such as meta, video, image and audio data, an immediately inaccurate time signal of a global time system, such as GPS time, NTP or PTP, is provided.

3. The method according to claim 1 or 2, characterized in that the detection of the time signal (25), the recording of the video signals (26), the detection of the camera parameters (17, 19) (27) and capturing the meta

Data (28) are synchronized in time.

4. The method according to any one of claims 1 to 3, characterized in that the detection of the time signal (A) and the recording of the video signals (B) and / or the detection of the camera parameters (17, 19) (C ) over the entire duration of the event.

5. The method according to claim 4, characterized in

that the acquisition of the meta data (D) takes place over the entire duration of the event.

6. The method according to any one of claims 1 to 4, characterized in that the detection of the meta data (D) takes place only when a parameter of the meta data falls below and / or exceeds a user-defined limit.

7. The method according to any one of claims 1 to 6, characterized in that the steps of evaluating (F) and providing (G) the part of the video signals (15, 16) take place during the event.

8. The method according to any one of claims 1 to 7, characterized in that during the evaluation (F) of the video signals (15, 16) additional meta data is generated.

9. The method according to any one of claims 1 to 8, characterized in that the video signals (15, 16) associated with the time signal, camera parameters (17, 19), meta data and / or the part provided Video signals (15, 16) are stored on a data carrier.

10. The device, in particular for carrying out a method according to claims 1 to 9, with data acquisition devices, such as at least one camera (11, 12) for recording video signals (15, 16), at least one camera sensor (13 , 14) for the detection of local

Camera parameters (17, 19), at least one meta sensor (21, 22) for acquiring meta data and with a data processing device (23) for receiving the video signals (15, 16) associated with a time signal, camera -Parameters (17, 19) and meta data to Evaluation of the video signals (15, 16) and to provide at least some of the video signals (15, 16), the camera (11, 12), the camera sensor (13, 14) and the meta Sensor (21, 22) are connected to the data processing device (23) and the camera sensor (13, 14) is connected to the camera (11, 12).

11. The device according to claim 1, characterized in that the data acquisition devices, such as cameras, have time recording modules for recording a uniform global time such as GPS time, NTP or PTP.

12. The apparatus according to claim 10, characterized in that at least one camera (11, 12) is arranged on a Flugkör by.