CN115002510A

CN115002510A - Game viewing system, method, device and storage medium

Info

Publication number: CN115002510A
Application number: CN202210611560.5A
Authority: CN
Inventors: 郑少林; 王磊; 侯欣如; 栾青; 杨奇勇
Original assignee: Beijing National Aquatics Center Co ltd; Beijing Sensetime Technology Development Co Ltd
Current assignee: Beijing National Aquatics Center Co ltd; Beijing Sensetime Technology Development Co Ltd
Priority date: 2022-05-31
Filing date: 2022-05-31
Publication date: 2022-09-02

Abstract

The present disclosure provides a competition watching system, a method, a device and a storage medium, wherein the system comprises: a trajectory capture device, an image rendering device and a presentation device; a track capture device for acquiring first video data of at least one track; determining motion information of the curling on each track according to the first video data corresponding to each track; sending the motion information of the curling on each track to image rendering equipment; the image rendering device is used for generating a curling motion track corresponding to each track according to the motion information of the curling on each track; acquiring second video data corresponding to the big book camp on each track; fusing the second video data corresponding to each track with the curling motion track corresponding to the track to generate display data corresponding to the track; transmitting the display data corresponding to each track to display equipment; and the display equipment is used for displaying the display data corresponding to each track.

Description

Match watching system, method, device and storage medium

Technical Field

The present disclosure relates to the field of machine vision technologies, and in particular, to a competition watching system, method, apparatus, and storage medium.

Background

With the improvement of living standard of people, more and more audiences begin to pay attention to and participate in the curling movement. For example, the spectator may watch the curling game on the game site, or on a terminal device such as a television or a computer. However, no matter in the competition field or watching the curling competition on the terminal equipment, the spectators see the instantaneous movement of the curling on the ice surface, so that the spectators cannot deeply know the movement rule of the curling and the tactics and tactics of the battle oppositing two parties in the competition, the competition experience of the spectators is influenced, and the display effect of the curling competition picture is reduced.

Disclosure of Invention

In view of the above, the present disclosure provides at least one contest viewing system, method, apparatus, and storage medium.

In a first aspect, the present disclosure provides a match watching system, comprising: a trajectory capture device, an image rendering device and a presentation device; wherein the image rendering device is connected with the track capturing device and the display device respectively;

the track capturing device is used for acquiring first video data of at least one track; determining motion information of the curling on each track according to the first video data corresponding to each track; sending the motion information of the curling on each track to the image rendering equipment;

the image rendering device is used for generating curling motion tracks corresponding to the tracks according to the motion information of the curling on each track; acquiring second video data corresponding to the big book camp on each track; fusing the second video data corresponding to each track with the curling motion track corresponding to the track to generate display data corresponding to the track; transmitting the display data corresponding to each track to the display equipment;

the display equipment is used for displaying the display data corresponding to each track.

The track capturing equipment in the system can determine the motion information of the curling on the track according to the first video data corresponding to each track; the image rendering equipment can generate curling motion tracks corresponding to the tracks according to the motion information of the curling on each track; fusing the second video data corresponding to each track with the curling motion track corresponding to the track to generate display data corresponding to the track; displaying display data corresponding to each track by using display equipment; the spectator can check the motion trail of the curling in the match picture corresponding to each track and the current match picture in real time on the display equipment, the movement law of the curling can be definitely known through the motion trail, the tactics and tactics of both sides of the match are determined, the display effect of the curling match picture is improved, and the display of the curling match process has flexibility and diversity.

In one possible implementation, the trajectory capture device includes: at least one edge computing server; the curling detection algorithm, the curling tracking algorithm and the picture fusion algorithm are loaded on the edge calculation server;

the edge computing server is used for receiving a plurality of first video data corresponding to each track; for each track, performing target detection on each first video frame in each first video data corresponding to the track by using the curling detection algorithm, and determining curling position information corresponding to the first video frames; and

by utilizing the picture fusion algorithm, in response to the determination of the virtual coordinate system corresponding to the plurality of first video data of the track, converting the curling position information corresponding to the plurality of first video data corresponding to the track to the virtual coordinate system respectively, and generating converted position information corresponding to the plurality of first video data respectively; and determining motion information of the curling on the track based on the converted position information respectively corresponding to the plurality of first video data by using the curling tracking algorithm, wherein the motion information comprises at least one of position information, speed information and acceleration information.

Here, the trajectory capture device may include at least one edge computing server; the curling detection algorithm, the curling tracking algorithm and the picture fusion algorithm are loaded on the edge calculation server, the motion information of the curling can be obtained in real time and efficiently through the edge calculation server, the first video data are detected by utilizing the loaded various algorithms, the motion information of the curling can be accurately obtained, the motion track of the curling can be accurately and efficiently determined according to the motion information of the curling in the subsequent process, the synchronous display of the picture and the track is realized, and the display effect of curling competition is improved.

In a possible implementation, the motion information further includes angular velocity information, and the edge calculation server further loads a key point detection algorithm;

the edge computing server is further configured to: after the position information of the curling corresponding to the first video frames is determined, carrying out key point detection on each first video frame in each first video data corresponding to the track based on the curling position information and the key point detection algorithm, and generating key point information of a curling handle corresponding to the first video frames; and

converting the key point information corresponding to the plurality of first video data corresponding to the track to the virtual coordinate system by using the picture fusion algorithm, and generating converted key point information corresponding to the plurality of first video data; and determining the angular velocity information of the curling on the track based on the converted key point information respectively corresponding to the plurality of first video data by using the curling tracking algorithm.

The edge calculation server is also loaded with a key point detection algorithm, key point information of the curling handle can be accurately determined through the key point detection algorithm, and then the angular speed information of the curling can be determined according to the key point information, so that the rotation process of the curling in motion can be simulated, the movement process of the curling can be simulated more accurately by the movement track of the curling generated subsequently, and the showing effect of curling competition is improved.

In one possible implementation, the trajectory capture device includes: a plurality of first cameras corresponding to each track; any position on each track is shot by a plurality of first cameras with different collection angles;

the first camera is used for collecting first video data of the track corresponding to the first camera according to the set collection angle.

Here, the track capture equipment comprises a plurality of first cameras corresponding to each track, and for any position of each track, the first cameras can be used for comprehensively acquiring first video data corresponding to the track, so that the problem that the movement track of the curling is discontinuous due to incomplete covering and dead angles of the track caused by the first cameras is solved, omnibearing data are provided for subsequently determining the movement track of the curling, and the accuracy of the movement track of the curling is guaranteed.

Meanwhile, considering any position on the track, a shielding condition may occur in the first video data acquired by the camera shooting the position, for example, a sportsman shields the curling, and the movement track of the curling cannot be accurately determined. In order to alleviate the above problem, in the present embodiment, any position on each track is captured by a plurality of first cameras with different capture angles, so that the curling motion track can be determined more accurately by using the first video data captured by the first cameras.

In one possible embodiment, the trajectory capture device further comprises: a core switch and at least one access stratum switch; each access layer switch is connected with at least one first camera, and the access layer switches are connected with the core switch;

the access layer switch is used for receiving first video data collected by the connected first camera and transmitting the received first video data to the core switch;

and the core switch is used for transmitting the received first video data to the edge computing server.

The trajectory capture device may further include a core switch and at least one access stratum switch; the access layer switch is used for receiving first video data collected by the first camera in real time, and the received first video data can be transmitted to the core switch without attenuation. The core switch receives the first video data transmitted by each access layer switch, and the first video data are efficiently transmitted to the edge computing server after being gathered, so that the edge computing server can detect the received first video data and provide data support for subsequently determining the motion track of the curling.

In a possible implementation, the image rendering device includes: a target number of second cameras, wherein the target number matches the number of big book tracks included in the at least one track; the second camera is deployed in a preset area above each track capital camp;

the second camera is used for collecting the second video data in the great camp emmetropic direction.

Considering that the result of curling competition is relevant to the position of curling in the big book camp, the throwing action of the athletes to the curling and other information, the second cameras with the target number can be set for better showing the competition process of the curling competition, and each second camera acquires second video data in the front view direction of the big book camp, so that the showing effect of the curling competition process is improved.

In a possible implementation, the image rendering apparatus further includes: a rendering server corresponding to each track; the rendering server loads an image distortion correction algorithm, a track rendering algorithm and an image track synchronization algorithm;

the rendering server is used for receiving the second video data of the track corresponding to the rendering server and the motion information of the curling on the track; correcting the second video data of the track by using the image distortion correction algorithm to obtain processed second video data corresponding to the track; generating a curling motion track corresponding to the track based on the motion information of curling on the track by utilizing a track rendering algorithm;

and synchronously fusing the processed second video data corresponding to the track and the curling motion track by using the picture track synchronization algorithm to generate the display data corresponding to the track.

In the disclosure, the image rendering device further comprises a rendering server corresponding to each track, the rendering server produces display data corresponding to the tracks through a loaded picture distortion correction algorithm, a track rendering algorithm and a picture track synchronization algorithm, the display data comprise video pictures of a big book game and a synchronous curling motion track, so that audiences can observe the current curling game pictures of the curling game and can also observe the synchronous curling motion track, and the display effect of the curling game is improved.

In a possible implementation, the image rendering apparatus further includes: a signal switching station; the signal switching platform is connected with the rendering server;

the rendering server is further configured to, after the display data corresponding to the track is generated, respond to that a size ratio of the display data is not consistent with a standard ratio indicated by a signal transmission protocol, splice the display data with preset data, and generate spliced data; wherein the size proportion of the spliced data is consistent with the standard proportion; sending the spliced data to the signal switching station by using the signal transmission protocol;

the signal switching station is used for acquiring the display data corresponding to each track from the splicing data corresponding to each track; splicing the display data corresponding to the multiple tracks to obtain target display data; wherein the size proportion of the target display data is consistent with the standard proportion; then, the signal transmission protocol is utilized to send the target display data to the display equipment;

the display device is used for displaying the display data corresponding to each track:

and displaying the received target display data.

The image rendering device further comprises a signal switching table, and the signal switching table can be used for realizing transmission of display data so as to display target display data on the display device and realize display of the curling competition process.

The following descriptions of the effects of the method, the apparatus, and the like refer to the description of the system, and are not repeated here.

In a second aspect, the present disclosure provides a tournament viewing method, comprising:

acquiring first video data of at least one track and second video data corresponding to a big book on each track;

determining motion information of the curling on each track according to the first video data corresponding to each track; generating a curling motion track corresponding to each track according to the motion information of the curling on each track;

fusing the second video data corresponding to each track with the curling motion track corresponding to the track to generate display data corresponding to the track;

and displaying the display data corresponding to each track.

In a third aspect, the present disclosure provides a contest viewing apparatus, comprising:

the system comprises an acquisition module, a display module and a display module, wherein the acquisition module is used for acquiring first video data of at least one track and second video data corresponding to big books on each track;

the determining module is used for determining motion information of the curling on each track according to the first video data corresponding to each track; generating a curling motion track corresponding to each track according to the motion information of the curling on each track;

the generating module is used for fusing the second video data corresponding to each track with the curling motion track corresponding to the track to generate display data corresponding to the track;

and the display module is used for displaying the display data corresponding to each track.

In a fourth aspect, the present disclosure provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the tournament viewing method as described in the second aspect above.

In order to make the aforementioned objects, features and advantages of the present disclosure more comprehensible, preferred embodiments accompanied with figures are described in detail below.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings required for use in the embodiments will be briefly described below, and the drawings herein incorporated in and forming a part of the specification illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the technical solutions of the present disclosure. It is appreciated that the following drawings depict only certain embodiments of the disclosure and are therefore not to be considered limiting of its scope, for those skilled in the art will be able to derive additional related drawings therefrom without the benefit of the inventive faculty.

FIG. 1 is a schematic diagram illustrating an architecture of a tournament viewing system according to an embodiment of the disclosure;

FIG. 2 is a schematic interface diagram of a display device provided by an embodiment of the present disclosure;

FIG. 3 is a schematic diagram illustrating an architecture of a trajectory capture device provided by an embodiment of the present disclosure;

fig. 4 is a schematic structural diagram of a curling provided by the embodiment of the disclosure;

FIG. 5 is a schematic diagram illustrating an architecture of an image rendering apparatus provided by an embodiment of the present disclosure;

FIG. 6 is a schematic diagram illustrating splicing data provided by an embodiment of the present disclosure;

FIG. 7 is a schematic diagram illustrating an architecture of another image rendering device provided by the embodiment of the disclosure;

FIG. 8 is an architectural diagram illustrating another tournament viewing system provided by embodiments of the present disclosure;

FIG. 9 is a flow chart illustrating a method for viewing a tournament provided by an embodiment of the present disclosure;

fig. 10 shows an architecture diagram of a competition watching device provided by an embodiment of the present disclosure.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present disclosure more clear, the technical solutions of the embodiments of the present disclosure will be described clearly and completely with reference to the drawings in the embodiments of the present disclosure, and it is obvious that the described embodiments are only a part of the embodiments of the present disclosure, not all of the embodiments. The components of the embodiments of the present disclosure, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present disclosure, presented in the figures, is not intended to limit the scope of the claimed disclosure, but is merely representative of selected embodiments of the disclosure. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the disclosure without making creative efforts, shall fall within the protection scope of the disclosure.

As the curling game is watched on the terminal equipment no matter on the game site, the spectators see the instantaneous motion of the curling on the ice surface, so that the spectators cannot deeply know the motion rule of the curling and the tactics and tactics of the battle opponent in the game, the match experience of the spectators is influenced, and the display effect of the curling game picture is reduced.

In order to alleviate the above problems, embodiments of the present disclosure provide a competition viewing system, a method, an apparatus, and a storage medium, where a track capture device in the competition viewing system can determine motion information of a curling on a track according to first video data corresponding to each track; the image rendering equipment can generate curling motion tracks corresponding to the tracks according to the motion information of the curling on each track; fusing the second video data corresponding to each track with the curling motion track corresponding to the track to generate display data corresponding to the track; displaying the display data corresponding to each track by using display equipment; the spectator can check the motion trail of the curling in the match picture corresponding to each track and the current match picture in real time on the display equipment, the movement law of the curling can be definitely known through the motion trail, the tactics and tactics of both sides of the match are determined, the display effect of the curling match picture is improved, and the display of the curling match process has flexibility and diversity.

It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.

Referring to fig. 1, a schematic diagram of an architecture of a tournament viewing system provided in an embodiment of the present disclosure is shown, where the system includes: a trajectory capture device 11, an image rendering device 12 and a presentation device 13; wherein the image rendering device 12 is connected to the trajectory capture device 11 and the presentation device 13, respectively.

The track capturing device 11 is used for acquiring first video data of at least one track; determining motion information of the curling on each track according to the first video data corresponding to each track; and send the motion information of the curling on each track to the image rendering device 12.

The image rendering device 12 is configured to generate curling motion tracks corresponding to the tracks according to the motion information of curling on each track; acquiring second video data corresponding to the big book camp on each track; fusing the second video data corresponding to each track with the curling motion track corresponding to the track to generate display data corresponding to the track; and transmitting the display data corresponding to each track to the display equipment 13.

The display device 13 is configured to display the display data corresponding to each track.

In implementation, the track capturing device can acquire the first video data of each track captured by the multiple cameras. And detecting the first video data of each track, and determining the motion information of the curling on the first video data. For example, the motion information may comprise at least one of position information, speed information, acceleration information, angular velocity information of the curling on respective video frames of the first video data. And then sends the motion information of the curling on each track to the image rendering device 12.

The image rendering equipment can acquire second video data corresponding to the big book camp on each track; and the areas in the concentric circles at the two ends of each track are capital camps corresponding to the tracks. And after receiving the motion information of the curling on each track, fitting and generating the curling motion track corresponding to the track according to the motion information of the curling on the track. For example, a motion curve of the curling can be generated by fitting according to the position information indicated by the motion information, and the motion curve is determined as a curling motion track. Or after the motion curve of the curling is generated through fitting, determining the speed information and the angle information of at least one track point, and determining the motion curve carrying the speed information and the acceleration information as the curling motion track.

Furthermore, the image rendering device may fuse the second video data corresponding to each track and the curling motion track corresponding to the track to generate display data corresponding to the track. The presentation data may be fused video data. For example, for each second video frame in the second video data, a current curling motion track corresponding to the second video frame may be determined, and the second video frame and the current curling motion track may be fused to obtain a fused image; the plurality of fused images form display data corresponding to the track.

The image rendering device may transmit the generated display data corresponding to each track to the display device, and display the display data on the display device. Wherein the display device may be a large screen. For example, where the track is four, the display device may be a 170 "large screen (approximately 23.4 metres wide and 7 metres high) so that the large screen can be positioned in a 1: and the large book camping pictures of the four tracks are displayed at the ratio of 1.

When the number of tracks is four, see the interface schematic of the display device shown in fig. 2. The display data comprises second video data corresponding to the big book camp of the track and a curling motion track corresponding to the track.

The following illustrates the architecture of the trajectory capture device.

In one possible implementation, referring to FIG. 3, the trajectory capture device 11 comprises: a plurality of first cameras 111 corresponding to each track; any position on each track is shot by a plurality of first cameras with different collection angles. The first camera is used for collecting first video data of the track corresponding to the first camera according to the set collection angle.

Considering that each track in the curling game is long, a plurality of first cameras can be arranged for each track in order to comprehensively and more specifically collect the game picture of each track. For example, 10 first cameras may be disposed on each track, and the 10 first cameras may be disposed at a plurality of installation locations. Any position on each track is shot by a plurality of first cameras with different collection angles, so that any position on the track can be shot by the first cameras with different collection angles. For example, for a position a of the track, the first camera 1 and the second camera 2 can both acquire video data at the position a. When the sportsman shelters from the condition to curling appearing in the first video data that first camera 1 was gathered, clear first video data can be shot to different collection angles to first camera 2 to alleviate the above-mentioned condition of sheltering from. Or any position on each track is shot by a plurality of first cameras with different collection angles, and when any first camera breaks down, other first cameras can also collect the first video data of the position of the track in real time.

In an alternative embodiment, the trajectory capture device 11 further comprises: a core switch 112 and at least one access layer switch 113; each access layer switch 113 is connected to at least one first camera 111, and the access layer switch 113 is connected to the core switch 112.

During implementation, the access layer switch and the first camera can be connected through a network cable, and the access layer switch and the core switch can be connected through optical fibers. Because the network cable has length limitation, in order to enable each first camera to have a connected access layer switch, the number and the position of the access layer switches can be determined according to the position of the first camera deployed in the competition field during specific implementation.

The access layer switch 113 is configured to receive first video data acquired by the connected first camera 111, and transmit the received first video data to the core switch 112; the core switch 112 is configured to transmit the received first video data to the edge computing server.

The access layer switch is responsible for providing a network node for the transmission of the video signal (i.e. the first video data) of each connected first camera, i.e. the access layer switch is capable of transmitting the received first video data to the core switch.

The number of the core switches may be one, and after each access layer switch transmits the received first video data to the core switch, the core switch aggregates the video signals (i.e., the first video data) collected by each first camera, and transmits each first video data to the edge computing server.

Here, the trajectory capture device may further include a core switch and at least one access layer switch; the access layer switch is used for receiving first video data collected by the first camera in real time, and the received first video data can be transmitted to the core switch without attenuation. The core switch receives the first video data transmitted by each access layer switch, and the first video data are efficiently transmitted to the edge computing server after being gathered, so that the edge computing server can detect the received first video data and provide data support for subsequently determining the motion track of the curling.

In an alternative embodiment, the trajectory capture device 11 comprises: at least one edge computing server 114; the edge calculation server 114 is loaded with a curling detection algorithm, a curling tracking algorithm and a picture fusion algorithm. The number of the edge computing servers can be set according to actual requirements. For example, when there are four tracks, in order to improve the processing efficiency, two edge calculation servers may be provided, the edge calculation server 1 is used to process the first video data corresponding to the track a and the track B, and the edge calculation server 2 is used to process the first video data corresponding to the track C and the track D.

The edge computing server 114 is configured to receive a plurality of first video data corresponding to each track; for each track, performing target detection on each first video frame in each first video data corresponding to the track by using the curling detection algorithm, and determining curling position information corresponding to the first video frames; by utilizing the picture fusion algorithm, responding to a virtual coordinate system corresponding to the first video data of the track, converting the curling position information corresponding to the first video data of the track to the virtual coordinate system, and generating converted position information corresponding to the first video data; and determining the motion information of the curling on the track by using the curling tracking algorithm based on the converted position information corresponding to the plurality of first video data respectively, wherein the motion information comprises at least one of position information, speed information and acceleration information. The motion information may be information in a real game scene.

The curling detection algorithm may include a trained neural network for curling detection. The curling tracking algorithm may include an algorithm that is capable of tracking curling. The picture fusion algorithm may include an algorithm capable of implementing virtual coordinate system construction and coordinate system conversion. Here, the virtual coordinate system may be a coordinate system constructed with an origin of any pixel position in any of the first video data.

In implementation, in response to receiving a plurality of pieces of first video data corresponding to each track, the edge computing server performs target detection on each first video frame in each piece of first video data corresponding to each track by using a curling detection algorithm, and determines curling position information corresponding to each first video frame, that is, position information of a curling included in each first video frame in a pixel coordinate system corresponding to the first video frame is obtained.

And the edge computing server constructs a virtual coordinate system corresponding to a plurality of first video data of the track in advance or in real time by using a picture fusion algorithm. For example, the picture fusion algorithm may associate the plurality of first video data according to an intersection image region between the plurality of first video data to obtain the associated large-range video data. And constructing a virtual coordinate system by taking any position on the large-range video data as an origin. And determining conversion relation information between the pixel coordinate system and the virtual coordinate system corresponding to each first video data. Furthermore, the image fusion algorithm may convert, based on the conversion relationship information between the pixel coordinate system and the virtual coordinate system corresponding to each piece of the first video data, curling position information corresponding to each piece of the first video data corresponding to the track to the virtual coordinate system, and generate converted position information corresponding to each piece of the first video data.

And the edge calculation server determines the motion information of the curling on the track based on the converted position information corresponding to the first video data by using a curling tracking algorithm. For example, the curling tracking algorithm may first map the converted position information corresponding to the plurality of first video data to the world coordinate system from the virtual coordinate system based on a conversion matrix between the determined virtual coordinate system and the constructed world coordinate system, to obtain the position information corresponding to the plurality of first video data in the world coordinate system, to obtain the position information in the motion information.

When the speed information is included in the movement information, the speed information of the curling can be determined based on the position information of the curling in the world coordinate system and the movement time length between different position information. When the motion information comprises acceleration information, the acceleration information can be further determined based on the speed information corresponding to the curling and the interval duration between different speeds.

In the disclosed embodiment, the track capture device may include at least one edge computing server; the curling detection algorithm, the curling tracking algorithm and the picture fusion algorithm are loaded on the edge calculation server, the motion information of the curling can be obtained in real time and efficiently through the edge calculation server, the loaded multiple algorithms are used for detecting the first video data, the motion information of the curling can be accurately obtained, so that the motion track of the curling can be accurately and efficiently determined according to the motion information of the curling in the later period, the synchronous display of the picture and the track is realized, and the display effect of curling games is improved.

And when the motion information further comprises angular velocity information, the edge calculation server further loads a key point detection algorithm. For example, the key point detection algorithm may include a trained neural network for performing curling key point detection.

The edge computing server is further configured to: after the position information of the curling corresponding to the first video frames is determined, carrying out key point detection on each first video frame in each first video data corresponding to the track based on the curling position information and the key point detection algorithm, and generating key point information of a curling handle corresponding to the first video frames; converting the key point information corresponding to the plurality of first video data corresponding to the track to the virtual coordinate system by using the image fusion algorithm, and generating converted key point information corresponding to the plurality of first video data; and determining the angular velocity information of the curling on the track based on the converted key point information respectively corresponding to the plurality of first video data by using the curling tracking algorithm.

In implementation, after the curling position information corresponding to the first video frames is determined, the edge computing server may further perform key point detection on each first video frame in each first video data corresponding to the track based on the curling position information and a key point detection algorithm to generate key point information of the curling handle corresponding to the first video frames. Referring to the structural schematic diagram of the curling shown in fig. 4, fig. 4 includes a curling handle 1, and the key point information corresponding to the curling handle 1 can be obtained through a key point detection algorithm. The key point information comprises position information of each key point on the curling handle under a pixel coordinate system corresponding to the first video frame. The number of key points can be set as desired.

The edge computing server may further convert, by using a picture fusion algorithm, the key point information corresponding to each of the plurality of first video data corresponding to the track to the virtual coordinate system based on the conversion relationship information between the pixel coordinate system and the virtual coordinate system corresponding to each of the first video data, and generate the converted key point information corresponding to each of the plurality of first video data. And determining the angular speed information of the curling on the track by using a curling tracking algorithm based on the converted key point information respectively corresponding to the plurality of first video data. For example, the orientation of the curling handle at different positions of the track can be determined according to the converted key point information corresponding to the plurality of first video data, and the angular velocity information of the curling can be determined according to the orientation at different positions of the track and the interval duration between the curling handle and the different positions.

In the embodiment of the disclosure, the edge calculation server further loads a key point detection algorithm, the key point information of the curling handle can be accurately determined through the key point detection algorithm, and then the angular velocity information of the curling can be determined according to the key point information, so that the rotation process of the curling in motion is simulated, the motion process of the curling can be more accurately simulated by the motion track of the curling generated subsequently, and the display effect of curling games is improved.

The following exemplifies the architecture of the image rendering apparatus.

In an alternative embodiment, referring to fig. 5, the image rendering device 12 comprises: a target number of second cameras 121, wherein the target number matches a number of capital camps included in the at least one track; the second camera is deployed in a preset area above each track capital camp; the second camera is used for collecting the second video data in the great camp emmetropic direction.

In the curling competition, each track is provided with two big barracks correspondingly. When 4 tracks are included in the competition field, the number of the targets is 8, and each second camera is used for collecting second video data of one big book camp. The second camera may be deployed in a preset region above the track camp, for example, the preset region may be a region centered at a position directly above the track camp and having a preset length (for example, 2 meters) as a radius, so as to collect second video data in the forward viewing direction of the track camp by using the second camera. Wherein, first camera and second camera can be high definition, wide angle camera.

Consider that the result of curling match is relevant with position, the sportsman of curling in the big book is to information such as curling throwing action, for the match process of better show curling match, can set up the second camera of target quantity, and every second camera gathers the second video data of a big book of camping direction, improves the bandwagon match process's bandwagon effect.

In an optional embodiment, the image rendering apparatus further comprises: a rendering server 122 corresponding to each course; the rendering server 122 loads a picture distortion correction algorithm, a trajectory rendering algorithm, and a picture trajectory synchronization algorithm.

The rendering server 122 is configured to receive the second video data of the track corresponding to the rendering server and the motion information of the curling on the track. Correcting the second video data of the track by using the image distortion correction algorithm to obtain processed second video data corresponding to the track; and generating the curling motion track corresponding to the track based on the motion information of the curling on the track by utilizing a track rendering algorithm. And synchronously fusing the processed second video data corresponding to the track and the curling motion track by using the picture track synchronization algorithm to generate the display data corresponding to the track.

In implementation, the rendering server may receive the second video data transmitted by the second camera. For example, the image rendering device may include an access layer switch and a core switch (the core switch may or may not be shared with the track capture device), the second video data collected by the second camera may be transmitted to the core switch through the connected access layer switch, and the core switch transmits each received second video data to the rendering server.

In consideration of the fact that the second video data acquired by the second camera may have distortion, in order to alleviate the problem of poor display effect caused by image distortion, the rendering server may correct the second video data of the track by using an image distortion correction algorithm to obtain the processed second video data corresponding to the track. In order to clearly and intuitively view the motion track of the curling, the rendering server can generate the curling motion track corresponding to the track based on the motion information of the curling on the track by utilizing a track rendering algorithm. For example, a motion curve corresponding to the curling can be generated according to the position information of the curling can, and the motion curve is determined as the motion track of the curling can.

After the curling motion track and the processed second video data are obtained, in order to guarantee the synchronization between the videos and the track and relieve the problem of poor display effect caused by obvious time delay between the videos and the track, the rendering server can synchronously fuse the processed second video data corresponding to the track and the curling motion track by using a picture track synchronization algorithm to generate display data corresponding to the track, so that the time when the curling passes through the front throwing line in the processed second video data in the display data is the same as the time when the curling passes through the front throwing line indicated by the curling motion track.

For example, the rendering server may perform synchronous fusion on the processed second video data corresponding to the track and the curling motion track based on the timestamp corresponding to the processed second video data and the processing time of the track rendering algorithm by using a picture track synchronization algorithm, so as to generate the display data corresponding to the track. For example, after the motion track of the curling is generated by the track rendering algorithm, the target time is determined according to the current time and the consumed time of the track rendering algorithm; determining a second video frame in the second video data corresponding to the curling motion track according to the target time and the timestamp of the video frame in the second video data; and fusing the curling motion track with the determined second video frame to generate display data corresponding to the track.

In an optional embodiment, the image rendering apparatus further comprises: a signal switching stage 123; wherein the signal switching station 123 is connected to the rendering server 122;

the rendering server 122 is further configured to, after the display data corresponding to the track is generated, respond that a size ratio of the display data is not consistent with a standard ratio indicated by a signal transmission protocol, splice the display data with preset data, and generate spliced data; wherein the size proportion of the spliced data meets the standard proportion; and sends the splicing data to the signal switching station 123 by using the signal transmission protocol. The preset data may be set as needed, for example, the preset data may be an interface image corresponding to the operation interface, or may be a blank image.

The signal switching station 123 is configured to obtain the display data corresponding to each track from the mosaic data corresponding to each track; splicing the display data corresponding to the multiple tracks to obtain target display data; wherein the size proportion of the target display data meets the standard proportion; then, the signal transmission protocol is utilized to send the target display data to the display equipment;

the display device 13, when displaying the display data corresponding to each track, is configured to: and displaying the received target display data.

The following description will be given taking an example in which a signal transmission protocol is a digital serial interface (SDI). The standard ratio indicated by SDI is the aspect ratio 16: 9. In order to better display the pictures of the tracks and the motion tracks of the curling, the width-to-height ratio of the display area corresponding to each track is determined to be 8:9, namely the size ratio of the generated display data is 8:9, and the standard ratio required by the SDI can not be met. In order to transmit the display data to the switching station, preset data may be set, where the image size ratio corresponding to the preset data is 8:9, and after the display data and the preset data are spliced, spliced data with an aspect ratio of 16:9 is generated, that is, the size ratio of the spliced data satisfies the standard ratio. Referring to the schematic diagram of the splicing data shown in fig. 6, the left side of the splicing data is preset data, and the preset data may be an interface image corresponding to an operation interface, where the operation interface is only an exemplary illustration, for example, the operation interface includes an adjustment parameter corresponding to the second camera 1 located in the northwest position and an adjustment parameter corresponding to the second camera 1 located in the northeast position; the right side of the stitched data is the presentation image in the presentation data.

And then, the spliced data is sent to a signal switching station by using a signal transmission protocol. For example, the spliced data may be sent to a signal switching station using SDI data lines. The signal switching station 123 is configured to obtain display data corresponding to each track from the mosaic data corresponding to each track; for example, the presentation data may be intercepted from the concatenated data.

Continuing to explain the above case, since the standard ratio indicated by the SDI is the aspect ratio of 16:9, and the aspect ratio of the display data is 8:9, the display data corresponding to the two tracks can be spliced to obtain the target display data, and the aspect ratio of the target display data is 16:9, which meets the transmission requirement of the SDI. And sending the spliced data to display equipment by using an SDI data line. And the display equipment displays the received target display data.

For example, referring to the schematic architecture diagram of the image rendering apparatus shown in fig. 7, the image rendering apparatus includes 6 rendering servers, where rendering server 1 and rendering server 2 are host machines (i.e., rendering servers used in real time), rendering server 3 is standby machines (i.e., standby rendering servers), rendering server 4 and rendering server 5 are host machines, and rendering server 6 is standby machines. In implementation, when any one of the rendering server 1 and the rendering server 2 fails, the rendering server 3 may be used to perform real-time processing.

During implementation, each host can correspond to one track, namely the rendering server 1 corresponds to the track a, the rendering server 2 corresponds to the track B, the rendering server 4 corresponds to the track C, the rendering server 5 corresponds to the track D, and the rendering server 1 performs real-time rendering on the second video data corresponding to the track a and the curling motion track on the track a. Specifically, after the display data corresponding to the track a is generated, the rendering server 1 splices the display data and the preset data in response to the inconsistency between the size ratio of the display data and the standard ratio required by the SDI, generates spliced data, and transmits the spliced data to the switching station. After the switching station receives the splicing data transmitted by each rendering server, the switching station can cut along the central line of the splicing data to acquire the display data corresponding to each track. Splicing the display data of the track A and the track B to obtain target display data 1; and splicing the display data of the track C and the track D to obtain target display data 2. The target display data 1 and the target display data 2 are transmitted to the display device (namely, a large-screen matrix) through SDI data lines respectively, an area A and an area B in the large-screen matrix are used for displaying the target display data 1, an area C and an area D are used for displaying the target display data 2, and therefore synchronous rendering display of curling game pictures and curling moving tracks is achieved.

Referring to fig. 8, a schematic diagram of an architecture of a competition watching system is shown, and a processing flow of the competition watching system is exemplarily described with reference to fig. 8. The trajectory capturing device 11 includes: a plurality of first cameras 111 corresponding to each track. The first camera is used for collecting first video data of the track corresponding to the first camera according to the set collection angle. The first camera sends the collected first video data to the core switch through the access layer switch. And the core switch transmits the received first video data to the edge computing server. The edge computing server detects the first video data by using a loaded curling detection algorithm, a key point detection algorithm, a curling tracking algorithm and a picture fusion algorithm to obtain motion information of the curling on the track, wherein the motion information comprises at least one of position information, speed information, acceleration information and angular velocity information.

The edge computing server sends the detected motion information of the curling on the track to the core switch, and the core switch sends the received motion information of the curling to a rendering server in the image rendering equipment. And a second camera in the image rendering equipment acquires second video data in a large-size looking direction, and sends the acquired second video data to the core switch through the access layer switch, and the core switch sends the second video data to the rendering server. Here, the image rendering device and the trajectory capture device share one core switch, that is, the data transmission is realized by one core switch. And the rendering server processes the second video data of the track and the motion information of the curling on the track by using the loaded picture distortion correction algorithm, the track rendering algorithm and the picture track synchronization algorithm to generate display data corresponding to the track, and sends the display data to the signal switching station.

After generating display data corresponding to the track, the rendering server splices the display data with preset data to generate spliced data when the size ratio of the display data is inconsistent with the standard ratio indicated by the signal transmission protocol; wherein the size proportion of the spliced data meets the standard proportion; and sending the spliced data to a signal switching station by using a signal transmission protocol.

The signal switching station acquires display data corresponding to the track from the splicing data corresponding to each track; splicing the display data corresponding to the multiple tracks to obtain target display data; the size proportion of the target display data meets the standard proportion; and then, transmitting the target display data to the display equipment by using a signal transmission protocol. And the display equipment displays the received target display data.

Based on the same concept, the embodiment of the present disclosure further provides a competition watching method, as shown in fig. 9, the method includes:

s901, acquiring first video data of at least one track and second video data corresponding to a big book on each track;

s902, determining motion information of a curling on each track according to the first video data corresponding to each track; generating a curling motion track corresponding to each track according to the motion information of the curling on each track;

s903, fusing the second video data corresponding to each track with the curling motion track corresponding to the track to generate display data corresponding to the track;

and S904, displaying the display data corresponding to each track.

In an optional implementation manner, determining motion information of a curling on each track according to the first video data corresponding to each track includes:

for each track, performing target detection on each first video frame in each first video data corresponding to the track by using a curling detection algorithm, and determining curling position information corresponding to the first video frames;

by using a picture fusion algorithm, in response to the determination of the virtual coordinate system corresponding to the plurality of first video data of the track, converting the curling position information corresponding to the plurality of first video data corresponding to the track to the virtual coordinate system, and generating converted position information corresponding to the plurality of first video data;

and determining the motion information of the curling on the track by using a curling tracking algorithm based on the converted position information corresponding to the plurality of first video data respectively, wherein the motion information comprises at least one of position information, speed information and acceleration information.

In an optional implementation manner, the motion information further includes angular velocity information, and the motion information of the curling on the track is determined according to the first video data corresponding to each track, further including:

after the position information of the curling corresponding to the first video frames is determined, performing key point detection on each first video frame in each first video data corresponding to the track based on the curling position information and a key point detection algorithm to generate key point information of a curling handle corresponding to the first video frames;

converting the key point information corresponding to the plurality of first video data corresponding to the track to the virtual coordinate system by using a picture fusion algorithm, and generating converted key point information corresponding to the plurality of first video data;

and determining the angular speed information of the curling on the track by using a curling tracking algorithm based on the converted key point information respectively corresponding to the plurality of first video data.

In an optional implementation manner, fusing the second video data corresponding to each track with the curling motion track corresponding to the track to generate display data corresponding to the track, includes:

correcting the second video data of the track by using a picture distortion correction algorithm to obtain processed second video data corresponding to the track; generating a curling motion track corresponding to the track based on the motion information of curling on the track by utilizing a track rendering algorithm;

and fusing the processed second video data corresponding to the track and the curling motion track by using a picture track synchronization algorithm based on a second video frame of the curling passing through the forward throwing line in the processed second video data and the track point of the curling passing through the forward throwing line indicated by the curling motion track to generate the display data corresponding to the track.

In an optional implementation manner, after the generating the presentation data corresponding to the track, the method further includes:

splicing the display data with preset data to generate spliced data in response to the fact that the size proportion of the display data is not consistent with the standard proportion indicated by a signal transmission protocol; wherein the size proportion of the spliced data meets the standard proportion;

acquiring the display data corresponding to each track from the splicing data corresponding to each track; splicing the display data corresponding to the multiple tracks to obtain target display data; wherein the size proportion of the target display data meets the standard proportion;

the displaying the display data corresponding to each track comprises:

and displaying the target display data.

It will be understood by those skilled in the art that in the method of the present invention, the order of writing the steps does not imply a strict order of execution and any limitations on the implementation, and the specific order of execution of the steps should be determined by their function and possible inherent logic.

Based on the same concept, the embodiment of the present disclosure further provides a competition watching device, as shown in fig. 10, the device including: the system comprises an acquisition module 1001, a determination module 1002, a generation module 1003 and a presentation module 1004.

An obtaining module 1001, configured to obtain first video data of at least one track and second video data corresponding to a big book on each track;

a determining module 1002, configured to determine, according to the first video data corresponding to each track, motion information of a curling on the track; generating a curling motion track corresponding to each track according to the motion information of the curling on each track;

a generating module 1003, configured to fuse the second video data corresponding to each track with the curling motion track corresponding to the track, and generate display data corresponding to the track;

the display module 1004 is configured to display the display data corresponding to each track.

In an optional implementation manner, the determining module 1002, when determining the motion information of the curling on each track according to the first video data corresponding to each track, is configured to:

In an optional implementation manner, the motion information further includes angular velocity information, and the determining module 1002, when determining the motion information of the curling on the track according to the first video data corresponding to each track, is further configured to:

In an optional implementation manner, the generating module 1003, when fusing the second video data corresponding to each track with the curling motion track corresponding to the track to generate the display data corresponding to the track, is configured to:

In an optional implementation manner, after the generating the presentation data corresponding to the track, the apparatus further includes: a stitching module 1005 for:

the display module 1004, when displaying the display data corresponding to each track, is configured to:

and displaying the target display data.

In some embodiments, the functions of the apparatus provided in the embodiments of the present disclosure or the included templates may be used to execute the method described in the above method embodiments, and for specific implementation, reference may be made to the description of the above method embodiments, and for brevity, details are not described here again.

In addition, the embodiments of the present disclosure also provide a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the computer program performs the steps of the competition viewing method in the above method embodiments. The storage medium may be a volatile or non-volatile computer-readable storage medium.

The embodiments of the present disclosure also provide a computer program product, where the computer program product carries a program code, and instructions included in the program code may be used to execute the steps of the competition observing method in the foregoing method embodiments, which may be referred to specifically in the foregoing method embodiments, and are not described herein again.

The computer program product may be implemented by hardware, software or a combination thereof. In an alternative embodiment, the computer program product is embodied in a computer storage medium, and in another alternative embodiment, the computer program product is embodied in a Software product, such as a Software Development Kit (SDK) or the like.

It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the system and the apparatus described above may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again. In the several embodiments provided in the present disclosure, it should be understood that the disclosed system, apparatus, and method may be implemented in other ways. The above-described apparatus embodiments are merely illustrative, and for example, the division of the units into only one type of logical function may be implemented in other ways, and for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not implemented. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or units through some communication interfaces, and may be in an electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present disclosure may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.

The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a non-volatile computer-readable storage medium executable by a processor. Based on such understanding, the technical solution of the present disclosure may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present disclosure. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.

The above are only specific embodiments of the present disclosure, but the scope of the present disclosure is not limited thereto, and any person skilled in the art can easily conceive of changes or substitutions within the technical scope of the present disclosure, and shall be covered by the scope of the present disclosure. Therefore, the protection scope of the present disclosure shall be subject to the protection scope of the claims.

Claims

1. A tournament viewing system, comprising: a trajectory capture device, an image rendering device and a presentation device; the image rendering device is respectively connected with the track capturing device and the display device;

2. The system of claim 1, wherein the trajectory capture device comprises: at least one edge computing server; the curling detection algorithm, the curling tracking algorithm and the picture fusion algorithm are loaded on the edge computing server;

3. The system of claim 2, wherein the motion information further comprises angular velocity information, the edge computation server further loaded with a keypoint detection algorithm;

4. The system of any of claims 1-3, wherein the trajectory capture device comprises: a plurality of first cameras corresponding to each track; any position on each track is shot by a plurality of first cameras with different collection angles;

5. The system of claim 4, wherein the trajectory capture device further comprises: a core switch and at least one access stratum switch; each access layer switch is connected with at least one first camera, and the access layer switches are connected with the core switch;

6. The system according to any of claims 1-5, wherein the image rendering device comprises: a target number of second cameras, wherein the target number matches a number of capital camps included in the at least one track; the second camera is deployed in a preset area above each track capital camp;

7. The system of any of claims 1-6, wherein the image rendering device further comprises: a rendering server corresponding to each track; the rendering server loads an image distortion correction algorithm, a track rendering algorithm and an image track synchronization algorithm;

8. The system of claim 7, wherein the image rendering device further comprises: a signal switching station; the signal switching platform is connected with the rendering server;

and displaying the received target display data.

9. A method of viewing a game, comprising:

and displaying the display data corresponding to each track.

10. A tournament viewing apparatus, comprising:

11. A computer-readable storage medium, having stored thereon a computer program for performing, when executed by a processor, the steps of the tournament viewing method of claim 9.