KR102378738B1 - System for broadcasting real-time streaming sportainment baseball league in dome baseball stadium - Google Patents

Abstract

The present invention provides a system for broadcasting a real-time streaming sportainment baseball league in a dome baseball stadium. The system comprises: a domed arena (110) in which various rails for photography are arranged; a movable camera (120) for performing general photography or VR photography along a rail; a photographing drone (130) for performing general photography or VR photography while flying; a broadcasting server (140) for editing baseball game videos and streaming the same in real time; and a recorded video (DB) (170) which records interview videos and growth story videos of baseball players in advance before a game, categorizes and stores the same, edits the recorded video corresponding to an event into documentary-type highlight digital content when the event occurs during the game, and provides video information or text information through a wired/wireless real-time streaming service platform and a broadcasting server (130). Therefore, the system relays a variety of dynamic and multi-view dramatic video content through a non-face-to-face IT platform, and provides related emotional content other than the game, thereby broadcasting a real-time baseball league in which sports and entertainment are combined.

Description

Spotainment baseball league real-time streaming system for dome baseball stadiums {SYSTEM FOR BROADCASTING REAL-TIME STREAMING SPORTAINMENT BASEBALL LEAGUE IN DOME BASEBALL STADIUM}

The present invention is a dome-type baseball stadium that can broadcast a baseball league that combines sports and entertainment in real time by providing emotional content related to extra-match while relaying various dramatic video contents of lively and multi-view through a non-face-to-face IT platform. It relates to spotainment baseball league operation and real-time streaming relay system.

As is well known, the importance of video content in modern sports is a universal fact that anyone can understand without further explanation. Although it enables the production of innovative sports video content from the perspective, in the case of popular sports that are still conservative, the full application of it in the field is not easy due to many interests.

However, due to the recent global spread of respiratory virus infection, that is, COVID-19, the need for non-face-to-face business has risen sharply. By contributing, the need for the rapid spread of new video content services is urgently raised.

In addition, Korea currently ranks first in the world in 4G (LTE) speed (524 Mbps), so there is a limit to appealing to consumers only with high speed of 5G services. Compared to BtoB (Business to Business) fields where AI is applied, such as relatively bright autonomous driving, telemedicine, or factory automation, the BtoC (Business to Consumer) field is somewhat opaque in its profit structure, so you can experience the benefits of 5G service. It is very important to secure a high-quality video, and sports is one of the most optimal fields for content composition suitable for 5G.

Accordingly, there is a need for a technology capable of relaying various dramatic image contents of a lively and multi-view through a non-face-to-face IT platform.

Korean Patent Application Laid-Open No. 10-2021-0001659 (Broadcasting system that provides video of sports competitions filmed using a VR camera attached to a drone, 2021.01.07) Korean Unexamined Patent Publication No. 10-2020-0139756 (System and method for integrating a physical movement target into a game, sport or other activity, December 14, 2020)

The technical task to be achieved by the idea of the present invention is to broadcast a baseball league combining sports and entertainment in real time by providing emotional content related to extra-match while relaying various dramatic video contents of lively and multi-view through a non-face-to-face IT platform. It is to provide a spotainment baseball league operation and real-time streaming relay system of a domed baseball stadium.

In order to achieve the above object, the present invention, the spectator seats for on-site spectating are removed or minimally arranged, the stadium area is covered with a dome-shaped structure, and the dome-shaped structure is installed in the lateral and longitudinal directions, oblique and vertical directions. A domed stadium with arc rails and straight rails; At least one mobile camera having a driving means to move or stop along the arc rail, the straight rail, and the fence line of the stadium area to photograph a baseball game in general or VR; At least one shooting drone for general shooting or VR shooting of baseball games by having a VR camera while flying over the stadium area; Controls the movement, shooting position and shooting angle of the mobile camera, and the flight, movement, and shooting position and shooting angle of the shooting drone, and receives and edits baseball game images shot by the mobile camera and the shooting drone A broadcasting server that transmits the real-time streaming video to the outside; And before the game, each interview video and growth story video of baseball players are recorded, categorized and stored, and when an event occurs during a baseball game, the recorded video corresponding to the event is edited into documentary-type highlight digital content, and the broadcast Provides a video information or text information through a server through a wired/wireless real-time streaming service platform, a recorded image DB; provides a spotainment baseball league operation and real-time streaming relay system of a dome-type baseball stadium, including.

Here, through the real-time streaming service platform, the semantic analysis information of the match, the growth story image of the major baseball player, the match preparation process sketch image, the process description image of immersion in the match, the match notice and follow-up program Provide pre-game sketch video including preview video, match highlight edit video, pre-game sketch prediction contrast result lighting video, major record of the match by baseball player, edited video interview after baseball game, and follow-up video of the match It is possible to provide a post game review video including a story development perspective video and a follow-up match preview video.

In addition, a commercial advertisement image DB that categorizes and stores advertisement images of uniforms, hats, shoes, player equipment, league official uniforms and hats and shoes, or league official bats, balls and gloves for each baseball player who participated in the match in advance. Further comprising, the broadcast server may provide a commercial advertisement image corresponding to the pre-game sketch image or the post-game review image by overlapping or dividing the screen.

In addition, the real-time streaming service platform may be any one of IPTV, OTT, cable PP, Afreeca TV, and YouTube.

In addition, the driving means of the mobile camera is a motor unit for driving the wheel coupled to the arc rail, the linear rail, and the fence line in the stadium area, and the motor unit includes an encoder for detecting the number of rotations to the broadcasting server. By this, it is possible to control the movement and the shooting position of the arc rail, the straight rail, and the fence line of the stadium area.

In addition, the drone for shooting includes a barometric pressure sensor for detecting the altitude inside or outside the dome-shaped structure, an acceleration sensor, and a GPS sensor to control the flight, movement, and shooting location of the shooting drone by the broadcasting server. can

In addition, the mobile camera and the drone for shooting are each provided with a directional sound sensor to distinguish the sound of hitting a game from a player's sound or a song and a performance sound from an online concert, and at least one of the mobile camera and the shooting The drone adjusts the shooting position and shooting angle in the direction in which the hitting sound and noise or song and performance sound are emitted. You can track the source of the performance sound and record it.

In addition, wearable cams that are worn by game operation personnel or online concert organizers to freely shoot general images or VR images according to the field of view and transmit them to the broadcasting server, or are fixedly installed at main points of the stadium ground to shoot general images or VR images Thus, the pylon cam to be transmitted to the broadcast server, or a super slow-mo cam that tracks the source of the hitting sound and noise or song and performance sound, shoots the super slow motion picture, and transmits it to the broadcast server.

In addition, when broadcasting a baseball game, a plurality of infrared sensors for dividing the strike zone area are disposed behind the home plate, and the strike ball is determined according to whether the strike zone area has passed by the infrared sensor, and the image received by the broadcasting server It may further include an AI referee unit that analyzes the image to determine whether or not a foul is caused by the first and third base run lines.

In addition, a large LED screen is installed along the fence line of the stadium area, and through the large LED screen, a simulated spectator seat is produced, a skyline is produced, a sponsor advertisement is displayed, various weather environments or a fantastic atmosphere. You can create a place for

In addition, the broadcast server receives and learns the drone control signal output by the drone pilot through the drone controller, and according to the learned result, a flight control unit that generates a drone control signal for autonomously flying the drone, and the mobile camera and an image processing unit that provides an image to a user terminal in a streaming manner so that a plurality of users can receive a 360˚ image in real time through the terminal for the image captured by the VR camera of the drone for shooting.

In addition, the flight control unit, an image collection module for collecting the image taken at the current point in time through the VR camera, a control signal collecting module for collecting the drone control signal output from the drone controller in response to the time of photographing the image , a learning module for learning by inputting the collected image and drone control signal into a neural network model, and after completion of learning of the neural network model, the image captured at the current time is input to the neural network model where the learning is completed, and autonomously according to the position of the player A control signal generating module that generates a drone control signal so that it can fly, a weight providing module that detects the movement of a player participating in the game, and assigns weights according to the player's motion vector to generate a corrected drone control signal, and It may include a communication module for transmitting the control signal to the drone.

In addition, the learning module constructs a CNN that divides the input image into a plurality of images and processes it, and inputs data that connects the currently photographed image and n previously photographed images to the CNN model, and the CNN By inputting the output value output through the RNN model, it is possible to output the drone control signal according to the temporal or sequencing aspect.

In addition, the image processing unit, the image collection module for receiving the image captured at the current point in time through the VR camera, when an ROI is set by the user for an image captured for a certain period of time, the set ROI is zoomed in and out A learning module for learning the CNN and RNN model to output the obtained image, an image extraction module for extracting an image to be provided to a user terminal by inputting an image photographed for a certain time to the RNN model on which the learning is completed, and the extracted image It may include an image providing module provided to the user terminal.

In addition, the learning module, CNN and RNN so that only the image of the set ROI is enlarged and output when the user sets the zoomed-in or zoomed-out part through the terminal among the images taken at 360˚ for a certain time and inputs it as the ROI. model can be trained.

In addition, the image extraction module provides an image according to a manual mode or a dynamic mode selected by a user through a terminal, and the manual mode represents a mode for providing an image according to the user's movement or gaze direction, and the dynamic The mode indicates a mode in which a partial region included in an image is automatically zoomed in or zoomed out based on the CNN and RNN models and provided to the user terminal. When a rotatable image is provided and a dynamic mode signal is received from the user's terminal, an enlarged portion of the region of interest may be provided on the front of the HMD worn by the user according to an AI algorithm.

In addition, it may be applied to an online sports event, an online cultural performance, an online exhibition, or an online corporate event held in the domed stadium.

In addition, it may further include a ball game video reading system.

In addition, the broadcast server controls the movement of the mobile camera, the photographing position and the photographing angle, and the flight and movement of the photographing drone, the photographing position and the photographing angle, and the baseball game image photographed by the movable camera and the photographing drone. It may be composed of a control center that receives and edits, and a platform streaming server that streams and transmits the edited video in real time to a mobile-based device or a TV-based device.

According to the present invention, it is possible to focus and immerse in baseball league games by streaming digital content that combines sports and entertainment with dramatic real documentary elements to meet the needs of viewers in a baseball league-only stadium equipped with special facilities. It can provide various spectating points, and it has the effect of making people feel interested in baseball league games by providing digital content that contains elements of a story-telling drama, show, or documentary.

In addition, in a situation where admission to baseball spectators is limited due to the recent global spread of respiratory virus infection, that is, COVID-19, but there is a demand for spectators, the construction cost is significantly reduced by eliminating or minimizing the spectator seats. Through the non-face-to-face IT platform using the 5G network, it is possible to broadcast various dramatic video contents of lively and multi-view, and through the domed stadium, ball games such as baseball or soccer can be played regardless of the weather and season In addition to baseball games, online sports events, online cultural performances, online exhibitions, or online corporate events can be relayed live through the 5G network. The RNN model can be used to minimize fertilization and shorten the game time, to increase concentration by creating a fantastic atmosphere and effect through a large LED screen, to maximize the advertising effect, and to dramatize the feeling of immersion for users. Through this, it is possible to derive the work control technology of the VR camera to provide images with a high sense of presence and realism.

Furthermore, by applying this to the business field, it is possible to provide new business items to the sports event industry field with a basic attribute of non-face-to-face, which is a trend in the post-COVID-19 era, and provides various sports video contents, IPTV/OTT It is possible to professionally produce multi-view-VR contents optimized for the characteristics of the platform, and has the effect of continuously hosting large-scale events such as performances, exhibitions, cultural events and corporate events with non-face-to-face themes.

1 shows a schematic configuration diagram of a spotainment baseball league operation and real-time streaming relay system of a domed baseball stadium according to an embodiment of the present invention.
2 and 3 illustrate the domed stadium of FIG. 1 .
FIG. 4 illustrates an implementation diagram of the spotainment baseball league operation and real-time streaming relay system of the dome-type baseball stadium of FIG. 1 .
FIG. 5 illustrates the operation of the spotainment baseball league of the dome-type baseball stadium of FIG. 1 and image reading of the real-time streaming relay system.
FIG. 6 illustrates a field to which the spotainment baseball league operation and real-time streaming relay system of the dome-type baseball stadium of FIG. 1 are applied.
7 illustrates the use of a large LED screen of the spotainment baseball league operation and real-time streaming relay system of the dome-type baseball stadium of FIG. 1 .
8 illustrates various views by the spotainment baseball league operation and real-time streaming relay system of the dome-type baseball stadium of FIG. 1 .
9 is an example of VR image viewing by the spotainment baseball league operation and real-time streaming relay system of the dome-type baseball stadium of FIG.
10 illustrates the installation of a mobile camera of the spotainment baseball league operation and real-time streaming relay system of the dome-type baseball stadium of FIG. 1 .
11 is a schematic diagram comparing the decision-making structure of the BBL to which the spotainment baseball league operation and real-time streaming relay system of the dome-type baseball stadium of FIG. 1 are applied compared with the existing decision-making structure.

Hereinafter, embodiments of the present invention having the above-described characteristics with reference to the accompanying drawings will be described in more detail.

The spotainment baseball league operation and real-time streaming relay system of the dome-type baseball stadium according to an embodiment of the present invention includes a dome-type stadium 110 in which various rails for shooting are arranged, and a mobile camera for general shooting or VR shooting along the rail ( 120), a drone 130 for general shooting or VR shooting while flying, a broadcasting server 140 that edits baseball game images and transmits them in real-time streaming, and interview images and growth story images of baseball players before the game. Each is recorded, categorized and stored, and when an event occurs during a baseball game, the recorded video corresponding to the event is edited into documentary-type highlight digital content, through the broadcasting server 130, video information or text through a wired/wireless real-time streaming service platform. A baseball that combines sports and entertainment by providing emotional content related to extra-game while relaying a variety of lively and multi-view dramatic video content through a non-face-to-face IT platform, including the recorded video DB 170 that is provided as information The main point is to broadcast the league in real time.

Hereinafter, with reference to FIGS. 1 to 10, the spotainment baseball league operation and real-time streaming relay system of the domed baseball stadium of the above configuration will be described in detail as follows.

First, the domed stadium 110, as shown in FIGS. 2 and 3, the spectator seats 111 for on-site spectating are removed or minimized, and the entire stadium area is covered with a dome-shaped structure 112, and the dome-shaped The structure 112 is provided with an arc rail 113 and a straight rail 114 (refer to FIG. 10) installed in the transverse, longitudinal, oblique and vertical directions, so that the arc rail 113 and the straight rail 114 are disposed. Accordingly, the mobile camera 120 moves to photograph the stadium area.

Here, the dome-type stadium 110 hosts baseball league games regardless of weather and season, and all of the spectator seats 111 are removed or minimized to reduce the overall scale to significantly reduce construction costs, and to significantly reduce construction costs in addition to baseball league games. , As illustrated in FIG. 6 , it is also possible to broadcast game VR, performance VR, online sports event, online cultural performance, online exhibition, or online corporate event live through a 5G network as a VR image.

Next, at least one mobile camera 120 is formed and provided with a driving means, so that the arc rail 113 and the straight rail 114 and the fence line 115 of the stadium area (refer to FIG. 7 (a)) It is transmitted to the broadcasting server 140 by moving or stopping to shoot a baseball game in general or VR.

For example, the driving means of the mobile camera 120 is a motor unit that drives the wheel 121 coupled to the arc rail 113 and the straight rail 114 and the fence line 115 in the stadium area, and the motor unit sets the number of rotations. Including an encoder (not shown) to detect, the movement and shooting position on the arc rail 113 and the straight rail 114 and the fence line 115 of the stadium area are controlled by the shooting position control signal of the broadcasting server 140 can do. Here, the motor unit is coupled to the mobile camera 120 to drive the wheel 121 to move along the rail and the fence line.

Alternatively, as shown in FIG. 10 , the motor unit 122 of the driving means of the mobile camera 120 is separated from the mobile camera 120 , and the arc rail 113 and the straight rail 114 and the fence line of the stadium area Coupled to one end of the 115, the mobile camera 120 is coupled to the timing belt 123 formed along the rail and fence line from the motor unit 122, the wheel 121 by the rotation of the motor unit 122 ) can be moved by being guided along rails and fence lines.

Next, at least one drone 130 for shooting may be formed so as to be equipped with a VR camera 131 while flying over the stadium area to generally shoot a baseball game or to shoot VR.

Here, the drone 130 for shooting includes a barometric pressure sensor for detecting the altitude inside or outside the dome-shaped structure 112, an acceleration sensor, and a GPS sensor, and the flight of the drone 130 for shooting by the broadcast server 140 It is possible to control the movement, shooting position, and shooting angle. It can fly by the operation of a drone pilot or receive a drone control signal from the broadcast server 140 to be described later to make it fly autonomously, so that it can shoot according to the motion vectors of the players. It can be made to fly inside or outside the dome-shaped structure 112 in accordance with the photographing position and photographing angle required to do so.

On the other hand, the mobile camera 120 and the drone 130 for shooting are each equipped with a directional sound sensor that detects noise of a specific dB or more, so that the hitting sound such as 'just' in a game, the sound of a player, or a song in an online concert and By filtering each performance sound from other noises, at least one mobile camera 120 and a shooting drone 130 are shot in a direction in which the hitting sound and noise of ball games such as baseball, soccer, and hockey or singing and playing sounds are emitted. By adjusting the location and shooting angle, and moving the mobile camera 120 and the drone 130 for shooting according to the movement of the hitting sound and noise or song and performance sound, tracking the source of the hitting sound and noise or song and performance sound while moving You can also keep zooming in to shoot.

In addition, referring to FIG. 9 , the mobile camera 120 or the drone 130 for shooting can take and transmit a general image or VR image by zooming in or out by tracking individually for each baseball game player or for each concert performer, and in addition It is also possible to convert the AR image by the broadcasting server 140 and transmit it so that it can be viewed realistically by the AR Head Mount Display (HMD) 30 .

In addition, VR shooting by the mobile camera 120 and the shooting drone 130 is designed based on a dual image sensor, and converts the 360° omnidirectional image into virtual reality content and provides it to the broadcasting server 140. .

Next, the broadcast server 140 controls the movement and shooting position and angle of the mobile camera 120 and the flight and movement of the drone 130 for shooting, the shooting position and the shooting angle, and the mobile camera 120 and the shooting drone ( 130), it is possible to receive and edit the baseball game video captured by the player, and to broadcast the edited video in real-time to the outside and then broadcast it live.

For example, the broadcast server 140, as shown in FIG. 4, controls the movement, the photographing position and the photographing angle of the mobile camera 120, and the flight and movement, the photographing position and the photographing angle of the drone 130 for photographing, A control center 140A that receives and edits the baseball game image captured by the mobile camera 120 and the shooting drone 130, and decodes the edited image by a decoder to a mobile based device (10) or a 4K/8K TV-based device (TV based device) (20) through a 5G network in real-time streaming through the transmission to select a preferred screen for viewing may be configured as a platform streaming server (140B).

On the other hand, the broadcast server 140, with reference to FIGS. 4 and 8, in addition to the mobile camera 120 and the drone 130 for shooting, is worn by game operation personnel or online concert organizers to display general images or VR images in the field of view. A wearable cam 141 that is freely filmed and transmitted to the broadcasting server 140, or a pylon cam that is fixedly installed at the main point of the stadium ground and takes a general image or VR image and transmits it to the broadcasting server 140 ( pylon cam) (142), or by tracking the source of the hitting sound and noise or song and performance sound, super slow-motion recording and transmitting to the broadcasting server 140, including a super slomo cam (143) further including , it is possible to provide a game video of various views as illustrated in FIG. 8, which cannot be provided in the on-site spectator.

Referring to FIG. 5 , in addition to image reading by identification of human resources, the decision result is systematized by the AI referee 150 so as to minimize the determination fertilization by the referee and shorten the game running time.

For example, the AI referee 150 has a plurality of infrared sensors 151 that divide the strike zone area S behind the home plate when the broadcast server 140 relays a baseball game in the dome-type stadium 110 . Deployed and determined by the infrared sensor 151 according to whether the strike zone area (S) has passed or not, or determined by applying a pitching trajectory system using radar to the reading, and the image received by the broadcast server 140 By analyzing the video, it is possible to determine whether there is a foul by the first and third base lines. Here, the strike zone area S is a false line corresponding to the upper and lower limit lines from the elbow to the knee of the batter and the left and right limit lines of the left and right widths of the home plate and can be divided by the infrared sensor 151 .

Alternatively, it may further include a ball game video reading system, ie, Hawk-Eye, to perform reading of in-play fouls on batted balls. For reference, in Hawkeye, several high-speed cameras located in various places in the stadium capture the movement of the batted ball at 340 frames per second, and the broadcast server 140 reconstructs it into a three-dimensional image, so that the AI referee 150 or human resource referee This final judgment may be performed.

For example, in the case of baseball, if it is unclear whether it is safe or out, the game is held by the referee's call, and then the game video at various shooting angles stored in the broadcasting server 140 is read in real time, or when the batted ball is in play, first base, second base Even at the time of judgment at each base of base, third base, and home, after holding the game by the referee's call, it is possible to read the game images from various shooting angles stored in the broadcasting server 140 in real time.

In addition, the broadcast server 140 analyzes the match video of the match, and utilizes radar technology, screen tracking, and data analysis technology, as well as the pitcher's speed, rotation speed, and trajectory, as well as the trajectory of the batted ball, speed of launch, launch angle and runner It is also possible to provide game data such as pitching data such as base speed, position and speed of a defender, hitting data, defense data, hitting range data, base base data, and defense data.

In addition, referring to FIG. 7 , a large LED screen 160 is installed along the fence line 115 of the stadium area (see (a)), and the spectator seats 111 in which on-site spectating is simulated through the large LED screen 160 ), create a skyline (see (b)), display a sponsored advertisement (see (c)), create virtual reality, various weather environments, or a place with a fantastic atmosphere (see (d)). . Therefore, even when watching live, the viewer ratings may be lowered due to the low concentration due to the monotonous and simple game operation that cannot feel the liveliness of the on-site spectators, so the large LED screen 160 creates a fantastic atmosphere and effect. can be increased, and advertising effectiveness can be maximized.

On the other hand, the broadcast server 140 receives and learns the drone control signal output by the drone pilot through the drone controller, and according to the learned result, the flight control unit 144 generates a drone control signal for autonomously flying the drone, And the mobile camera 120 and the image taken by the VR camera 131 of the drone 130 for shooting, a plurality of users mobile-based device 10, TV-based device 20 or a terminal such as an AR head mounted display It may include an image processing unit 145 that provides an image to the user terminal in a streaming manner by the platform streaming server 140B so as to receive a 360˚ image in real time through the

Here, the flight control unit 144 is output from the drone controller in response to the image collection module 144a that collects the image captured at the current point in time through the VR camera 131 of the drone 130 for shooting, and the time of capturing the image. The control signal collection module 144b for collecting the drone control signal, the learning module 144c for learning by inputting the collected images and drone control signals to the neural network model, and the current time point in the neural network model where the learning is completed after completing the learning of the neural network model A control signal generating module (144d) that generates a drone control signal so that the drone can fly autonomously according to the player's position by inputting an image captured in the It may include a weight providing module 144e that generates a corrected drone control signal by assigning weights, and a communication module 144f that transmits the control signal to the drone.

For example, the image collection module 144a collects game images through the VR camera 131, the control signal collection module 144b collects a control signal output through the drone controller, and the control signal collection module 144b The drone control signal outputted from the drone controller is collected in response to the time of shooting the game video, and the control signal collection module 144b transmits the collected drone control signal to the learning module 144c.

Thereafter, the learning module 144c inputs the game image collected in the image collection module 144a and the drone control signal collected in the control signal collection module 144b into the built-up neural network model to learn, and the learning of the neural network model is completed. When done, the control signal generating module 144d generates a drone control signal so that it can fly autonomously according to the position of the player by inputting the game image captured at the current time point into the neural network model, wherein the weight providing module 144e A weight is given to the generated drone control signal to generate a corrected drone control signal.

Here, the learning module 144c constructs a Convolutional Neural Network (CNN) that divides and processes the input image into a plurality of images, and n images currently captured and previously captured in the CNN model. It is possible to input the data connected to , and input the output value output through the CNN to the RNN (Recurrent Neural Network) model to output the drone control signal according to the temporal or sequencing aspect.

In addition, the learning module 144c is configured with the CNN so that when the user sets the zoomed-in or zoomed-out portion through the terminal among the images taken at 360° for a certain time and inputs it as the region of interest, enlarges and outputs only the image of the set region of interest. RNN models can be trained.

That is, the weight providing module 144e predicts the movement direction of the player using any one of the number of players participating in the game, the average value of the player's motion vectors, and the relative position of the player, and the weight providing module 144e ) adds weights to the movement of the batted ball or the movement of the player based on the predicted direction of movement of the player, and then applies the weight to the drone control signal to correct the drone control signal.

Thereafter, the communication module 144f transmits the generated drone control signal to the drone 130 for shooting, and the drone 130 for shooting autonomously flies over the dome-type stadium according to the received drone control signal, while shooting mainly for players.

On the other hand, the image processing unit 145, the image collection module 145a that receives the image captured at the current point in time through the VR camera 131, when the user sets the region of interest for the image photographed for a certain period of time, the set A learning module 145b that trains the CNN and RNN models to output the zoomed-in and zoomed-out images of the region of interest, and image extraction for extracting images to be provided to the user terminal by inputting images captured for a certain period of time into the RNN model that has been trained It may include a module 145c, and an image providing module 145d that provides the extracted image to the user terminal.

Here, the image extraction module 145c provides an image according to a manual mode or a dynamic mode selected by the user through the terminal, and the manual mode provides an image according to the user's movement or gaze direction. represents a mode to provide, and the dynamic mode represents a mode in which a partial area included in an image is automatically zoomed in or zoomed out based on the CNN and RNN models and provided to the user terminal, and the image providing module 145d is the user's gaze An image that can be rotated 360˚ according to the direction is provided, but when a dynamic mode signal is received from the user's terminal, the portion of the region of interest can be enlarged and provided on the front of the HMD worn by the user according to the AI algorithm. .

Accordingly, the autonomous flight of the drone can be controlled based on the image captured by the VR camera and the drone control signal generated while the user operates the drone for shooting, and the drone control signal is generated using CNN and RNN to generate the drone control signal. Minimize the creation process, provide drone control signals that can precisely and accurately control drones for shooting, and draw a VR camera work control technology through RNN model to maximize immersion for users to provide a sense of presence and realism. It can provide a high-quality image.

In addition, as shown in FIG. 1 , the recorded image DB 170 records and categorizes and stores the interview images and individual growth story images of teams or baseball players participating in the corresponding match in the baseball league before the baseball game in advance, respectively. In addition, when an event occurs during a baseball game, the recorded video corresponding to the event is edited into highlight digital content in the documentary format and provided as video information or text information through the broadcast server 140 through a wired/wireless real-time streaming service platform.

Specifically, through the real-time streaming service platform, that is, the platform streaming server 140B, the recorded image DB 170 provides semantic analysis information of each corresponding match in the baseball league, a growth story image of a major baseball player, and a sketch of the match preparation process. Provide pre-game sketch video including video, video depicting the process of being immersed in the match, match preview and follow-up program preview video, or match highlight editing video and pre-game sketch prediction results By providing a post-game review video including a lighting video, a main record of the match by each baseball player, an interview edited video after a baseball game, a story development prospect video following the match, and a follow-up match preview video, It is possible to provide game information for understanding the match, and program content in the form of a real documentary that allows you to enjoy and immerse yourself in the match.

Here, the real-time streaming service platform may be any one of IPTV, OTT, cable PP, Afreeca TV, and YouTube.

Accordingly, it minimizes the boring elements of baseball in the existing baseball framework, provides a thrilling game that allows you to be immersed in the game, and contains story elements such as players' skills, appearance, home environment, and external talent and personality. By providing digital content in the same form as a real documentary program, it is possible to keep fans and viewers emotionally interested in the baseball league.

In addition, the commercial advertisement image DB 180 is, in advance, advertisement images of uniforms, hats, shoes, player equipment, league official uniforms and hats and shoes, or league official bats, balls and gloves for each baseball player who participated in the match. Stored by categorization, the broadcast server 140 extracts a commercial advertisement image corresponding to a pre-game sketch image or a post-game review image from the commercial advertisement image DB 180 by AI and overlaps it with a baseball game image or divides the screen. By providing it, it is possible to provide related product information for providing the viewer with a sense of unity with the player.

On the other hand, FIG. 11 is a diagram illustrating the decision-making structure of BBL to which the spotainment baseball league operation and real-time streaming relay system of the dome-type baseball stadium of FIG. 1 are applied compared with the existing decision-making structure. ), the independent league decision-making structure characterized by the following is detailed.

All professional leagues, including the existing KBO league, are basically an organization in the form of a corporation, and the club exists in the form of a member of the association, and all decisions in the actual league are decided by the club. However, in BBL, the league secretariat (e.g., SL Company) oversees the overall league operation, and the affiliated teams exist under a contractual relationship with the secretariat. can

In addition, as summarized in the following [Table 1], when comparing the progress of the existing league and the BBL, in the BBL, the general judgment and the pitching judgment are the aforementioned AI judge unit 150, the judgment control system (judgment control) system) can be used to determine whether or not a strike/ball has been automatically determined by an auto ball-strike system.

On the other hand, differentiated from the existing leagues, BBL is the actual play during the game as described in [Table 1] in the game development process such as the number of season games, the time required for one game, the pitcher management, the batter management, and the player's salary. All actions that consume time outside of active playing time are prohibited, and penalties are imposed for all delaying actions so that fans can immerse themselves in the spectator without losing interest in the match due to boredom. It is operated with an accuracy of 99.9% by the AI referee unit 150 of the system to minimize judgment fertilization to induce a fair game, and to limit the number of season games to provide an environment where you can focus on every game, By limiting participation, players are encouraged to concentrate by pouring everything into the given time to induce performance improvement, and player salaries are also measured and paid in real time according to the record and team performance contribution of the year so that players are more active in the game. It can also motivate you to do it.

BBL operation method KBO (existing professional baseball) operation method number of season games 2 times per week, total of 64 games 6 times a week, about 144 games in total 1 game time 7 innings or 1 hour and 30 minutes over (time rule applies) Up to 15 innings until the game is decided (takes over 3 hours) general judgment Judgment in the central reading control room using the Judgment Control System The referee makes the decision, but refers to the VAR pitch judgment Automatic determination of strike/ball by a machine called Auto Ball-Strike system judge only pitcher management 1 pitcher 1 inning rule no limit typewriter 1 batter, 3 at-bats or less no limit player's salary Measure and pay in real time according to your records One-year contract according to individual record, team performance, age or proportion within the team Other Prohibition of appeal to judges, prohibition of visiting coaching staff mounds, prohibition of pitching during innings, no substitution of batters during batting Any action other than running the game is allowed at the discretion of the coach and the referee

Therefore, by the configuration of the spotainment baseball league operation and real-time streaming relay system of the dome-type baseball stadium as described above, sports and dramatic real documentary elements are provided to meet the needs of viewers in a baseball league-only stadium equipped with special facilities. By streaming digital content combined with entertainment with added entertainment, it can provide various spectating points to focus and immerse in the baseball league game, and provide digital content with story-telling drama, show, or documentary elements to provide a baseball league game. In order to make the game interesting, and in a situation where admission to baseball fans is limited due to the recent global spread of respiratory virus infection, that is, COVID-19, but there is a demand to watch, the size of the stadium by removing or minimizing the spectator seats construction cost can be significantly reduced by reducing In addition to baseball games, online sports events, online cultural performances, online exhibitions, or online corporate events can be broadcast live through the 5G network, and the decision result is judged by the AI jury By systematizing it, it is possible to minimize the judgment quarrel by the referee and shorten the running time of the game. In order to dramatize it, it is possible to provide images with a high sense of realism and realism by deriving the work control technology of the VR camera through the RNN model.

By applying this to the business field, it is possible to provide new business items to the sports event industry with the basic attribute of non-face-to-face, which is a trend in the post-COVID-19 era, and provides various sports video contents, and features of IPTV/OTT platform It is possible to professionally produce multi-view-VR contents optimized for video games, and to continuously host large-scale events such as performances, exhibitions, cultural events and corporate events with non-face-to-face themes.

Configurations shown in the embodiments and drawings described in this specification are only the most preferred embodiment of the present invention, and do not represent all of the technical spirit of the present invention, so various equivalents that can be substituted for them at the time of the present application It should be understood that there may be water and variations.

110: domed stadium 111: spectators seat
112: dome-shaped structure 113: arc rail
114: straight rail 115: fence line
120: mobile camera 121: wheel
122: motor unit 123: timing belt
130: drone for shooting 131: VR camera
140: broadcasting server 140A: control center
140B: platform streaming server 141: wearable cam
142: pylon cam 143: super slow-mo cam
144: flight control unit 145: image processing unit
150: AI referee 160: large LED screen
170: Recorded image DB 180: Commercial advertisement image DB
10: mobile-based device 20: TV-based device
30: AR head mounted display

Claims (19)

Spectator seats for on-site spectators are removed or minimized, and the stadium area is covered with a dome-shaped structure, and arc rails and straight rails installed in the transverse, longitudinal, oblique and vertical directions of the dome-shaped structure are arranged. ;
At least one mobile camera having a driving means to move or stop along the arc rail, the straight rail, and the fence line of the stadium area to photograph a baseball game in general or VR;
At least one shooting drone for general shooting or VR shooting of a baseball game with a VR camera while flying over the stadium area;
Controls the movement, shooting position and shooting angle of the mobile camera, and the flight, movement, and shooting position and shooting angle of the shooting drone, and receives and edits baseball game images shot by the mobile camera and the shooting drone A broadcasting server that transmits the real-time streaming video to the outside; and
Before the game, each interview video and growth story video of baseball players are recorded, categorized and stored, and when an event occurs during a baseball game, the recorded video corresponding to the event is edited into documentary-type highlight digital content, and the broadcast server Recorded image DB, which is provided as image information or text information through a wired/wireless real-time streaming service platform through
The mobile camera and the drone for shooting are each equipped with a directional sound sensor to distinguish the sound of hitting a game from a player's sound or a song and a performance sound from an online concert, at least one of the mobile camera and the drone for shooting The shooting position and shooting angle are adjusted in the direction in which the hitting sound and noise or song and performance sound are emitted, and the mobile camera and the drone for shooting use the hitting sound and noise or song and performance sound according to the movement of the hitting sound and noise or song and performance sound. Characterized in tracking the source of the shooting,
Spotainment baseball league real-time streaming system of domed baseball stadium.
According to claim 1,
Through the real-time streaming service platform, the semantic analysis information of the match, the growth story video of the major baseball player, the match preparation process sketch video, the process description video of getting immersed in the match, the match preview and follow-up program preview video provide a pre-game sketch video containing, or
Includes match highlight editing video, pre-game sketch prediction contrast result lighting video, main record of the match for each baseball player, interview edited video after baseball game, video story development prospect following the match, and follow-up match preview video A spotainment baseball league real-time streaming relay system of a domed baseball stadium, characterized in that it provides a post game review image.
3. The method of claim 2,
A commercial advertisement image DB that categorizes and stores advertisement images of uniforms, hats, shoes, player equipment, league official uniforms and hats and shoes, or league official bats, balls and gloves for each baseball player who participated in the match in advance. including,
The broadcast server is a spotainment baseball league real-time streaming relay system of a dome-type baseball stadium, characterized in that it provides overlapping or split screen commercial advertisement images corresponding to the pre-game sketch image or the post-game review image.
According to claim 1,
The real-time streaming service platform is IPTV, OTT, cable PP, African TV, or YouTube, characterized in that any one, spotainment baseball league real-time streaming relay system of a domed baseball stadium.
According to claim 1,
The driving means of the mobile camera is a motor unit for driving the arc rail, the linear rail, and a wheel coupled to the fence line in the stadium area, and the motor unit includes an encoder for detecting the number of rotations. A spotainment baseball league real-time streaming relay system of a domed baseball stadium, characterized in that it controls the movement and shooting position of the arc rail, the straight rail, and the fence line of the stadium area.
According to claim 1,
The photographing drone is provided with a barometric pressure sensor, an acceleration sensor, and a GPS sensor for detecting an altitude inside or outside the dome-shaped structure to control the flight, movement, and photographing location of the photographing drone by the broadcast server The spotainment baseball league real-time streaming system of the domed baseball stadium.
delete The method of claim 1,
A wearable cam that is worn by game operation personnel or online concert organizers to freely shoot general images or VR images according to the field of view and transmits them to the broadcasting server, or is fixedly installed at the main points of the stadium ground to shoot general images or VR images A dome-type baseball stadium, characterized in that it further comprises a super slow-mo cam for transmitting to the broadcast server by tracking the source of the pylon cam or the hitting sound and noise or song and performance sound to transmit to the broadcast server. Spotainment Baseball League real-time streaming system.
The method of claim 1,
During the broadcast of a baseball game, a plurality of infrared sensors for dividing the strike zone area are disposed behind the home plate, and the strike ball is determined according to whether the strike zone area has passed by the infrared sensor, and the image received by the broadcasting server is displayed as an image. Spotainment baseball league real-time streaming relay system of a dome-type baseball stadium, characterized in that it further comprises an AI referee, which analyzes and determines whether or not a foul is caused by the first and third base run lines.
According to claim 1,
A large LED screen is installed along the fence line of the stadium area, and through the large LED screen, a simulated spectator seating area is produced, a skyline is created, a sponsor advertisement is displayed, a place with a different weather environment or a fantastic atmosphere. A spotainment baseball league real-time streaming relay system of a dome-type baseball stadium, characterized in that it directs.
According to claim 1,
The broadcast server receives and learns the drone control signal output by the drone pilot through the drone controller, and a flight controller that generates a drone control signal for autonomously flying the drone according to the learned result, and the mobile camera and the Dome-type baseball, characterized in that it includes an image processing unit that provides an image to a user terminal in a streaming manner so that a plurality of users can receive a 360˚ image in real time through the terminal for the image captured by the VR camera of the drone for shooting The stadium's spotainment baseball league live streaming system.
12. The method of claim 11,
The flight control unit, an image collection module for collecting the image taken at the current time through the VR camera, a control signal collection module for collecting the drone control signal output from the drone controller in response to the time of capturing the image, the Learning module for learning by inputting the collected images and drone control signals to the neural network model, after the learning of the neural network model is completed, the image captured at the current time is input to the neural network model where the learning is completed, and autonomously flying according to the position of the player a control signal generating module that generates a drone control signal to enable Spotainment baseball league real-time streaming relay system of a domed baseball stadium, characterized in that it includes a communication module for transmitting a signal to the drone.
13. The method of claim 12,
The learning module constructs a CNN that divides and processes the input image into a plurality of images, and inputs the data that connects the currently photographed image and n previously photographed images to the CNN model, and through the CNN A spotainment baseball league real-time streaming relay system of a dome-type baseball stadium, characterized in that it outputs a drone control signal according to a temporal or sequential aspect by inputting the output value to the RNN model.
12. The method of claim 11,
The image processing unit, an image collection module for receiving an image photographed at a current point in time through the VR camera. When a region of interest is set by a user for an image captured for a certain time, the set region of interest is zoomed in and out of the image. A learning module for learning CNN and RNN models to output Spotainment baseball league real-time streaming relay system of a dome-type baseball stadium, characterized in that it includes an image providing module to provide to.
15. The method of claim 14,
The learning module is a CNN and RNN model to enlarge and output only the image of the set ROI when the user sets the zoomed-in or zoom-out part through the terminal among the images taken at 360˚ for a certain time and inputs it as the ROI. A spotainment baseball league real-time streaming relay system of a domed baseball stadium, characterized in that it is taught.
16. The method of claim 15,
The image extraction module provides an image according to a manual mode or a dynamic mode selected by a user through a terminal, the manual mode represents a mode for providing an image according to the user's movement or gaze direction, and the dynamic mode is , represents a mode for automatically zooming in or zooming out some areas included in an image based on the CNN and RNN models and providing them to the user terminal,
The image providing module provides an image that can be rotated 360˚ according to the user's gaze direction in normal times, and when receiving a dynamic mode signal from the user's terminal, interest in the front of the HMD worn by the user according to the AI algorithm Spotainment baseball league real-time streaming relay system of a dome-type baseball stadium, characterized in that it provides an enlarged part of the area.
According to claim 1,
Spotainment baseball league real-time streaming relay system of a domed baseball stadium, characterized in that it is applied to an online sports event, an online cultural performance, an online exhibition, or an online corporate event held in the domed stadium.
According to claim 1,
The spotainment baseball league real-time streaming relay system of the domed baseball stadium, characterized in that it further comprises a ball game video reading system.
According to claim 1,
The broadcast server controls the movement of the mobile camera, the photographing position and the photographing angle, and the flight and movement of the photographing drone, the photographing position and the photographing angle, and receiving the baseball game image photographed by the movable camera and the photographing drone. A spotainment baseball league real-time streaming relay system of a dome-type baseball stadium, characterized in that it consists of a control center for editing and a platform streaming server that streams and transmits the edited video in real-time to a mobile-based device or a TV-based device.

Images