KR102184379B1 - Sports relay streaming system using machine learning technology and automatic production of sports relay streaming using the same - Google Patents

Abstract

The present invention relates to a sports relay streaming system using a machine learning technology and an automatic sports relay streaming production method using the same, capable of directly responding to game conditions, which are rapidly changed, through machine learning. According to the present invention, the sports relay streaming system using the machine learning technology includes: at least two motion photographing units configured to continuously photograph a pitching motion of a pitcher in divided motions; a measurement unit configured to measure coordinate values for an arm, a leg, and a joint of the pitcher; at least two tracking units configured to measure coordinate values of a baseball; a storage unit; a prediction unit configured to predict a coordinate value to which the baseball is to travel; at least two game photographing units configured to determine a photographing direction according to the predicted coordinate value, and photograph a game video in the determined photographing direction; a video editing unit; and a transmission unit.

Description

Sports relay streaming system using machine learning technology and automatic production of sports relay streaming using the same}

The present invention relates to a sports relay streaming system using machine learning technology and a method for automatically producing sports relay streaming using the same, and more particularly, based on machine learning technology, shooting units inside the stadium according to the pitcher's throwing motion during a baseball game It relates to a method and system for automatically producing sports relay streaming in action.

As video and communication media develop, the sports broadcast system is also developing day by day. In particular, efforts to provide various images in real time to sports broadcasting programs such as each broadcaster and portal site due to the coincidence of sports continues.

However, efforts to provide a variety of videos at each broadcaster and portal site inevitably lead to large-scale investments, and the resulting cost is increasing year by year. In addition, the participation of experienced staff is essential for smooth shooting and editing of sports broadcasts, and the reality is that the demand of experienced staff is absolutely insufficient compared to the required manpower in the field.

In order to solve this problem, various methods have been proposed to solve the cost and manpower shortage of a sports relay broadcasting system.

Patent Document 1 generates a high-quality pitching slow motion image from an image acquired from a position where the exact pitching trajectory can be identified, and provides it to be suitable for real-time broadcast, and more accurately confirms the pitcher's pitching and the batter's swing state accordingly. It relates to a system for providing baseball pitching video for broadcast broadcasting.

In this case, the broadcast video is concentrated only on the pitcher and the batter, making it difficult to grasp the situation of the stadium other than the pitcher and the batter, and expensive equipment is introduced to obtain high-quality pitching slow motion motion, and experienced staff are also required. there was.

Patent Document 2 aims to provide a broadcasting viewing viewpoint diversification system and method using 3D modeling data of broadcasting objects that allow viewers to select and view various viewpoints by providing motion data in real time by 3D modeling a broadcasting object. do.

In this case, there is a disadvantage in that a number of cameras must be input to 3D model all information in the stadium, and the amount of information flowing from these cameras is enormous, so that it is necessary to expand a server to process it.

Korean Patent Registration No. 10-1215058 (2012. 06. 21) Korean Patent Application Publication No. 10-20310070035 (2011. 12. 19)

The present invention was created to solve the problems of the prior art, based on machine learning technology, the shooting units inside the stadium are automatically driven according to the pitcher's throwing motion during a baseball game, and the driven shooting units are photographed. An object of the present invention is to provide a sports relay streaming system using machine learning technology that produces sports relay broadcast video based on one video, and a sports relay streaming automatic production method using the same.

In order to achieve the above object, a sports relay streaming system using a machine learning technology according to the present invention and a method for automatically producing sports relay streaming using the same include at least two to continuously photograph a pitcher's throwing motion in a baseball game as a separate motion. A motion photographing unit, a measuring unit that measures coordinate values for the pitcher's arms, legs, and joints among the divided motions photographed by the motion photographing units, and the baseball ball thrown by the pitcher is tracked and set in the stadium At least two tracking units that measure coordinate values of the baseball when the baseball reaches a plurality of areas, a storage unit that accumulates and stores coordinate values measured by the measuring unit and the tracking units, and the storage When the pitcher's arm, leg, and joint pass through a point having specific coordinate values by comparing the coordinate values of the pitcher's arms, legs, and joints accumulated and stored in the unit, it is accumulated and stored in the storage unit. A prediction unit that compares the coordinate values of the baseball and predicts a coordinate value to which the baseball will travel, and determines each photographing direction according to the coordinate value to which the baseball will travel predicted by the prediction unit, A game photographing unit provided at least two or more in a baseball stadium to shoot a baseball game image, and the game images photographed by the game photographing units are collected to edit the game images to fit a baseball broadcast, and the edited as necessary It characterized in that it comprises a video editing unit for producing a baseball broadcast video by inserting subtitles, computer graphics, material images and advertisements in the game video, and a transmission unit for transmitting the video of the baseball broadcast produced by the video editing unit to the outside. .

In addition, in the process of producing a broadcast broadcast of a baseball game in a sports relay streaming system using machine learning technology, the motion photographing units continuously divide the pitching motion of the pitcher as a separate motion from the start of the pitching motion to the end. And the measurement unit measuring coordinate values for the pitcher's arms, legs, and joints during the division motion of the pitcher photographed by the motion photographing units; and the tracking unit measures the baseball ball thrown by the pitcher. Tracking and measuring the coordinate values of the baseball when the baseball reaches one of the plurality of areas preset in the stadium, and the storage unit accumulating and storing the coordinate values measured by the measuring unit and the tracking unit Step and, when the prediction unit compares the coordinate values of the pitcher's arms, legs, and joints accumulated and stored in the storage unit, and performs a throwing motion in which the pitcher's arms, legs, and joints pass through points having specific coordinate values , Comparing the coordinate values of the baseball that are accumulated and stored in the storage to predict the coordinate value of the baseball, and the progress coordinate value of the baseball predicted by the prediction unit is the baseball during an actual baseball game. The step of verifying whether the ball's progression coordinate value is identical, and by comparing the progression coordinate value of the baseball during the actual baseball game and the progression coordinate value of the baseball predicted by the prediction unit, a level suitable for actual baseball broadcast. Examining whether the prediction accuracy is visible, and when the progression coordinate value of the baseball predicted by the prediction unit reaches a level suitable for actual baseball broadcast in the review step, the progression coordinate value of the baseball predicted by the prediction unit is used. Thus, the shooting direction of the game photographing units is determined, and when the progress coordinate value of the baseball predicted by the prediction unit does not reach a level suitable for actual baseball broadcast, the coordinate values measured by the measurement unit and the tracking unit are further accumulated. , The measurement unit accumulates and measures coordinate values for the pitcher's arms, legs, and joints during the division motion of the pitcher photographed by the motion photographing units, and the tracking unit Tracking a true baseball ball, accumulating and measuring coordinate values of the baseball when the baseball reaches a plurality of areas previously set in the stadium, and storing the accumulated coordinate values in the storage unit, and the photographing The game photographing units whose direction is determined shoots the baseball game image and transmits the captured game image to the image editing unit, and the image editing unit matches the baseball game images photographed by the game shooting units to a baseball broadcast. Editing the game images so that the game video is edited and inserting subtitles, computer graphics, material images, and advertisements into the edited game video as necessary to produce a baseball broadcast video, and a baseball broadcast video produced by the video editing unit. It characterized in that it comprises the step of transmitting to the outside through the transmitting unit.

According to the present invention, it is possible to directly respond to rapidly changing game conditions through machine learning learning, and to obtain an image optimized for sports relaying for each position.

In addition, since each photographing unit can be automatically driven to select the most suitable screen among a plurality of relay cameras, there is an effect of economically obtaining a sports broadcast image without large-scale staffing and equipment.

1 is a diagram conceptually showing photographing units installed inside a baseball stadium.
FIG. 2 is a conceptual diagram illustrating a method of measuring coordinate values based on a photographed division motion when a pitcher makes a pitching motion during a baseball game.
3 is a diagram conceptually illustrating a method of measuring coordinate values of a location where a baseball thrown by a pitcher reaches a baseball game during a baseball game.
4 is a block diagram conceptually showing a sports relay streaming system using the machine learning technology of the present invention.
5 is a flow chart showing a sequence in which the game shooter determines a photographing direction in the method for automatically producing sports relay streaming using the machine learning technology of the present invention.

Hereinafter, a sports relay streaming system using machine learning technology according to an embodiment of the present invention and an automatic sports relay streaming production method using the same will be described in detail with reference to the accompanying drawings. Since the present invention can apply various changes and have various forms, specific embodiments will be illustrated in the drawings and described in detail in the text. However, this is not intended to limit the present invention to a specific form of disclosure, it is to be understood as including all changes, equivalents, or substitutes included in the spirit and scope of the present invention. In describing each drawing, similar reference numerals have been used for similar elements. With respect to the accompanying drawings, the dimensions of the structures are shown to be enlarged than the actual size for clarity of the present invention.

Terms such as first and second may be used to describe various components, but the components should not be limited by the terms. These terms are used only for the purpose of distinguishing one component from another component. For example, without departing from the scope of the present invention, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element.

The terms used in the present application are used only to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In the present application, terms such as "comprise" or "have" are intended to designate the presence of features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, but one or more other features. It is to be understood that the presence or addition of elements or numbers, steps, actions, components, parts, or combinations thereof, does not preclude in advance the possibility of the presence or addition.

Unless otherwise defined, all terms including technical or scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs. Terms such as those defined in a commonly used dictionary should be interpreted as having a meaning consistent with the meaning of the related technology, and should not be interpreted as an ideal or excessively formal meaning unless explicitly defined in this application. Does not.

1 to 4 are diagrams conceptually showing a sports relay streaming system 10 using the machine learning technology of the present invention.

1 to 4, a sports relay streaming system 10 using the machine learning technology of the present invention includes a motion photographing unit 100, a measurement unit 150, a tracking unit 200, a storage unit 300, and prediction. It includes a unit 400, a game shooting unit 500, an image editing unit 600, and a transmission unit 700.

The motion photographing unit 100 continuously photographs the pitcher's throwing motion in a baseball game as a separate motion and is provided with at least two.

The measurement unit 150 measures coordinate values (P) for the arm, leg, and joint of the pitcher among each divisional motion photographed by the motion photographing units 100.

The tracking unit 200 tracks the baseball ball (B) thrown by the pitcher, and when the baseball ball (B) reaches a plurality of areas previously set in the stadium (G), coordinate values of the baseball ball (B) are Measure and have at least two or more.

The storage unit 300 accumulates and stores coordinate values measured by the measurement unit 150 and the tracking unit 200.

The prediction unit 400 includes coordinate values (P) for the pitcher's arms, legs, and joints accumulated and stored in the storage unit 300 and the baseball ball (B) accumulated and stored in the storage unit 300 When the pitcher's arm, leg, and joint pass through a point having specific coordinate values by comparing the coordinate values of the pitcher, the coordinate values of the baseball ball B accumulated and stored in the storage unit 300 By comparison, the coordinate value of the baseball (B) to proceed is predicted.

The game photographing unit 500 determines each photographing direction according to the coordinate value in which the baseball ball B predicted by the predicting unit 400 will proceed, photographs the baseball game image in the determined photographing direction, and At least two or more are provided in (G).

The image editing unit 600 collects game images photographed by the game photographing units 500 and edits the game images to fit the baseball broadcast, and includes subtitles, computer graphics, and video on the edited game images as necessary. Produce a baseball broadcast video by inserting material images and advertisements.

The transmission unit 700 transmits the image of the baseball broadcast produced by the image editing unit 600 to the outside.

The process of producing a baseball game broadcast in the sports relay streaming system 10 using machine learning technology according to an embodiment of the present invention configured as described above will be described in detail as follows.

5 is a flow chart showing a sequence in which the game photographing unit 500 determines a photographing direction in the method for automatically producing sports relay streaming using the machine learning technology of the present invention.

Referring to FIG. 5, the motion photographing units 100 continuously photograph the pitcher's throwing motion as a separate motion from the time the pitcher starts to the end of the pitching motion.

Thereafter, the measurement unit 150 measures the coordinate values P of the pitcher's arm, leg, and joint during the division motion of the pitcher photographed by the motion photographing units 100.

Thereafter, the tracking unit 200 tracks the baseball ball (B) thrown by the pitcher, and when the baseball ball (B) reaches one of a plurality of areas previously set in the stadium (G), the baseball ball (B) Measure the coordinate value of ).

In this case, the storage unit 300 accumulates and stores coordinate values measured by the measurement unit 150 and the tracking unit 200.

Thereafter, the prediction unit 400 accumulates and stores coordinate values P for the pitcher's arms, legs, and joints stored in the storage unit 300, and the baseball ball accumulated and stored in the storage unit 300 ( Comparing the coordinate values of B), when the pitcher's arm, leg, and joint pass through a point having specific coordinate values, the coordinates of the baseball ball (B) accumulated and stored in the storage unit 300 Corresponding to the values, the coordinate value of the baseball B is predicted.

Thereafter, it is verified whether the progress coordinate value of the baseball ball B predicted by the prediction unit 400 coincides with the progress coordinate value of the baseball ball B during an actual baseball game, and the baseball ball ( By comparing the progressing coordinate value of B) and the progressing coordinate value of the baseball B predicted by the prediction unit 400, it is examined whether or not the prediction accuracy is at a level suitable for actual baseball broadcasting.

At this time, when the progress coordinate value of the baseball (B) predicted by the prediction unit 400 in the review step reaches a level suitable for actual baseball broadcast, the baseball ball (B) predicted by the prediction unit 400 Although the shooting direction of the game shooting unit 500 is determined using the progress coordinate value of, the progress coordinate value of the baseball ball B predicted by the prediction unit 400 does not reach a level suitable for actual baseball broadcasting. If not, so that the coordinate values measured by the measurement unit 150 and the tracking unit 200 are further accumulated, the measurement unit 150 is the arm of the pitcher during the division operation of the pitcher photographed by the motion photographing unit 100. , While measuring by accumulating coordinate values (P) for the legs and joints, the tracking unit 200 tracks the baseball ball (B) thrown by the pitcher, and the baseball in a number of areas previously set in the stadium (G) When the ball (B) arrives, the step of accumulating and measuring the coordinate values of the baseball (B), and storing the accumulated coordinate values in the storage unit 300 is repeatedly performed.

Thereafter, based on the predicted value of the prediction unit 400 that has reached a level suitable for actual baseball broadcast, the game photographing units 500 whose photographing direction has been determined take the image of the baseball game and record the captured game image to the image editing unit. Send to 600.

Thereafter, the image editing unit 600 edits the game images so that the baseball game images photographed by the game shooting units 500 fit into a baseball broadcast, and the edited game image is edited as necessary with a caption and a computer. A video of a baseball broadcast is produced by inserting graphics, material images, and advertisements, and the video of the baseball broadcast produced by the video editing unit 600 is transmitted to the outside through the transmission unit 700.

According to the present invention, it is possible to directly respond to rapidly changing game conditions through machine learning learning, and to obtain an image optimized for sports relaying for each position.

In addition, since each photographing unit can be automatically driven to select the most suitable screen among a plurality of relay cameras, there is an effect of economically obtaining a sports broadcast image without large-scale staffing and equipment.

Description of the presented embodiments is provided to enable any person skilled in the art to use or practice the present invention. Various modifications to these embodiments will be apparent to those of ordinary skill in the art, and the general principles defined herein can be applied to other embodiments without departing from the scope of the present invention. Thus, the present invention is not to be limited to the embodiments presented herein, but is to be construed in the widest scope consistent with the principles and novel features presented herein.

10: Sports relay streaming system using machine learning technology
100: motion photographing unit 150: measuring unit
200: tracking unit 300: storage unit
400: prediction unit 500: game shooting unit
600: image editing unit 700: transmission unit
B: Baseball G: Stadium
P: The coordinates of the pitcher's arm, leg, and joint

Claims (2)

A motion photographing unit provided with at least two to continuously photograph the pitcher's throwing motion in a baseball game as a separate motion;
A measuring unit for measuring coordinate values of the pitcher's arm, leg, and joint among each divisional motion photographed by the motion photographing units;
A tracking unit provided with at least two or more tracking the baseball ball thrown by the pitcher and measuring coordinate values of the baseball when the baseball reaches a plurality of areas preset in the stadium;
A storage unit for accumulating and storing coordinate values measured by the measurement unit and the tracking units;
By comparing the coordinate values of the pitcher's arm, leg, and joint accumulated and stored in the storage unit and the coordinate values of the baseball ball accumulated and stored in the storage unit, the pitcher’s arm, leg, and joint is a specific coordinate value A prediction unit for predicting a coordinate value to which the baseball will travel in correspondence with the coordinate values of the baseball that are accumulated and stored in the storage unit when a throwing operation passing through the point having the ball
A game photographing unit provided at least two in a baseball stadium to determine each photographing direction according to a coordinate value in which the baseball will travel predicted by the prediction unit, and to photograph the baseball game image in the determined photographing direction;
The game images photographed by the game cameras are collected and the game images are edited to fit the baseball broadcast, and subtitles, computer graphics, data images, and advertisements are inserted into the edited game images as needed. An image editing unit for producing an image; And
And a transmission unit for transmitting the image of the baseball relay broadcast produced by the video editing unit to the outside.
Sports relay streaming system using machine learning technology.
A motion photographing unit provided with at least two to continuously photograph the pitcher's throwing motion as a segmentation motion in a baseball game, and coordinate values for the pitcher's arms, legs, and joints among the segmentation motions photographed by the motion photographing units. A measuring unit that measures the baseball ball thrown by the pitcher, and a tracking unit that measures coordinate values of the baseball when the baseball reaches a plurality of pre-set areas in the stadium, and the measurement unit and the tracking unit measure it A storage unit that accumulates and stores the coordinate values, and a point at which the pitcher's arms, legs, and joints have specific coordinate values by comparing the coordinate values of the pitcher's arms, legs, and joints accumulated and stored in the storage unit. When a passing throwing motion is performed, a prediction unit for predicting a coordinate value at which the baseball will travel by comparing the coordinate values of the baseball stored accumulated in the storage unit, and a coordinate value at which the baseball will travel predicted by the prediction unit Each shooting direction is determined according to the shooting direction, and at least two game shooting units provided in the baseball stadium to shoot the baseball game images in the determined shooting direction, and game images captured by the game shooting units are collected to match the baseball broadcast. Editing the game images so that the game video is edited, and inserting subtitles, computer graphics, material images, and advertisements into the edited game video as needed to create a baseball broadcast video, and the game video produced by the video editing unit to an external device. The process of producing a broadcast broadcast for a baseball game in a sports broadcast streaming system using machine learning technology including a transmitter that is sent to
Continuously photographing the pitcher's throwing motion as a separate motion by the motion photographing units from the start of the pitcher to the end of the pitching motion;
Measuring coordinate values of the pitcher's arm, leg, and joint during the division operation of the pitcher photographed by the motion photographing units by the measuring unit;
Measuring a coordinate value of the baseball when the tracking unit tracks the baseball thrown by the pitcher and the baseball reaches one of a plurality of pre-set areas on the stadium;
Accumulating and storing coordinate values measured by the measuring unit and the tracking unit by the storage unit;
The prediction unit compares the coordinate values of the pitcher's arms, legs, and joints accumulated and stored in the storage unit and the coordinate values of the baseball ball accumulated and stored in the storage unit, so that the arms, legs, and joints of the pitcher are Predicting a coordinate value to which the baseball will travel in correspondence with the coordinate values of the baseball accumulated and stored in the storage unit when a throwing operation passes through a point having specific coordinate values;
Verifying whether the progress coordinate value of the baseball predicted by the prediction unit coincides with the progress coordinate value of the baseball during an actual baseball game;
Comparing the progressing coordinate value of the baseball during the actual baseball game with the progressing coordinate value of the baseball predicted by the prediction unit, and examining whether a prediction accuracy suitable for actual baseball relaying is shown;
In the review step, when the progression coordinate value of the baseball predicted by the prediction unit reaches a level suitable for actual baseball broadcast, the shooting direction of the game photographing units is determined using the progression coordinate value of the baseball predicted by the prediction unit. Decide,
When the progression coordinate value of the baseball predicted by the prediction unit does not reach a level suitable for actual baseball broadcast, the measurement unit is photographed by the motion photographing units so that the coordinate values measured by the measurement unit and the tracking unit are further accumulated. During the division operation of the pitcher, coordinate values of the pitcher's arms, legs, and joints are accumulated and measured, and the tracking unit tracks the baseball thrown by the pitcher, and when the baseball reaches a number of pre-set areas in the stadium, the Accumulating and measuring coordinate values of a baseball, and storing the accumulated coordinate values in the storage unit;
Transmitting the game image captured by shooting the baseball game image to the image editing unit by the game cameras whose shooting direction is determined;
The image editing unit edits the game images so that the baseball game images captured by the game filming units fit into a baseball broadcast, and inserts captions, computer graphics, data images, and advertisements into the edited game images as necessary. Producing a video of a baseball broadcast; And
And transmitting the video of the baseball relay broadcast produced by the video editing unit to the outside through the transmission unit.
A method of automatically producing sports broadcast streaming using machine learning technology

Images