KR20210061641A - Apparatus and method for information detecting of sports game - Google Patents

Abstract

According to an embodiment of the present invention, an apparatus for detecting sports game information includes at least one processor. The at least one processor acquires a video including a sports game and log information that sequentially records events that occurred in the sporting event, splits at least one scene including a preset motion in the video including the acquired sports game, and detects at least one piece of preset sports game information based on a frame constituting the divided at least one scene.

Description

Sports event information detection device and sports event information detection method {APPARATUS AND METHOD FOR INFORMATION DETECTING OF SPORTS GAME}

The following embodiments relate to an apparatus for detecting sports event information and a method for detecting sports event information.

In general, scene change refers to a phenomenon in which a video is restarted with another scene after one scene is finished in a continuous video scene. Scene transitions are usually fade-out and fade-in, where an arbitrary scene gradually disappears and then another scene gradually appears, and an overlap in which any two scenes slowly overlap over time in different directions. overlap) and simple scene change.

Machine learning is a field of artificial intelligence, a field that has evolved from the study of pattern recognition and computer learning theory, and refers to the field of developing algorithms and technologies that enable computers to learn.

The core of machine learning lies in representation and generalization. Representation is the evaluation of data, and generalization is the processing of data that is not yet known. It is also a field of computational learning theory.

Deep learning is defined as a set of machine learning algorithms that attempt a high level of abstraction through a combination of several nonlinear transducers. It can be said that it is a field.

According to an embodiment of the present invention, it is possible to provide a sports event information detection apparatus and a sports event information detection method capable of detecting at least one piece of information related to a sports event.

In addition, according to another embodiment of the present invention, it is possible to provide a sports event information detection apparatus and a sports event information detection method capable of selecting a scene in which a preset motion occurs.

In addition, according to another embodiment of the present invention, it is possible to provide a sports event information detection apparatus and a sports event information detection method capable of tagging preset information on a scene in which a preset motion occurs.

According to an embodiment of the present invention, in the apparatus for detecting sports event information, the apparatus includes at least one processor, and the at least one processor includes a video including a sports event and a log sequentially recording events occurring in the sports event. Obtaining information, dividing at least one scene including a preset motion from a video including the obtained sporting event, and at least one pre-set sport based on a frame constituting the divided at least one scene Detect game information.

In addition, the at least one processor detects a scene change in the video including the sports game, divides the video including the sports game into a plurality of scenes, and applies scene recognition to each of the divided plurality of scenes. Classify each of the plurality of scenes into a preset class, select a class matching the preset motion from among the preset classes, and select a scene in which the preset motion occurs in the scene classified as the selected class. Can be selected.

In addition, the scene change may be detected by converting RGB to HSV and using a histogram difference of the HSV.

In addition, the scene change may be determined by determining a histogram difference between a previous frame and a current frame for each channel of the HSV, and comparing the determined histogram difference with a preset reference.

In addition, the scene recognition may classify each of the plurality of scenes based on a scene recognizer score for a frame constituting each of the plurality of scenes.

In addition, the at least one processor may tag log information related to the scene in which the preset operation occurs to the scene in which the preset operation occurs.

In addition, the at least one processor, through a relay board included in the scene in which the preset operation occurs, detects situation information of the scene in which the preset operation occurs, and compares the determined situation information with the log information. , Tag information on a scene in which the preset operation occurs may be extracted, and the extracted tag information may be tagged to a scene in which the preset operation occurs.

In addition, the at least one processor detects a position of at least one object based on a frame constituting the at least one divided scene, estimates at least one zone in the at least one divided scene, and At least one pre-set information on the sporting event may be detected based on the estimated at least one zone and the detected position of the at least one object.

According to another embodiment of the present invention, at least one video including a sports event and log information sequentially recording an event occurring in the sports event are obtained, and a preset motion is included in the video including the acquired sports event. And an operation of dividing the scene of at least one of the divided scenes, and an operation of detecting at least one pre-set information on the sporting event based on a frame constituting the divided at least one scene.

In addition, the operation of dividing the at least one scene may include an operation of detecting a scene change in a video including the sports game and dividing the video including the sports game into a plurality of scenes. An operation of classifying each of the plurality of scenes into a preset class by applying scene recognition, an operation of selecting a class matching the preset operation from among the preset classes, and It may include an operation of selecting a scene in which a set operation occurs.

In addition, the scene change may be detected by converting RGB to HSV and using a histogram difference of the HSV.

In addition, the scene change may be determined by determining a histogram difference between a previous frame and a current frame for each channel of the HSV, and comparing the determined histogram difference with a preset reference.

In addition, the scene recognition may classify each of the plurality of scenes based on a scene recognizer score for a frame constituting each of the plurality of scenes.

In addition, the method for detecting sports event information may further include an operation of tagging log information related to a scene in which the preset motion occurs to a scene in which the preset motion occurs.

In addition, the tagging of the scene in which the preset operation occurs may include an operation of grasping situation information of the scene in which the preset operation occurs through a relay board included in the scene in which the preset operation occurs, and the identified situation. An operation of extracting tag information on a scene in which the preset operation occurs by comparing information and the log information, and an operation of tagging the extracted tag information to a scene in which the preset operation occurs.

In addition, the detecting of the at least one sports event information may include detecting the position of at least one object based on a frame constituting the divided at least one scene, and at least in the divided at least one scene. It may include an operation of estimating one zone and an operation of detecting at least one pre-set information about the sporting event based on the estimated at least one zone and the detected position of the at least one object.

According to an embodiment of the present invention, there is an effect of detecting at least one piece of information related to a sports event.

In addition, there is an effect of being able to select a scene in which a preset operation occurs.

In addition, there is an effect of tagging preset information on a scene in which a preset motion occurs.

1 is a diagram illustrating a configuration of an apparatus for detecting sports event information according to an exemplary embodiment.
2 is a flowchart illustrating a method of detecting sports event information according to an exemplary embodiment.
3 is a diagram illustrating a graph that serves as a reference for detecting a scene change according to an exemplary embodiment.
4 is a diagram illustrating scene recognition based on deep learning according to an embodiment.
5 is a diagram illustrating an example of extracting tag information about a scene in which a preset operation occurs, according to an exemplary embodiment.
6 is a diagram illustrating a state in which objects are extracted from a scene in which a preset operation occurs, according to an exemplary embodiment.
7 is a diagram illustrating a state in which a zone set according to a photographing position of a video including a sports event is modified according to an exemplary embodiment.
8 is a diagram illustrating a state in which a zone set according to an embodiment is normalized and information on objects is displayed in the normalized zone.
9 is a diagram illustrating a state in which tagging information for a scene in which a preset operation occurs is utilized according to an exemplary embodiment.

Specific structural or functional descriptions of the embodiments according to the concept of the present invention disclosed in the present specification are exemplified only for the purpose of describing the embodiments according to the concept of the present invention, and the embodiments according to the concept of the present invention are It may be implemented in various forms and is not limited to the embodiments described herein.

Since the embodiments according to the concept of the present invention can apply various changes and have various forms, the embodiments will be illustrated in the drawings and described in detail in the present specification. However, this is not intended to limit the embodiments according to the concept of the present invention to specific disclosed forms, and includes all changes, equivalents, or substitutes included in the spirit and scope of the present invention.

Terms such as first or second may be used to describe various constituent elements, but the constituent elements should not be limited by the terms. The terms are only for the purpose of distinguishing one component from other components, for example, without departing from the scope of the rights according to the concept of the present invention, the first component may be referred to as the second component, and similarly The second component may also be referred to as a first component.

When a component is referred to as being "connected" or "connected" to another component, it is understood that it may be directly connected or connected to the other component, but other components may exist in the middle. It should be. On the other hand, when a component is referred to as being "directly connected" or "directly connected" to another component, it should be understood that there is no other component in the middle. Other expressions describing the relationship between components, such as "between" and "directly between" or "adjacent to" and "directly adjacent to" should be interpreted as well.

The terms used in the present specification are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise.

In this specification, terms such as "comprise" or "have" are intended to designate that the described feature, number, step, action, component, part, or combination thereof is present, but one or more other features or numbers It is to be understood that the possibility of addition or presence of, steps, actions, components, parts, or combinations thereof is not preliminarily excluded.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs.

Terms as defined in a commonly used dictionary should be construed as having a meaning consistent with the meaning of the related technology, and should not be interpreted as an ideal or excessively formal meaning unless explicitly defined in the present specification. Does not.

In the following description, the same identification symbols mean the same configuration, and unnecessary redundant descriptions and descriptions of known technologies will be omitted.

In an embodiment of the present invention,'communication','communication network', and'network' may be used with the same meaning. The three terms refer to wired/wireless local and wide area data transmission/reception networks capable of transmitting and receiving files between a user terminal, a terminal of other users, and a download server.

Hereinafter, the present invention will be described in detail by describing a preferred embodiment of the present invention with reference to the accompanying drawings.

1 is a diagram illustrating a configuration of an apparatus for detecting sports event information according to an exemplary embodiment.

Referring to FIG. 1, the apparatus 100 for detecting sports event information includes a processor 110, an input/output interface module 120, and a memory 130.

The processor 110, the input/output interface module 120, and the memory 130 constituting the sports event information detection apparatus 100 are interconnected, and data can be transmitted to each other.

The processor 110 may execute programs or instructions stored in the memory 130. In this case, an operation program (eg, OS) for operating the sports event information detection apparatus 100 may be stored in the memory 130.

The processor 110 may execute a program for managing information on the sports event information detection apparatus 100.

The processor 110 may execute a program for managing the operation of the apparatus 100 for detecting sports event information.

The processor 110 may execute a program for managing the operation of the input/output interface module 120.

The processor 110 may acquire a video including a sports event and log information sequentially recording an event occurring in the sports event through the input/output interface module 120. In this case, the video including the sports event may be a video transmitted by a broadcasting station, a video provided as a streaming service, or a video stored in a separate storage device, but the video including the sports game is not limited thereto. In addition, the log information is information sequentially recording events occurring in the sports event, and may be information recorded according to the order in which the events occur. In addition, according to an embodiment, when the sporting event is baseball, the log information may include information on the situation and information on the subject, and the information on the situation may include innings, scores, ball counts, runner status, pitches, etc. The information on the subject may be a batter, a pitcher, a runner, and the like.

Ⅰ) Scene segmentation with preset motions

The processor 110 may divide at least one scene including a preset motion from the video including the acquired sports event.

The processor 110 utilizes scene change detection, scene recognition, motion recognition, or relay board (relay board caption information) recognition to divide at least one scene including a preset motion from the video including the acquired sports event. However, a method that can be used to divide at least one scene including a preset motion in the video including the obtained sports event is not limited thereto.

The processor 110 may detect a scene change in the video including the sports game and divide the video including the sports game into a plurality of scenes.

The processor 110 may color convert RGB into HSV (Hue, Saturation, Value, hereinafter referred to as HSV) in the video including the sports event.

The processor 110 may perform HSV Pixel Quantization on the converted HSV. For example, the processor 110 may perform HSV Pixel Quantization from 24 bits to 10 bits. Here, 10 bits is an example and represents a bit of a corresponding frame corresponding to the degree to which a scene change is detected.

The processor 110 may determine a histogram difference between the previous frame and the current frame for each channel of the HSV. In this case, the processor 110 may determine an average of the histogram differences in relation to all bins (one section of the histogram).

The processor 110 may detect whether a scene is changed by comparing the average of the histogram difference with a preset reference. For example, when the average of the histogram differences is higher than a preset threshold (eg, 7000), it may be detected that the current frame has changed the scene from the previous frame. In addition, when the average of the histogram difference is lower than the threshold, it may be detected that the current frame and the previous frame have not changed scenes.

The processor 110 may classify each of the plurality of scenes into a preset class by applying scene recognition to each of the divided plurality of scenes.

The processor 110 may apply deep learning-based image scene recognition to each of the plurality of scenes classified by scene change detection.

The processor 110 may classify each of the plurality of scenes classified by scene change detection into a preset class by applying deep learning having a ResNet v2 structure as an example of deep learning. In this case, the preset class may be different for each sporting event.

According to an embodiment, when the sports game is a baseball game, the processor 110 may classify each of the plurality of scenes classified by scene change detection into a class related to a baseball game video. At this time, the class related to the baseball game video may be pitching, pitching enlargement, close-up, field (ball tracking), advertisement, commentary, spectator seat, dugout, game sketch, field view, or double screen (PIP), but the baseball game video The class related to is not limited thereto.

The processor 110 applies deep learning having a ResNet v2 structure to the frames included in each of the plurality of scenes classified by scene change detection, and classifies the scenes of the corresponding frame into the most matching class among certain criteria. . In this case, the processor 110 accumulates a scene recognizer score for all frames included in each of the plurality of scenes classified by scene change detection, and identifies each of the plurality of scenes classified by scene change detection as a class having the highest score. Can be classified.

The processor 110 may select a class matching the preset operation from among the preset classes.

According to an embodiment, when the sports game is a baseball game and the preset motion is pitching, the processor 110 may select only scenes classified as pitching among the classes.

The processor 110 may delete each of the plurality of scenes classified by scene change detection included in an unselected class.

According to an embodiment, when the sports game is a baseball game and the preset motion is pitching, the processor 110 may delete scenes classified into classes other than pitching.

The processor 110 may select a scene in which the preset operation occurs from a scene classified by the selected class.

The processor 110 may use motion recognition or relay board (relay board subtitle information) recognition to select a scene in which the preset operation occurs, but it is limited to use it to select a scene in which the preset operation occurs. It does not become.

According to an embodiment, when the sports game is a baseball game and the preset motion is pitching, the processor 110 selects only the scene in which the actual pitching occurs through motion recognition or relay board (relay board caption information) recognition. I can.

Ii) tagging

The processor 110 may tag log information related to the scene in which the preset operation occurs to the scene in which the preset operation occurs.

The processor 110 may grasp the situation information of the scene in which the preset operation occurs through the relay board (relay board caption information) included in the scene in which the preset operation occurs.

According to an embodiment, when the sporting game is a baseball game and the preset motion is pitching, the processor 110 transmits an inning, a top, a bottom, and the like through a relay board (relay board caption information). It is possible to grasp the situation information of the scene where the pitching occurs, such as score, ball count, runner status, and number of pitches.

The processor 110 may compare the identified situation information with the log information to extract tag information about a scene in which the preset operation occurs.

The processor 110 may extract tag information about a scene in which a preset operation occurs by comparing the identified information with log information sequentially recording an event occurring in the sports event.

According to an embodiment, when the sports game is a baseball game and the preset action is pitching, the processor 110 detects the situation information of the scene where the pitching occurs and the baseball game log identified through the relay board (relay board subtitle information). Tag information may be extracted from the baseball game log information through comparison with the information. At this time, the tag information extracted from the baseball game log information may be a sequence number, an inning, a ball count, a runner situation, a pitcher name, a pitcher's number of pitches, a batter's name, the number of pitches of a batter's at-bat, pitch quality, pitching result, etc., but the baseball game log The tag information extracted from the information is not limited thereto.

The processor 110 may extract tag information about the scene for each scene in which a preset operation occurs.

The processor 110 may tag the extracted tag information to a scene in which the preset operation occurs.

According to an embodiment, when the sports game is a baseball game and a preset motion is pitching, the processor 110 may tag the extracted tag information to a scene in which the pitching action occurs.

The processor 110 may tag tag information extracted with respect to the scene for each scene in which the preset operation occurs.

Iii) Detecting sports event information

The processor 110 may detect the position of at least one object based on the frame constituting the divided at least one scene.

The processor 110 may detect the location of the object as an (x, y) coordinate of an image reference point.

According to an embodiment, when the sporting event is a baseball game and the preset motion is pitching, the processor 110 determines the pitching request position of the catcher and the pitcher actually thrown with respect to the frame constituting the divided at least one scene. The position of the ball can be detected by the (x, y) coordinate of the image reference point.

The processor 110 may use a deep learning-based position detector to detect the position of at least one object based on the frame constituting the divided at least one scene.

According to an embodiment, when the sporting event is a baseball game and the preset motion is pitching, the processor 110 requests the catcher's throwing position where the catcher's mitt position, detected in the frame before the pitching motion is recognized, stays the longest. It can be detected by location.

According to an embodiment, when the sporting event is a baseball game and the preset motion is pitching, the processor 110 determines the position of the catcher's pitching request that is detected immediately after the catcher receives the ball with the meat and no longer detects the baseball. Can be detected as the position of the pitcher's actual thrown ball.

The processor 110 may estimate at least one zone in the at least one divided scene. In this case, the estimated position and size of the at least one zone may vary according to a sports event.

The processor 110 may use the sporting event rule to estimate the at least one zone.

According to an embodiment, when the sports game is a baseball game and a preset motion is pitching, the processor 110 may estimate the strike zone using the baseball game rules. At this time, the strike zone according to the baseball game rules exists above the home plate, the upper limit is the middle of the batter's shoulder and waist, and the lower limit is below the batter's knee.

The processor 110 may estimate the at least one zone through deep learning learning.

According to an embodiment, when the sports game is a baseball game and the preset motion is pitching, the processor 110 learns a model through the strike zone Ground Truth DB, and estimates the strike zone through the learned model. I can.

The processor 110 may modify the estimated location or size of the zone in consideration of the location of the cameras photographing the sports event, the number of cameras, and images transmitted through broadcast.

The processor 110 compares the size or position of the at least one zone estimated in the at least one divided scene by comparing the size of the standard strike zone to reduce the at least one zone set in the at least one divided scene. Or you can decide the magnification ratio.

The processor 110 normalizes the at least one zone set in the at least one divided scene based on the determined reduction or enlargement ratio, and detects the at least one divided scene in the at least one normalized zone. The location of at least one object that has been created may be displayed.

According to an embodiment, when the sports game is a baseball game and the preset motion is pitching, the processor 110 normalizes the estimated strike zone in the at least one divided scene based on the determined reduction or expansion ratio. In the normalized strike zone, the catcher's mitt position, which is the detected pitcher's pitching request position, and the actual pitcher's pitching position may be displayed.

The processor 110 normalizes the at least one zone estimated in the at least one divided scene based on the determined reduction or enlargement ratio, and the position of the at least one object displayed in the at least one normalized zone. At least one of the sports event information set in advance may be detected on the basis of.

According to an embodiment, when the sports game is a baseball game and the preset motion is pitching, the processor 110 normalizes the strike zone set in the at least one divided scene based on the determined reduction or expansion ratio. And, based on the actual pitcher's pitching position indicated in the normalized strike zone, whether or not the pitcher throws a strike can be detected.

According to an embodiment, when the sports game is a baseball game and the preset motion is pitching, the processor 110 normalizes the strike zone set in the at least one divided scene based on the determined reduction or expansion ratio. And, based on the pitcher's pitch request position and the actual pitcher's pitching position indicated in the normalized strike zone, information on how accurately the pitcher throws to the position requested by the catcher can be detected.

The input/output interface module 120 may be connected to an external device (eg, a server) through a network.

The input/output interface module 120 may obtain data from an external device.

The input/output interface module 120 may acquire a video including a sports event and log information sequentially recording an event occurring in the sports event.

The input/output interface module 120 may acquire learning data for learning.

The input/output interface module 120 may obtain a user's input.

The input/output interface module 120 may be provided integrally with the sports event information detection device 100.

The input/output interface module 120 may be provided separately from the sports event information detection device 100.

The input/output interface module 120 may be a separate device to be communicatively connected with the sports event information detection device 100.

The input/output interface module 120 may include a port (eg, a USB port) for connecting to an external device.

The input/output interface module 120 may include a monitor, a touch screen, a mouse, an electronic pen, a microphone, a keyboard, a speaker, an earphone, a headphone, or a touch pad.

The memory 130 may store data acquired through the input/output interface module 120.

The memory 130 may store a plurality of scenes divided by the processor 110.

The memory 130 may store a class for the processor 110 to classify scenes.

The memory 130 may store context information identified by the processor 110.

The memory 130 may store tag information extracted by the processor 110.

The memory 130 may store a scene tagged by the processor 110.

The term'module' used herein denotes a logical structural unit, and it is obvious to those skilled in the art that the present invention is not necessarily a physically classified component.

2 is a flowchart illustrating a method of detecting sports event information according to an exemplary embodiment.

Referring to FIG. 2, the apparatus for detecting sports event information acquires a video including a sports event and log information sequentially recording an event occurring in the sports event (200).

In this case, the apparatus for detecting sports event information may obtain log information sequentially recording the sport event and the event occurring in the sport event from different devices (eg, servers).

The sports event information detection device selects a scene in which a preset motion occurs in the sports event video (210).

In this case, the apparatus for detecting sports event information may detect a scene change in the video including the sports event and divide the video including the sports event into a plurality of scenes.

In addition, the apparatus for detecting sports event information may classify each of the plurality of scenes classified by scene change detection into a preset class by applying deep learning having a ResNet v2 structure as an example of deep learning.

In addition, the apparatus for detecting sports event information may select a class matching the preset motion from among the preset classes.

In addition, the apparatus for detecting sports event information may select a scene in which the preset operation occurs using motion recognition or relay board (relay board caption information) recognition.

The sports event information detection apparatus detects situation information of a scene in which the preset motion occurs through a relay board included in the scene in which the preset motion occurs (220).

In this case, the apparatus for detecting sports event information may recognize a character included in the relay board to determine situation information of a scene in which the preset operation occurs.

The sports event information detection apparatus compares the identified situation information with the log information, extracts tag information for a scene in which the preset motion occurs, and tags the extracted tag information to a scene in which the preset motion occurs. (230).

In this case, the apparatus for detecting sports event information may extract tag information on a scene in which a preset operation occurs by comparing the identified information with log information sequentially recording events occurring in the sports event.

In addition, the apparatus for detecting sports event information may tag tag information extracted with respect to the scene for each scene in which the preset operation occurs.

The sports event information detection device detects the position of an object in a scene in which a preset motion occurs in the selected sports event video (240).

In this case, the sports event information detection apparatus may use a deep learning-based position detector to detect the position of at least one object based on a frame constituting a scene in which a preset motion occurs in the selected sports event video. .

The sports event information detection apparatus estimates at least one zone in a scene in which a preset motion occurs in the selected sports event video (250).

In this case, the apparatus for detecting sports event information may use the sports event rule to estimate the zone.

In addition, the sports event information detection apparatus compares the size of the at least one zone set in a scene in which a preset motion occurs in the selected sports event video with the size of the zone in the sports event rule, and the divided at least one The reduction or enlargement ratio of the at least one zone set in the scene of may be determined.

In addition, the apparatus for detecting sports event information may normalize the estimated zone by applying the determined reduction or expansion ratio of the zone to the estimated zone.

The sports event information detection apparatus detects at least one pre-set information about the sport event based on the estimated zone and the position of the detected at least one object (260).

In this case, the apparatus for detecting sports event information may detect sports event information by displaying the position of the detected at least one object in the estimated zone.

3 is a diagram illustrating a graph that serves as a reference for detecting a scene change according to an exemplary embodiment.

Referring to FIG. 3, the apparatus for detecting sports event information may color convert RGB into Hue, Saturation, Value (HSV). The apparatus for detecting sports event information may perform HSV Pixel Quantization on the converted HSV.

The apparatus for detecting sports event information may determine a difference in histogram between the previous frame and the current frame for each channel of the HSV.

The apparatus for detecting sports event information may determine an average of the histogram differences in relation to all bins (one section of the histogram).

The apparatus for detecting sports event information may detect whether a scene is changed by comparing an average of the histogram difference with a preset reference.

When the average of the histogram differences is higher than a preset threshold (eg, 7000), the apparatus for detecting sports event information may determine that the current frame has changed the scene from the previous frame.

When the average of the histogram difference is lower than a preset threshold (eg, 7000), the apparatus for detecting sports event information may determine that the current frame has not changed the scene from the previous frame.

4 is a diagram illustrating scene recognition based on deep learning according to an embodiment.

Referring to FIG. 4, the apparatus for detecting sports event information may classify each of the plurality of scenes classified by scene change detection into a preset class by applying deep learning having a ResNet v2 structure as an example of deep learning. have. In this case, the preset class may be different for each sporting event.

The apparatus for detecting sports event information may acquire training data to apply deep learning having a ResNet v2 structure, and train a scene recognizer using the acquired training data.

According to an embodiment, when the sports game is a baseball game, the apparatus for detecting sports game information may classify each of the plurality of scenes classified by scene change detection into a class related to a baseball game video. At this time, the class related to the baseball game video is pitching 401, pitching enlargement 403, close-up 405, field (ball tracking) 407, advertisement 409, commentary 411, audience seats 413, The dugout 415, the game sketch 417, the field view 419, or the double screen (PIP) 421 may be used, but the class related to the baseball game video is not limited thereto.

The apparatus for detecting sports event information can classify the scenes of the frame into the class that matches the most among certain criteria by applying deep learning having a ResNet v2 structure to the frames included in each of the plurality of scenes classified by scene change detection. have. In this case, the apparatus for detecting sports event information accumulates a scene recognizer score for all frames included in each of the plurality of scenes classified by scene change detection, and each of the plurality of scenes classified by scene change detection into a class having the highest score. Can be classified.

5 is a diagram illustrating an example of extracting tag information about a scene in which a preset operation occurs, according to an exemplary embodiment.

Referring to FIG. 5, the apparatus for detecting sports event information may recognize situation information of a scene in which a preset motion occurs through a relay board.

The apparatus for detecting sports event information may extract tag information on a scene in which a preset operation occurs by comparing the identified situation information with log information sequentially recording an event occurring in a sports event.

The tag information extracted by the sports event information detection device may be a sequence number, innings, a ball count, a runner situation, a pitcher's name, a pitcher's number of pitches, a batter's name, the number of pitches of a batter's at-bat, pitch quality, and pitching result.

According to an embodiment, when the information detected by the sports event information detection device through the relay board is the first, score 0-0, the ball count 0/0/2, no runners, and the number of pitches is 8, the sports event information detection device logs The tag information 500 corresponding to the game information 501-a or the speed 501-q of the pitcher thrown by the pitcher may be extracted from the information.

Specifically, the tag information 500 is the 8th pitch, the first inning, 0/0/2, no runners, Luchinski, the 8th pitch of the pitcher, Jeon Jun-woo, the first ball at bat, two-seam, and the ball in the game between NC and Lotte. .

According to an embodiment, when the information detected by the sports event information detection device through the relay board is 8 seconds, score 1-1, ball count 0/1/2, no runners, and number of pitches 7, the sports event information detection device logs From the information, tag information 510 corresponding to the game information 511-a or the speed 511-q of the pitcher thrown may be extracted.

Specifically, the tag information 510 is the 220th pitch, the 8th inning, 0/1/2, no runner, Bae Jae-hwan, the pitcher's 7th pitch, Lee Dae-ho, the second ball at bat, the direct ball, and the ball in the NC and Lotte game.

The apparatus for detecting sports event information may tag the extracted tag information to a scene in which a preset motion occurs.

6 is a diagram illustrating a state in which objects are extracted from a scene in which a preset operation occurs, according to an exemplary embodiment.

Referring to FIG. 6, the apparatus for detecting sports event information may detect the location of at least one object based on a frame constituting a scene in which a selected predetermined motion occurs.

When the preset motion is pitching, the apparatus for detecting the sports event information may detect the pitching request position of the baseball and the catcher for all frames in the pitching scene section.

According to an embodiment, when the preset motion is pitching, the apparatus for detecting sports event information determines the position where the catcher's mitt position, detected in the frame 600 before the pitching motion is recognized, stays the longest, the catcher's pitching request position (601). Can be detected.

According to an embodiment, when the preset motion is pitching, the apparatus for detecting sports event information may detect the position of the pitcher thrown at the same time as the pitcher thrown out of the pitcher's hand.

According to an embodiment, the apparatus for detecting sports event information pitches the position 611 of the catcher's mitt detected in the first frame 610 when the pitcher receives the ball thrown by the pitcher as a mitt and the ball thrown by the pitcher is no longer detected. Can be detected by the position of the actual thrown ball.

7 is a diagram illustrating a state in which a zone set according to a photographing position of a video including a sports event is modified according to an exemplary embodiment.

Referring to FIG. 7, according to an embodiment, when a preset motion is pitching, the apparatus for detecting sports event information may estimate a position of a strike zone in a video including a sports event. At this time, since the camera that shoots the sports game is not a fixed camera, the field of view (FOV) may vary for each batsman. Therefore, it is necessary to estimate the position of the strike zone for each pitching scene.

According to an embodiment, when the preset motion is pitching, the apparatus for detecting sports game information may estimate a strike zone from a video including a baseball game using a baseball game rule.

According to an embodiment, when the preset motion is pitching, the apparatus for detecting sports event information may learn a model through a strike zone Ground Truth DB, and estimate a strike zone through the learned model.

According to an embodiment, when the preset motion is pitching, the apparatus for detecting sports game information may estimate the strike zone 701 using baseball rules or deep learning in the frame 700 included in the baseball game video. However, since the position of the pitcher and the catcher in the baseball game video is not a straight line due to the location of the camera that photographs the frame 700 included in the baseball game video (702), the sports game information detection device is the location of the camera that shoots the baseball game video. It is possible to correct the position 703 of the strike zone in consideration of.

According to an embodiment, when the preset motion is pitching, the apparatus for detecting sports game information may estimate the strike zone 711 using a baseball rule or deep learning in a frame 710 included in a baseball game video. However, because the position of the pitcher and the catcher in the baseball game video is not a straight line due to the location of the camera that photographs the frame 710 included in the baseball game video (712), the sports game information detection device is the location of the camera that shoots the baseball game video. The position 713 of the strike zone may be corrected in consideration of.

The range (711, 713) in which the strike zone of the frame 710 included in the baseball game video in which the position of the camera for shooting the baseball game video is skewed to the right is corrected, and the position of the camera for shooting the baseball game video is less to the right. It is larger than the strike zone correction ranges 701 and 703 of the frame 700 included in the biased baseball game video.

8 is a diagram illustrating a state in which a zone set according to an embodiment is normalized and information on objects is displayed in the normalized zone.

Referring to FIG. 8, according to an embodiment, when the preset motion is pitching, the apparatus for detecting sports game information includes a strike zone estimated from a baseball game image and a strike zone estimated using a baseball rule or deep learning. It is possible to detect a reduction or enlargement ratio of the size of the strike zone estimated from the baseball game image by comparing the size of.

According to an embodiment, when the preset motion is pitching, the apparatus for detecting sports game information may normalize the estimated strike zone in the baseball game image according to the detected reduction or enlargement ratio information.

The sports game information detection device may display the estimated position of the catcher and the actual pitching position estimated in the baseball game in the normalized strike zone.

According to an embodiment, when the preset motion is pitching, the sports game information detection apparatus may normalize (801) the estimated strike zone from the baseball game image, and the detected pitching motion is detected in the frame 800 before being recognized. The position where the catcher's mitt position stays the longest may be detected as the position 802 of the catcher's pitching request.

According to an embodiment, when the preset motion is pitching, the sports game information detection apparatus may normalize (811) the estimated strike zone from the baseball game image, and the pitcher may be at the same time as the ball thrown by the pitcher leaves the pitcher's hand. The position of the thrown ball can be detected, and the position 811 of the catcher's mitt detected in the first frame 810 when the pitcher's thrown ball is received as the catcher's mitt and the pitcher's thrown ball is no longer detected It can be detected by the position of the thrown ball.

The apparatus for detecting sports event information may display the pitching request position 821 of the catcher and the position 822 of the ball actually thrown by the pitcher in the normalized strike zone 820.

9 is a diagram illustrating a state in which tagging information for a scene in which a preset operation occurs is utilized according to an exemplary embodiment.

Referring to FIG. 9, FIGS. 9(a) and 9(b) show a state in which tagging information is expressed.

Referring to FIG. 9 (a), in FIG. 9 (a), information on the game date and time, innings, pitcher and batter 901 are displayed, and the requested position in the normalized strike zone (the position requested by the catcher, hereinafter referred to as ``request''). Position') and pitching position 903 are displayed.

In addition, information 902 indicating the requested position, the pitching position, and the pitch type is also displayed.

In Fig. 9(a), the fourth pitch (marked 4) about the batter Oh Yoon-seok almost matches the position requested by the catcher and the position the pitcher pitched, but the first pitch (marked 1) is the position where the catcher requested pitch and the pitcher. It can be seen that the pitched position is very different. Using such tagging information, a history of the pitcher's actual pitching position compared to the pitcher's pitching request position is created, and the created history is analyzed to determine the pitcher's ability to throw.

Referring to FIG. 9(b), in FIG. 9(b), information on the game date, innings, pitcher, and batter 911 is displayed, and the required position 913 of the catcher is displayed in the normalized strike zone.

In addition, information 912 indicating the requested position, the pitching position, and the pitch type is also displayed.

Using the tagging information shown in FIG. 9(b), a history of the pitcher's pitching request position for each batter is created, and by analyzing the created history, the intention of the pitcher or the catcher against the batter can be grasped.

In the above, even if all the constituent elements constituting the embodiments of the present invention are described as being combined into one or operating in combination, the present invention is not necessarily limited to these embodiments. That is, within the scope of the object of the present invention, all the constituent elements may be selectively combined into at least one and operated.

In addition, although all of the components may be implemented as one independent hardware, a program module that performs some or all functions combined in one or more hardware by selectively combining some or all of the components. It may be implemented as a computer program having Codes and code segments constituting the computer program may be easily inferred by those skilled in the art of the present invention.

Such a computer program is stored in a computer-readable storage medium, and is read and executed by a computer, thereby implementing an embodiment of the present invention. The storage medium of the computer program may include a magnetic recording medium, an optical recording medium, and the like.

In addition, the terms such as "include", "consist of" or "have" described above mean that the corresponding component may be included unless otherwise stated, excluding other components. It should not be construed as being able to further include other components.

All terms, including technical or scientific terms, unless otherwise defined, have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs. Terms generally used, such as terms defined in the dictionary, should be interpreted as being consistent with the meaning in the context of the related technology, and are not interpreted as ideal or excessively formal meanings unless explicitly defined in the present invention.

The methods disclosed in the present invention comprise one or more actions or steps for achieving the above-described method. Method actions and/or steps may be interchanged with each other without departing from the scope of the claims. In other words, unless a specific order for the actions or steps is specified, the order and/or use of specific actions and/or steps may be modified without departing from the scope of the claims.

As used herein, a phrase referring to “at least one of” in a list of items refers to any combination of these items, including single members. As an example, “at least one of a, b, or c:” means a, b, c, ab, ac, bc, and abc, as well as any combination with multiples of the same element (e.g., aa , aaa, aab, aac, abb, acc, bb, bbb, bbc, cc, and ccc or any other ordering of a, b, and c).

As used herein, the term "determining" encompasses a wide variety of actions. For example, “determining” may include computing, computing, processing, deriving, examining, looking up (eg, looking up in a table, database, or other data structure), identifying, and the like. . Further, “determining” may include receiving (eg, receiving information), accessing (accessing data in a memory), and the like. Also, “determining” may include resolving, choosing, choosing, establishing, and the like.

The above description is merely illustrative of the technical idea of the present invention, and those of ordinary skill in the art to which the present invention pertains will be able to make various modifications and variations without departing from the essential characteristics of the present invention.

Accordingly, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention, but to explain the technical idea, and the scope of the technical idea of the present invention is not limited by these embodiments. The scope of protection of the present invention should be construed by the following claims, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present invention.

100... sports event information detection device

Claims (16)

In the sports event information detection device,
Including at least one processor,
The at least one processor,
Acquiring log information sequentially recording a video including a sports event and an event occurring in the sports event, and dividing at least one scene including a preset motion from the video including the acquired sports event,
A sports event information detection apparatus configured to detect at least one of the sports event information set in advance based on a frame constituting the divided at least one scene. The method of claim 1,
The at least one processor,
By detecting a scene change in the video including the sports game, the video including the sports game is divided into a plurality of scenes,
By applying scene recognition to each of the divided plurality of scenes, each of the plurality of scenes is classified into a preset class,
Selecting a class matching the preset operation from among the preset classes,
A sports event information detection device that selects a scene in which the preset motion occurs from a scene classified by the selected class. The method of claim 2,
The above scene change,
An apparatus for detecting sports event information that converts RGB to HSV and is detected using a histogram difference of the HSV. The method of claim 3,
The above scene change,
A sports event information detection apparatus for determining a histogram difference between a previous frame and a current frame for each channel of the HSV, and comparing the determined histogram difference with a preset reference to determine the difference. The method of claim 2,
The scene recognition,
A sports event information detection apparatus for classifying each of the plurality of scenes based on a scene recognizer score for a frame constituting each of the plurality of scenes. The method of claim 2,
The at least one processor,
A sports event information detection device for tagging log information related to a scene in which the preset motion occurs to a scene in which the preset motion occurs. The method of claim 6,
The at least one processor,
Recognizing situation information of the scene in which the preset motion occurs through a relay board included in the scene in which the preset motion occurs,
By comparing the identified situation information and the log information, tag information for a scene in which the preset operation occurs is extracted,
A sports event information detection device for tagging the extracted tag information to a scene in which the preset operation occurs. The method of claim 1,
The at least one processor,
Detecting the position of at least one object based on the frame constituting the divided at least one scene,
Estimating at least one zone in the at least one divided scene,
A sports event information detection apparatus configured to detect at least one preset sport event information based on the estimated at least one zone and the detected position of the at least one object. Obtaining log information sequentially recording a video including a sports game and an event occurring in the sports game, and dividing at least one scene including a preset motion from the video including the obtained sports game; And
An operation of detecting at least one pre-set sports event information based on a frame constituting the divided at least one scene
Sports event information detection method comprising a. The method of claim 9,
The operation of dividing the at least one scene,
Detecting a scene change in the video including the sports game and dividing the video including the sports game into a plurality of scenes;
Classifying each of the plurality of scenes into a preset class by applying scene recognition to each of the divided plurality of scenes;
Selecting a class matching the preset operation from among the preset classes; And
Selecting a scene in which the preset operation occurs from scenes classified by the selected class
Sports event information detection method comprising a. The method of claim 10,
The above scene change,
A method of detecting sports event information by converting RGB into HSV and detecting the difference in histogram of the HSV. The method of claim 11,
The above scene change,
A method for detecting sports event information for determining a histogram difference between a previous frame and a current frame for each channel of the HSV, and comparing the determined histogram difference with a preset criterion. The method of claim 10,
The scene recognition,
A sports event information detection method for classifying each of the plurality of scenes based on a scene recognizer score for a frame constituting each of the plurality of scenes. The method of claim 10,
The sports event information detection method,
An operation of tagging log information related to the scene in which the preset operation occurs to the scene in which the preset operation occurs
Sports event information detection method further comprising. The method of claim 14,
An operation of tagging a scene in which the preset operation occurs,
Grasping context information of a scene in which the preset operation occurs through a relay board included in a scene in which the preset operation occurs;
Comparing the identified situation information with the log information, and extracting tag information about a scene in which the preset operation occurs; And
An operation of tagging the extracted tag information to a scene in which the preset operation occurs
Sports event information detection method comprising a. The method of claim 9,
The operation of detecting the at least one sports event information,
Detecting a position of at least one object based on the frame constituting the divided at least one scene;
Estimating at least one zone in the at least one divided scene; And
An operation of detecting at least one pre-set information about the sporting event based on the estimated at least one zone and the detected position of the at least one object
Sports event information detection method comprising a.

Images