KR102248456B1 - Method and apparatus for highlight extraction of sports game - Google Patents

Abstract

It may be a highlight extraction method for automatically extracting highlights from a video including a sports event. In this case, the highlight extraction method includes: identifying a video including a sports event, log information sequentially recording an event occurring in the sports event, and a keyword related to the video; Tagging game information related to the video to the video; Extracting at least one log information corresponding to the keyword and determining at least one frame corresponding to the extracted log information from the tagged video; And generating a highlight video by combining the determined at least one frame.

Description

Method and apparatus for extracting highlights of sports events {METHOD AND APPARATUS FOR HIGHLIGHT EXTRACTION OF SPORTS GAME}

The following description relates to a method and apparatus for extracting highlights of a sports event.

A user or an expert can create a highlight video of a sports event by designating a starting point and an ending point of the frame while watching the video. In the case of such a manual operation, time and cost are consumed in generating a highlight video from a large amount of video. Therefore, there is a need for a highlight extraction method that automatically generates a highlight video of a sports event instead of manually.

According to an aspect, a method for extracting highlights includes an operation of identifying a video including a sports event, log information sequentially recording an event occurring in the sports event, and a keyword related to the video; Tagging game information related to the video to the video; Extracting at least one log information corresponding to the keyword and determining at least one frame corresponding to the extracted log information from the tagged video; And generating a highlight video by combining the determined at least one frame.

The operation of generating the highlight video in the highlight extraction method may include generating one highlight video corresponding to a single keyword when the keyword is one, or at least one highlight video corresponding to a plurality of keywords when the keyword is plural. You can create a highlight video by combining the highlight video.

In the highlight extraction method, the tagging of the video includes: when the relay board is displayed on the video, the game information displayed on the relay board is recognized, and the game information is tagged to a corresponding frame of the video, or the relay board When not displayed on the video, tagging that the game information corresponding to the relay board does not exist in the corresponding frame of the video may be included.

The highlight extraction method may further include an operation of extracting a feature from the relay board displayed in a preset area in the video, and classifying the extracted feature using previously learned data.

In the highlight extraction method, game information displayed on the relay board may include a score, a ball count, and an inning.

In the highlight extraction method, when the game information displayed on the relay board further includes a runner situation, the runner situation is tagged to a corresponding frame of the video, and the game information displayed on the relay board further includes the number of pitches. , The pitched number may be tagged to a corresponding frame of the video.

In the highlight extraction method, the tagging of the video may include detecting a scene change and tagging a corresponding frame of the video, or recognizing a scene according to a certain criterion and tagging a corresponding frame of the video. .

In the highlight extraction method, the scene change may be detected by converting RGB to HSV and using a histogram difference of the HSV.

In the highlight extraction method, the scene change may be determined by determining a histogram difference between a previous frame and a current frame for each channel of the HSV, and comparing the determined histogram difference with a preset reference.

In the highlight extraction method, the scene recognition may be set as a reference to which a corresponding frame is matched from among certain criteria set by the highlight extraction device.

In the highlight extraction method, the at least one frame may include a start frame of a target pitch number-1 corresponding to a situation included in the keyword to a last frame of the target pitch number.

The highlight extraction method may further include an operation of excluding a frame that does not correspond to the keyword from the start frame of the target pitch number-1 to the last frame of the target pitch number.

According to an aspect, it includes a processor and a memory including a computer-readable instruction, and when the instruction is executed by the processor, the processor sequentially records a video including a sports event and an event occurring in the sports event. Identify log information and a keyword related to the video, tag game information related to the video to the video, extract at least one log information corresponding to the keyword, and extract log information from the tagged video It may be a highlight extraction device that determines at least one corresponding frame and combines the determined at least one frame to generate a highlight video.

When generating the highlight video, the processor of the highlight extraction device generates one highlight video corresponding to a single keyword when the keyword is one, or at least one corresponding to a plurality of keywords when the keyword is plural. You can create a highlight video by combining the highlight videos of.

When tagging the video, the processor of the highlight extraction device recognizes game information displayed on the relay board when the relay board is displayed on the video, and tags the game information to a corresponding frame of the video, or When the video is not displayed, it is possible to tag the corresponding frame of the video that the game information corresponding to the relay board does not exist.

The processor of the highlight extraction device may extract a feature from the relay board displayed in a preset area in the video, and classify the extracted feature using previously learned data.

The game information displayed on the relay board in the highlight extraction device may include a score, a ball count, and an inning.

The processor of the highlight extraction device may tag the runner situation to a corresponding frame of the video when the game information displayed on the relay board further includes a runner situation, or the game information displayed on the relay board indicates the number of pitches. If further included, the number of pitches may be tagged to a corresponding frame of the video.

When tagging the video, the processor of the highlight extraction device may detect a scene change and tag a corresponding frame of the video, or recognize a scene according to a certain criterion and tag a corresponding frame of the video. .

The processor of the highlight extraction device may convert RGB to HSV and detect the scene change by using a histogram difference of the HSV.

The processor of the highlight extraction apparatus may determine a histogram difference between a previous frame and a current frame for each channel of the HSV, and compare the determined histogram difference with a preset reference to determine whether to change the scene.

The scene recognition in the highlight extraction apparatus may be set as a reference to which a corresponding frame is matched among certain criteria set by the highlight extraction apparatus.

In the highlight extraction apparatus, the at least one frame may include a start frame of a target pitch number-1 corresponding to a situation included in the keyword to a last frame of the target pitch number.

The processor of the highlight extraction apparatus may exclude a frame that does not correspond to the keyword from the start frame of the target pitch number -1 to the last frame of the target pitch number.

1 is a diagram illustrating an apparatus for extracting highlights according to an exemplary embodiment.
2 is a diagram illustrating tagging of game information on a video according to an exemplary embodiment.
3 is a diagram illustrating that a relay board is displayed on a video and a relay board is not displayed on a video according to an embodiment.
4 is a diagram illustrating a process of learning a relay board by applying an SVM by a highlight extraction apparatus according to an exemplary embodiment.
5 is a diagram illustrating a process of tagging game information by recognizing a relay board by a highlight extraction apparatus according to an exemplary embodiment.
6 is a diagram illustrating that a highlight extraction apparatus detects a scene change from a frame and recognizes a scene of a corresponding frame according to an exemplary embodiment.
7 shows a graph of scene change detection according to an embodiment.
8 illustrates scene recognition based on deep learning, according to an embodiment.
9 illustrates a process of extracting log information corresponding to a keyword according to an embodiment.
10 illustrates processing a generated highlight video according to an embodiment.
11 illustrates a highlight extraction method performed by a highlight extraction apparatus according to an exemplary embodiment.

Specific structural or functional descriptions of the embodiments are disclosed for the purpose of illustration only, and may be changed and implemented in various forms. Accordingly, the embodiments are not limited to a specific disclosure form, and the scope of the present specification includes changes, equivalents, or substitutes included in the technical idea.

Terms such as first or second may be used to describe various components, but these terms should be interpreted only for the purpose of distinguishing one component from other components. For example, a first component may be referred to as a second component, and similarly, a second component may be referred to as a first component.

When a component is referred to as being "connected" to another component, it is to be understood that it may be directly connected or connected to the other component, but other components may exist in the middle.

Singular expressions include plural expressions unless the context clearly indicates otherwise. In the present specification, terms such as "comprise" or "have" are intended to designate that the described feature, number, step, action, component, part, or combination thereof is present, but one or more other features or numbers, It is to be understood that the presence or addition of steps, actions, components, parts or combinations thereof does not preclude the possibility of preliminary exclusion.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the relevant technical field. Terms as defined in a commonly used dictionary should be construed as having a meaning consistent with the meaning of the related technology, and should not be interpreted as an ideal or excessively formal meaning unless explicitly defined in the present specification. Does not.

Hereinafter, the present embodiment will be described in detail with reference to the accompanying drawings.

1 is a diagram illustrating an apparatus for extracting highlights according to an exemplary embodiment.

The highlight extraction device 100 may receive a video including a sports event. The highlight extraction device 100 may receive a video including a sports event from a broadcasting station or identify it through a separate storage device. The highlight extraction device 100 may generate a highlight video that meets a specific condition from a video including a sports event. Accordingly, a highlight video for a video including a sports event may not be manually generated, but may be automatically generated through the highlight extraction device 1000.

Specifically, the highlight extraction device 100 may identify log information sequentially recording events occurring in a sports event. For example, the log information is information that sequentially records events occurring in a baseball game, such as text relay, and may be information recorded according to the order in which the events occur.

In addition, the highlight extraction device 100 may identify a keyword related to a video. In this case, the keyword may be directly input from the user or may be generated by a separate keyword generation module. For example, a user enters Na Seong-beom No. 30 home run as a keyword, or a separate keyword generation module analyzes articles on various portals generated after a baseball game, and generates Na Seong-beom No. 30 home run as a keyword of the baseball game. I can.

Here, the video and log information including the sports event may be searched by keyword. For example, when the Na Seong-beom No. 30 home run is input as a keyword, the highlight extraction apparatus 100 may generate a highlight video by searching for a video including the Na Seong-beom No. 30 home run and log information related to the video.

The highlight extraction device 100 may tag the video game information related to the video. Tagging the game information on the video will be described in more detail in FIG. 2 below.

The highlight extraction apparatus 100 may extract at least one log information corresponding to a keyword, and determine at least one frame corresponding to the log information extracted from the tagged video. In this case, the highlight extraction apparatus 100 may generate a highlight video by combining at least one frame.

If at least one log information corresponding to the keyword cannot be extracted from the video including the sports event, the highlight video corresponding to the keyword cannot be generated from the video including the corresponding sports event. For example, when a home run is input as a keyword, a highlight video corresponding to the home run in the baseball game cannot be generated if a home run does not occur in a baseball game.

According to an embodiment, the highlight extraction apparatus 100 may generate a highlight video corresponding to a main scene from a sports event. Alternatively, the highlight extraction apparatus 100 may generate a summary image summarizing a corresponding sports event by combining a plurality of highlight videos. As an example of a sports game, a method of extracting highlights for a baseball game will be described below. However, the baseball game is merely an example for a sports game, and is not limited to a baseball game.

2 is a diagram illustrating tagging of game information on a video according to an exemplary embodiment.

The highlight extraction device may identify a video including a baseball game composed of a plurality of frames. The highlight extraction device may receive a video from a broadcasting station in real time or may identify it through a separate storage device.

The highlight extraction device may tag game information and scene information corresponding to a corresponding frame. For example, in frame 1, a score of 0:0 at the end of the first inning, 2 out, 18 pitches, game information including pitch, and scene information may be tagged. For another example, in frame 2, game information and scene information including a score of 1:0 in the second inning, 1 ball-2 strike, 4 pitches, and pitch may be tagged.

Here, the game information may be identified through a relay board as an example. The baseball game video may or may not include a relay board. 3 is a diagram illustrating a picture in which a relay board is displayed on a video and a picture in which the relay board is not displayed on a video, as an embodiment. The relay board may be displayed in a preset area of the video. The relay board may contain information about the baseball game in question. For example, the relay board may include game information for a corresponding frame such as innings, scores, ball counts, runner status, and number of pitches.

According to an embodiment, in the case of a frame on which a relay board is displayed, the highlight extraction device may tag the match information (eg, innings, scores, ball counts, runner status, pitches, etc.) identified through the relay board to the frame. have. According to another embodiment, in the case of a frame in which the relay board is not displayed, the highlight extraction apparatus may tag the frame with no relay board.

4 is a diagram illustrating a process of learning a relay board by applying an SVM by a highlight extraction apparatus according to an exemplary embodiment.

The relay board displayed in the video is displayed at a fixed position, so the position may not change during the game. Accordingly, the highlight extraction device may extract features of the relay board displayed in a specific area of the moving picture.

According to an embodiment, the apparatus for extracting highlights may recognize a pattern of features extracted using a support vector machine (SVM). Specifically, all relay boards displayed on a video broadcast by each broadcasting station may be different. For example, relay board 1 displayed on a video broadcast by broadcasting station 1 and relay board 2 displayed on a video broadcast by broadcasting station 2 have the same information (e.g., innings, scores, ball counts, runner status, number of pitches, etc.) Even if it includes, the location of each piece of information in the relay board may be different.

According to an embodiment, the highlight extraction apparatus may learn a relay board of each broadcasting station in advance and extract features from the relay board corresponding to a current frame. Accordingly, the highlight extraction device may recognize the extracted features by applying the SVM, and extract information such as innings, scores, ball counts, runner status, and number of pitches. Even if the relay board of the broadcasting station is changed, the highlight extraction apparatus may extract information by learning the changed relay board.

5 is a diagram illustrating a process of tagging game information by recognizing a relay board by a highlight extraction apparatus according to an exemplary embodiment.

The highlight extraction device may determine whether a relay board is displayed in the current frame of the video. The highlight extraction device may learn in advance about the relay board of each broadcasting station through the process of FIG. 4.

In this case, when the relay board is not displayed on the current frame, the highlight extraction device cannot extract information from the relay board, so that no game information may be tagged in the frame. When the relay board is displayed on the current frame, the highlight extraction device may extract information from the relay board and tag game information on the frame. At this time, the input game information may include innings, scores, and ball counts.

When the relay board further includes game information on the runner situation, the highlight extraction apparatus may extract the runner situation and tag the corresponding frame. If the relay board does not include game information on the runner situation, the highlight extraction device may tag that there is no runner situation.

When the relay board further includes game information on the number of pitches, the highlight extraction device may extract the number of pitches and tag the corresponding frame. If the relay board does not include game information on the number of pitches, the highlight extraction device may tag that there is no number of pitches.

6 is a diagram illustrating that a highlight extraction apparatus detects a scene change from a frame and recognizes a scene of a corresponding frame according to an exemplary embodiment.

According to an embodiment, the apparatus for extracting highlights may recognize not only game information but also scene information, and tag a corresponding frame. In this case, the scene information may include whether to change the scene and recognize the scene.

The scene change detection indicates that the screen of a displayed baseball game has been changed. For example, a frame before the mark 610 indicates a scene related to a pitcher, a frame between the mark 610 and the mark 620 indicates a scene related to a runner stealing, and the frame after the mark 620 indicates a runner Shows the scene showing. Accordingly, the highlight extraction apparatus may detect that a scene is switched between frames in the displays 610 and 620. That is, the highlight extraction apparatus may detect whether a scene is changed from a continuous frame, and tag information on whether or not the detected scene is changed to a corresponding frame. For a detailed description of the scene change detection, refer to FIG. 7 below.

The highlight extraction device may classify scenes of a baseball game video according to a preset criterion. In this case, scene recognition refers to recognizing a criterion matching the current frame. Specifically, the highlight extraction device can set preset criteria such as pitching / pitching enlargement / close-up / field (ball tracking) / advertisement / commentary / spectator seat / dugout / game sketch / ballpark view / double screen (PIP), and It is possible to recognize which criterion the frame corresponds to. For example, the highlight extraction device can recognize the picture 630 as a scene corresponding to a pitch, the picture 640 can be recognized as a scene corresponding to a close-up, and the picture 650 is a scene corresponding to a field. It can be recognized as, the picture 660 can be recognized as a scene corresponding to the commentary, and the picture 670 can be recognized as a scene corresponding to an advertisement. Accordingly, the highlight extraction device may recognize a scene of a corresponding frame and tag the recognized information on the frame. For a detailed description of scene recognition, refer to FIG. 8 below.

7 shows a graph of scene change detection according to an embodiment.

The highlight extraction device may color convert RGB to HSV (Hue, Saturation, Value). The highlight extraction device may perform HSV Pixel Quantization on the converted HSV. For example, the highlight extraction device may perform HSV Pixel Quantization from 24 bits to 10 bits. Here, 10 bits is an example and represents a bit of a corresponding frame corresponding to the degree to which a scene change is detected.

The highlight extraction apparatus may determine a histogram difference between the previous frame and the current frame for each channel of the HSV. In this case, the highlight extraction apparatus may determine an average of the histogram differences in relation to all bins (one section of the histogram). Here, the highlight extraction apparatus may detect whether a scene is changed by comparing the average of the histogram difference with a preset reference. Specifically, when the average of the histogram differences is higher than a threshold (eg, 7000) that is a preset reference, the current frame indicates that the scene has been switched from the previous frame. Alternatively, when the average of the histogram difference is lower than the threshold, the current frame and the previous frame indicate that the scene has not been switched. For example, a frame before display 610 in FIG. 6 is a case in which the average of the histogram differences is lower than a threshold value. For another example, frames before and after the display 610 in FIG. 6 are cases in which the average of the histogram difference is higher than the threshold value.

8 illustrates scene recognition based on deep learning, according to an embodiment.

As an example of deep learning, the highlight extraction device may recognize the type of a screen displayed by a video including a sports event by applying deep learning having a ResNet v2 structure. Specifically, the highlight extraction device may classify the type of screen output from the moving picture according to a certain criterion, and recognize which criterion the displayed screen corresponds to. In this case, a certain criterion may be different for each sporting event. For example, in the case of a baseball game, the highlight extraction device uses certain criteria such as pitching / pitching enlargement / close-up / field (ball tracking) / advertisement / commentary / spectator seat / dugout / game sketch / ballpark view / double screen (PIP). Can be set in advance.

The highlight extraction apparatus may recognize the scene of the frame as the most matching criterion among certain criteria by applying deep learning having a ResNet v2 structure to the frame. In this case, the highlight extraction apparatus may recognize a corresponding frame as a criterion having the highest score by using the scene recognizer score for the frame.

9 illustrates a process of extracting log information corresponding to a keyword according to an embodiment.

The highlight extraction device may identify a keyword. In this case, the keyword is an example of a sports game, and in the case of a baseball game, it may include innings, outs, runner situations, batters, pitchers, events (home runs, etc.). The keyword may be input from the user or may be automatically generated by a separate keyword generation module.

The highlight extraction device may search log information corresponding to a keyword. A picture 910 represents a keyword, and a picture 920 represents searched log information corresponding to the keyword. In this case, the highlight extraction device extracts log information corresponding to information included in the keyword, and may not consider information not included in the keyword.

For example, log information for the keyword 911 for eID 3002 is 7 (921-a, 921-b, 921-c, ~, 921-g) out of a plurality of log information (eg, 630). ) Can be searched. The searched 7 log information 921 represents log information including eID 3002 among a total of 630 log information generated during one game. In this case, eID 3002 is an example, and may represent an event occurring during a sports game such as a home run/scoring situation.

Specifically, the log information (921-a) corresponds to the occurrence of eID 3002 in the 50th pitch thrown by pitcher Park Se-woong against batter Kang Baek-ho, in a situation where a runner is on second out and second base at the end of the third inning, and log information (921- b) corresponds to the occurrence of eID 3002 in the 94th pitch thrown by pitcher Park Se-woong against batter Hwang Jae-gyun, in the situation where there is a runner on second base at the end of the 5th inning, and the log information (921-g) is 2 out of the 10th inning In the situation where there is a runner on the base, it is possible to respond to the occurrence of eID 3002 in the 21st pitch thrown by pitcher Yoon Gil-hyun with the batter Otaegon.

For another example, log information about the keyword 912 for the 8th second/eID 1005 may be searched for two (922-a, 922-b) of a plurality of log information (eg, 630). . The searched two log information 922 represents log information including 8 seconds/eID 1005 out of a total of 630 log information generated during one game. In this case, eID 1005 is an example, and may indicate an event occurring during a sports event such as strikeout.

Specifically, the log information (922-a) corresponds to eID 1005 in a situation where there are 2 pitchers who threw 6 pitchers in the beginning of the 8th, of which the first pitcher Yoon Geun-young, batter Lee Byeong-gyu, and no-out runners are not present. (922-b) has two pitchers who threw two pitchers at the beginning of the 8th inning and can respond to eID 1005 in the absence of the second pitcher Sovereignty, batter Lee Dae-ho, and one-out runner.

According to an embodiment, the highlight extraction apparatus may determine at least one frame corresponding to the searched log information. Here, the highlight extraction device may determine whether to correspond to the log information by using the game information tagged to the frame. Specifically, the highlight extraction device may determine whether to correspond through matching with the searched log information, using the game information identified using the relay board.

In this case, the highlight extraction device may determine at least one frame for generating the highlight video based on the number of target pitches in which the searched log information is generated. In this case, at least one frame for generating the highlight video may include a start frame of the target pitch number -1 to a last frame of the target pitch number.

Specifically, when log information is searched as the keyword Na Seong-beom No. 30 home run, the log information may include target pitchers and pitchers corresponding to Na Seong-beom No. 30 home run. The highlight extraction device may extract frames from the start frame of the target pitch number-1 to the last frame of the target pitch number, and may generate a highlight video corresponding to the Na Sung-beom No. 30 home run by combining the extracted frames. If the number of pitchers throwing the same target pitcher in the same inning overlaps, the number of pitchers may be determined using the pitcher's name or the number of pitchers.

For example, if the first pitcher who threw 50 pitches at the beginning of the 8th inning has a home run No. 30 to the first pitcher, the highlight extraction device may identify log information corresponding to the keyword Na Seong-beom No. 30 home run. The highlight extraction device may determine a frame corresponding to the No. 30 home run by using the target pitching number 50 and the first pitcher information included in the log information, and combine the frames to correspond to the No. 30 home run of Na Seong-beom. You can create a highlight image.

10 illustrates processing a generated highlight video according to an embodiment.

As described above, the highlight video generated based on the last frame of the target pitch number from the start frame of the target pitch number-1 may include noise. Here, the noise is a frame that does not correspond to the keyword, and the highlight extraction apparatus may remove the frame corresponding to the noise from the frame extracted using log information.

For example, if the target pitching number corresponding to the keyword Na Seong-beom 30 home run is 50 pitches, frames such as the pitcher/batter as well as the spectator/commentary/dugout/replay scene section are included between the start frame of 49 pitches and the last frame of 50 pitches. can do. Here, the noise is a scene that does not correspond to a keyword (for example, Na Seongbeom No. 30 home run), and the audience seat/replay scene section may be included as noise. Accordingly, the highlight extraction apparatus may generate a highlight video by removing a frame corresponding to noise from among the start frame of the target pitch number-1 to the last frame of the target pitch number.

For example, from the start frame 1 of the target pitcher-1 to the last frame 30 of the target pitcher corresponding to the keyword Na Seong-beom No. 30 home run, frames 3 to 5 correspond to the replay frame, and frames 13 to 17 correspond to the spectator frame. If frames 21 to 22 correspond to the commentary frames, the highlight extraction device combines frames excluding frames from 3 to 5/13 to 17/21 to 22 out of 30 frames to respond to Na Sung-beom No. 30 home run. You can create an image.

11 illustrates a highlight extraction method performed by a highlight extraction apparatus according to an exemplary embodiment.

In operation 1110, the highlight extraction device may identify a video including a sports event, log information sequentially recording an event occurring in the sports event, and a keyword related to the video.

In operation 1120, the highlight extraction device may tag the video game information related to the video. In this case, the highlight extraction device may tag the video with the game information identified by using the relay board. The relay board may be located in a preset area of the video, and the relay board may include innings/scores/ball counts/runner status/number of pitches.

In this case, when the relay board is displayed on the video, the highlight extraction device may recognize the game information displayed on the relay board and tag the game information to a corresponding frame of the video. Alternatively, when the relay board is not displayed on the video, the highlight extraction device may tag the corresponding frame of the video that there is no game information corresponding to the relay board.

The highlight extraction device may tag not only game information but also scene information to a corresponding frame of a video. Specifically, the highlight extraction apparatus may detect a scene change and tag the frame, or may recognize the scene according to a certain criterion and tag the frame.

In this case, the scene change may be detected by converting RGB to HSV and using a histogram difference of HSV. In addition, a difference in histogram between the previous frame and the current frame may be determined for each channel of the HSV, and whether to change the scene may be determined by comparing the determined histogram difference with a preset reference. In addition, scene recognition may be set as a criterion matching a corresponding frame among certain criteria set by the highlight extraction device.

In operation 1130, the highlight extraction apparatus may extract at least one log information corresponding to the keyword and determine at least one frame corresponding to the log information extracted from the tagged video.

If at least one log information corresponding to the keyword cannot be extracted from the video including the sports event, the highlight video corresponding to the keyword cannot be generated from the video including the corresponding sports event. For example, when a home run is input as a keyword, a highlight video corresponding to the home run in the baseball game cannot be generated if a home run does not occur in a baseball game.

Here, the at least one frame may include from the start frame of the target pitch number-1 corresponding to the situation included in the keyword to the last frame of the target pitch number. In this case, the highlight extraction apparatus may exclude a frame that does not correspond to the keyword from the start frame of the target pitch number-1 to the last frame of the target pitch number.

In operation 1140, the highlight extraction device may generate a highlight video by combining at least one determined frame.

According to an embodiment, when there is only one keyword, the highlight extraction apparatus may generate one highlight video corresponding to the single keyword. For example, in the case of the Na Seong-beom No. 30 home run, the highlight extraction device may generate one highlight video corresponding to the Na’s No.

According to an embodiment, when there are a plurality of keywords, the highlight extraction apparatus may generate a highlight video by combining at least one highlight video corresponding to the plurality of keywords. For example, if the keyword is Na Seong-beom No. 1 home run to No. 30 home run, the highlight extraction device is from a baseball game corresponding to Na Seong-beom No. By combining the extracted highlight video 30, a highlight video may be generated.

The embodiments described above may be implemented as a hardware component, a software component, and/or a combination of a hardware component and a software component. For example, the devices, methods, and components described in the embodiments are, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate (FPGA). array), programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications executed on the operating system. Further, the processing device may access, store, manipulate, process, and generate data in response to the execution of software. For the convenience of understanding, although it is sometimes described that one processing device is used, one of ordinary skill in the art, the processing device is a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that it may include. For example, the processing device may include a plurality of processors or one processor and one controller. In addition, other processing configurations are possible, such as a parallel processor.

The software may include a computer program, code, instructions, or a combination of one or more of these, configuring the processing unit to operate as desired or processed independently or collectively. You can command the device. Software and/or data may be interpreted by a processing device or, to provide instructions or data to a processing device, of any type of machine, component, physical device, virtual equipment, computer storage medium or device. , Or may be permanently or temporarily embodyed in a transmitted signal wave. The software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiment, or may be known and usable to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -A hardware device specially configured to store and execute program instructions such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those produced by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware device described above may be configured to operate as one or more software modules to perform the operation of the embodiment, and vice versa.

As described above, although the embodiments have been described by the limited drawings, a person of ordinary skill in the art can apply various technical modifications and variations based on the above. For example, the described techniques are performed in a different order from the described method, and/or components such as systems, structures, devices, circuits, etc. described are combined or combined in a form different from the described method, or other components Alternatively, even if substituted or substituted by an equivalent, an appropriate result can be achieved.

Claims (21)

An operation of identifying a video including a sports event, log information sequentially recording an event occurring in the sports event, and a keyword related to the video;
Tagging game information related to the video to the video;
Extracting at least one log information corresponding to the keyword and determining at least one frame corresponding to the extracted log information from the tagged video;
Generating a highlight video by combining the determined at least one frame; And
And removing a frame corresponding to noise included in the highlight video by detecting a scene change and recognizing the scene according to a predetermined criterion by applying deep learning. The method of claim 1,
The operation of generating the highlight video,
When there is one keyword, one highlight video corresponding to the single keyword is generated, or
When there are a plurality of keywords, a highlight extraction method for generating a highlight video by combining at least one highlight video corresponding to the plurality of keywords. The method of claim 1,
The operation of tagging the video,
When the relay board is displayed on the video, by recognizing the game information displayed on the relay board, the game information is tagged to the corresponding frame of the video, or
When the relay board is not displayed on the video, tagging a corresponding frame of the video indicating that the match information corresponding to the relay board does not exist. The method of claim 3,
Extracting features from the relay board displayed in a preset area in the video, and classifying the extracted features using previously learned data
Further comprising a, highlight extraction method. The method of claim 3,
The game information displayed on the relay board includes scores, ball counts, and innings. The method of claim 5,
When the game information displayed on the relay board further includes a runner situation, the runner situation is tagged to a corresponding frame of the video,
When the game information displayed on the relay board further includes the number of pitches, the number of pitches is tagged to a corresponding frame of the video. The method of claim 1,
The above scene change,
Converting RGB to HSV, and detecting using a histogram difference of the HSV, highlight extraction method. The method of claim 7,
The above scene change,
And determining a histogram difference between a previous frame and a current frame for each channel of the HSV, and comparing the determined histogram difference with a preset reference to determine the highlight extraction method. The method of claim 1,
The scene recognition,
Among certain criteria set by the highlight extraction device, a method for extracting a highlight is set as a reference to which a corresponding frame is matched. The method of claim 1,
The at least one frame,
A method for extracting highlights, including from the start frame of the target pitches-1 corresponding to the situation included in the keyword to the last frame of the target pitches. A computer-readable recording medium in which a program for executing the method of any one of claims 1 to 10 is recorded. Processor and memory containing computer-readable instructions
Including,
When the command is executed by the processor, the processor identifies a video including a sports game, log information sequentially recording an event occurring in the sports game, and a keyword related to the video,
Tagging game information related to the video to the video,
Extracting at least one log information corresponding to the keyword, determining at least one frame corresponding to the extracted log information from the tagged video,
By combining the determined at least one frame, generating a highlight video,
A highlight extraction device for removing a frame corresponding to noise included in the highlight video by detecting a scene change and recognizing a scene according to a certain criterion by applying deep learning. The method of claim 12,
The processor,
When generating the highlight video, when the keyword is one, one highlight video corresponding to a single keyword is generated, or when the keyword is plural, at least one highlight video corresponding to a plurality of keywords is combined to create a Generating, highlight extraction device. The method of claim 12,
The processor,
When tagging the video, when the relay board is displayed on the video, the game information displayed on the relay board is recognized and the game information is tagged to the corresponding frame of the video, or when the relay board is not displayed on the video, the relay A highlight extraction device for tagging a corresponding frame of the moving picture that the game information corresponding to the plate does not exist. The method of claim 14,
The processor,
A highlight extraction device for extracting features from the relay board displayed in a preset area in the video, and classifying the extracted features using previously learned data. The method of claim 14,
The game information displayed on the relay board includes scores, ball counts, and innings. The method of claim 16,
The processor,
When the game information displayed on the relay board further includes a runner situation, the runner situation is tagged to a corresponding frame of the video, or when the game information displayed on the relay board further includes the number of pitches, the number of pitches is indicated. A highlight extraction device for tagging the corresponding frame of the video. The method of claim 12,
The processor,
A highlight extraction device for converting RGB to HSV and detecting the scene change using a histogram difference of the HSV. The method of claim 18,
The processor,
A highlight extraction apparatus for determining whether to change the scene by determining a histogram difference between a previous frame and a current frame for each channel of the HSV, and comparing the determined histogram difference with a preset reference. The method of claim 12,
The scene recognition,
A highlight extraction device that is set as a reference to which a corresponding frame is matched from among certain criteria set by the highlight extraction device. The method of claim 12,
The at least one frame,
A highlight extraction device comprising from the start frame of the target pitch number-1 corresponding to the situation included in the keyword to the last frame of the target pitch number.

Images