KR20020078449A - An Appratus and Method for Automatic Soccer Video Analysis - Google Patents

Abstract

PURPOSE: A system and a method for automatically analyzing soccer video are provided to propose a method of analyzing soccer games using characteristics of soccer and automatically analyze the contents of the games to provide information about the analyzed result. CONSTITUTION: A system for automatically analyzing soccer video includes an input unit(21) for receiving analog video data obtained by recording a soccer game broadcasted through a TV, a digital converter(22) for converting the analog video data into digital data, and a scene change detector(23) for filtering scenes that are not required for analyzing the game or scenes to which a camera operation analysis technique is not applied. The system further includes a game information storage(26) for storing images to be processed that were filtered by the scene change detector, basic analysis information and information about players and ball traces in a stadium model, and an output device(25) for outputting images including game analysis information.

Description

An Appratus and Method for Automatic Soccer Video Analysis

The present invention relates to an apparatus and method for automatically analyzing a football game using characteristics of a football. After extracting a field, a goal, and a stadium line from a game video using a digital image processing method, a camera motion is predicted and the football is extracted. It is a device that can analyze the game by tracking the movement of players and the ball by calculating the mapping coefficient with the 2D stadium model made according to the stadium specification information.

In particular, it is possible to analyze the progress of the game by extracting the soccer game relaying video image, extracting the scene related to the goal by extracting the goal, and tracking the movement of the players and the ball in the stadium.

In recent years, as interest in soccer increases, people are not satisfied with watching a soccer game, but want to obtain a lot of more detailed analysis. In this game, we need to provide the viewers with the information they want to know, such as how the attack and defense are done, how well the players have done their positions, and how the ball was passed and scored by the goal. Game analysis is attempted.

Looking at the existing patents related to the present invention, as a domestic patent application number 10-2000-002652 (called patent 1), application number 10-2000-0048301 (called patent 2), application number 10-1996-0026877 (Patent 3), application number 10-1997-0023473 (patent 4), application number 10-1998-0019130 (patent 5), and the like, and US patent 6,071002 (patent 6). And the like).

In general, the analysis of the football game was to provide only the statistical values such as the number of goals, the number of shots, the number of corners, the penalty kick, and the number of free kicks. Furthermore, there is a more advanced method (patent 1, 3, 6) to manually track the position of the soccer ball, which indicates the absolute position in the stadium model by the analyst's subjective judgment. The accuracy is also unreliable and there are other drawbacks, such as penalty kicks and free kicks.

In addition, the related patent 4 detects the acting motion of a gymnastics game using a difference image, and the patent 5 separates the foreground and the background from golf and tracks the swing motion and the ball, even if the image processing technique is used. Presented to provide a suitable analysis method that does not fit the football match, Patent 2 is to analyze the comprehensive sports video image using the characteristics of the football match effective shooting number, free kick number, penalty kick number, off-line determination, etc. It is not able to analyze the specialized football event of, and has the disadvantage of installing several expensive cameras in the stadium.

Unlike the existing systems, the present invention proposes a method of analyzing a game utilizing the characteristics of the football, grasps the progress of the game, and automatically analyzes the content of the game and provides the result information accordingly, thereby analyzing a new game. It is characterized by deriving information.

In this analysis process, the digital image processing technique aims at an automatic image analysis method that minimizes manual work.

It detects only the scenes related to the flow of the football game by receiving the broadcasted video image, extracts the characteristic image information such as the goal post, the field, and the stadium line, and enables the game to be analyzed. 3D soccer game by tracking the motion of the players and the ball on the 2D stadium model by obtaining the arena model of the 2D coordinate model and obtaining the mapped coefficients by matching the feature points in the original image with the corresponding points on the model. It is a device that enables analysis, which is possible because the movement of players and balls in the stadium is the flow of the football game.

1 is a block diagram of a device used to practice the present invention.

2 is a block diagram of a soccer game analysis system according to the present invention.

3 is a flowchart illustrating a process of the scene change detection unit of the present invention.

4 is a diagram schematically showing a game analysis processing unit at 2 degrees.

5 is a diagram schematically showing a stadium information extractor at 4 degrees.

FIG. 5a schematically illustrates the goal extraction section process at 5 degrees. FIG.

6 is a view schematically showing a process of the camera motion analysis unit of the present invention.

7 is a view schematically showing the process of the stadium model matching unit of the present invention.

8 is a flow chart illustrating a player tracker process of the present invention.

8a is a diagram showing the overlapping process of the other team players in the player tracker process in 8 degrees.

8b is a diagram showing the overlapping process of the same team player in the player tracker at 8 degrees.

9 is a flow chart illustrating a ball tracking process of the present invention.

10 is a diagram illustrating a user interface of a soccer game analysis system according to the present invention.

According to the present invention, a personal computer equipped with an input device 11 such as video broadcasted on a TV, a conversion device 12 for digitizing the image thereof, and an image processing board 13-b for improving the processing speed ( 13) is a system composed of an output device 14 such as a monitor (FIG. 1).

In addition, a video input unit 21 for inputting a soccer relay broadcast video, a digital image converter 22 for making digital image data capable of game analysis, and camera motion analysis technology can be applied to determine whether the game can be analyzed. Scene change detection unit 23 to be played, the game information storage unit 26 for storing the image to be processed and the analysis information, the game analysis processing unit 24 for actually calculating the game analysis information and the analysis information output through the monitor or printer It consists of the output part 25 (FIG. 2).

Herein, the present invention is characterized by using a TV football game relay broadcast as a data without a separate camera. As such, the progress of the game is analyzed depending on the broadcast data, so preprocessing is required. When watching a regular football broadcast, suddenly the camera moves quickly to close up the screen, or to illuminate a candidate's seat with a cheering audience or coach. The data in these cases are actually far from the data used for game analysis.

Therefore, in order to accomplish this, the present invention filters only scenes requiring analysis processing through the process of the scene change detection unit 23. Looking at the process in detail (Fig. 3), the pixel value of the video data is obtained (31) and it is determined whether the frame is started. In operation 32, the pixel value Int _n (i) of the current frame i is obtained. Then, the following equation is used to obtain an average (Diff) of the absolute value of the difference between pixel values of the adjacent frame difference images from i to j frame (33). If the value is greater than or equal to the threshold value (34), it is detected as a scene change detection position. The scene is output as the representative scene (35).

And until the last frame j of the video (36) is applied to this technique, it is to determine the availability of the position prediction method by analyzing the camera motion.

In this way, when these scenes determined as scenes capable of game analysis processing through the scene change detection unit are input, game analysis is performed through the game analysis processing unit. Using the characteristic basic information for game analysis obtained through the stadium information extractor 41, the player / ball tracker 42 processes the player and the ball trajectory, and the camera motion analyzer 44 provides the camera position information. It is possible to track the feature points so that this information is provided to the stadium model matching unit 43 to obtain a model mapping coefficient and consequently to predict the position of the player and the ball in the two-dimensional stadium model.

Looking at the configuration of the stadium information extraction unit in detail (Fig. 5), it is composed of a field extraction unit 51, a goal extraction unit 52, the stadium line extraction unit 53.

Looking at the video of a football game, it can be assumed that the field takes up most of the video and that the players and balls in the field are an important part of the game flow. Thus, the present invention is characterized by obtaining a histogram of an RGB channel of an image for extracting a field, and dividing a portion having values within a predetermined range from the most frequently appeared RGB values into fields (51). When the field information obtained here is used to detect the position of the player or the ball and the stadium line, the information of the part outside the field can be derived so that they can be easily detected.

The formula for extracting a field is:

Based on the value that occupies the largest portion of the RGB image, the field within the threshold range is divided into fields to make a binary image (F).

And it is characterized in that it comprises a goal extraction unit 52 to extract a goal that is closely related to the score in a soccer game. X-axis projection of the color of the goal in the game image. Then, the portion corresponding to the goal portion will have two maximum points ((a) and (b) in FIG. 5a), and the distance (D) between the heights (H (a) and H (b)) of these two maximum points By using the following equation to obtain the height (H) and the distance (D) of the post pillar, it is possible to determine whether the image containing the goal post.

In this way, the height H of the goal post is higher than the reference value T _H , and the distance D between the two posts is within the reference value (between T _Dmax and T _Dmin ). It is possible to determine whether the shooting of the player is a valid motion for scoring, and further, the effective shooting number of the team can be measured (FIG. 5A).

Finally, the stadium information extracting unit 41 may include a stadium line extracting unit 53 as a feature point extracting operation for matching a match video to a stadium model. Separating the color of the stadium line to extract the part corresponding to the line, it is possible to distinguish whether the player is a corner kick or a free kick scene. In this case, if the players are on the line, the broken line may be extracted. In order to solve this problem, in the present invention, the slopes of the portions corresponding to the extracted lines are obtained, and the average of the lines having similar slopes is found to find the stadium line.

The feature point is obtained from the intersection of the found stadium lines. The camera motion analyzer 44 estimates the camera motion and processes the information by analyzing the image of the continuous frame including the extracted stadium lines. Since the device is an image obtained through a camera having various movements such as pan / tilt / zoom rather than a fixed camera, it is difficult to calculate the absolute moving distance between the player and the ball without predicting the movement of the camera. Referring to FIG. 6 in detail, although there are actually many distance changes, it is difficult to determine a change in the position of two players on the screens (a) and (b) due to the movement of the camera. So the device predicts the camera's motion based on the points on the stadium line. Before the camera moves, we can see that the players outside point A, which is one point on the playing field line, moved to point A after the camera move (B). Can be analyzed and calculated. By using this technology, it is possible to analyze the game considering the camera's motion, and it is possible to match the stadium and the stadium model in the image and to predict the position in the stadium model by the position of the players and the ball in the image. Can be generated.

And the game analysis processing unit 24 of the present invention is characterized in that it comprises a stadium model matching unit 43 in order to express the movement of the player and the ball in the three-dimensional game image in the absolute position in the two-dimensional stadium model. In addition, in the stadium model matching unit of the present invention, since the feature point is obtained by the intersection point of the stadium line extracted by the stadium line extracting unit 53, the coordinate value can be obtained even for the feature point (Fig. 7 (e)) not visible on the screen. There is this.

To obtain the trajectory information on the stadium model, first, a stadium model is created and stored in accordance with the size ratio of the soccer field standardized in the game analysis storage unit 26. If the line intersection feature is found to be at least 4 points, the relationship between the feature point (x _i , y _i ) and the feature point (u _i , v _i ) corresponding to the soccer stadium model is determined by the following equation using the projection displacement field model. You can get it. In the present invention, if the analyst extracts four or more feature points to match the stadium model and selects a position in the corresponding stadium model (Fig. 7), the conversion coefficients a, b, c, d, e, f, g, h) can be obtained.

In the field extraction image, the part corresponding to the field and the background are separated, and the player and the ball are tracked only in the part corresponding to the field. The player tracking unit 42 distinguishes the positions of the players by selecting uniform colors of the upper and lower uniforms of the players of each team. At this time, the color model used is an RGB model, which does not need color conversion, and thus has a high processing speed and a game analyst can easily adjust colors.

However, the uniform color area thus obtained cannot extract the player, and when the color on the uniform is located above the lower color in consideration of the positional relationship of the color area, it is defined as the position of the player. The midpoint is called the bow position (81).

And the process of tracking the extracted player position (Fig. 8) is that of one athlete position (X _n (i), Y _n (i)) extracted in the current frame (n) and all athletes (j) extracted in the next frame The position X _{n + 1} (j), Y _{n + 1} (j) is enabled by connecting the position at which the Euclidean distance _Pi (n) is minimum (82).

If the minimum exceeds the threshold, the athlete is assumed to have disappeared (83). Soccer players, unlike other sports, often have overlapping players with frequent and fast movements. The present invention is characterized by resolving the player overlap by dividing the case between the team players (Fig. 8b), such as the case where the other team players overlap (Fig. 8a).

If two or more athlete positions are stored in one athlete trajectory information (85), it is determined that the athletes overlap and the separated athlete area of the frame before the view is examined. In the case of overlapping players of different teams, if they overlap completely, the position of the overlapping players is continuously stored in the trajectory information. Even if the midpoint position obtained by grouping does not appear, if a part of the color area of the top and bottom of the athlete's uniform comes out, the midpoint of the position is determined as the position of the athlete and the overlap is solved (86).

If the players of the same team overlap each other, it is difficult to distinguish them by the uniform color. At this time, the relationship between the midpoint positions of the upper and lower regions is examined. If it is completely overlapped as in N frame of Fig. 8B, it is stored in the track information of the folded player, and if the part of the upper and lower uniforms is visible in the N + 1th frame, the midpoint of the uniform of the athlete's upper jacket located at the front of the overlapping athlete's area (s1) Judgment is made based on the relationship between the midpoint of the bottom and bottom (p1), the midpoint of the overlapping athlete's top (s2) and the bottom midpoint (p2). According to the positional relationship, s1, which is the color gamut of the upper part, is also present on the lower color gamut p2, but by s1 and p1, s2 and p2 are predicted by estimating the distance relation between s1 and p1, s2 and p2 by grasping the relationship between the middle point of the other athlete's top and the bottom middle. Recognize and p2 as one bow position and separate. The newly appearing players on both sides of the screen add the track information to the new player for the player area (87).

Finally, in the ball tracking unit 42, because the speed of the ball is fast and small, the analyst first directly specifies the position (91). The color of the ball and its size are then obtained (92). Distance (B (n) between the midpoint coordinates (X _n-1 , Y _n-1 ) of the portion determined to be the ball in the previous frame (n-1) and the midpoint coordinates (Xn, Yn) of the portion corresponding to the ball in the current frame. If)) is included within the threshold considering the ball's velocity and camera motion analysis (93, 94), add its position to the ball trajectory (95).

However, there may be cases where the player is obscured or disappeared by players because of the small size and high speed of the ball (96). In this case, the user uses a method of repositioning the ball (Fig. 9).

By simply graphicizing the actual range of movement in 3D with the trajectory of the player and the ball in the 2D stadium model obtained from the game analysis department, the attack, defense pattern and the tactics of the football can be predicted. You can see if the player's motion dribble or shoot.

Looking at the process of analysis using the user interface for game analysis of the present invention based on FIG. 10 in detail, first, the analyst has already obtained a digitized video image, the scene that can apply the camera motion analysis technology through the scene change detection The bay is extracted. If a scene that is valid for game analysis is selected from the scene-changed representative images 102 obtained through the scene change process, a part of the game video to be processed is brought to the source window 101 to be analyzed.

The analyst then selects the top and bottom colors of both team uniforms to extract the player position (104a). The two selected colors are grouped by the color of the team. Through this process, the position of each player can be extracted by RGB color value, and the team of the player can be identified. And it is possible to preview (104b) the extracted players and their teams, so that the analyst can adjust the color. This enables tracking (104c) of the player's position in the three-dimensional image. When the player overlaps, the player is automatically analyzed and separated, and when a new player appears outside the screen, the new player appears (104d). do.

In the case of a ball, the analyst must choose the location of the ball (105a). Then, the color information around it obtains area information such as the size and shape of the ball. This allows ball tracking 105b in the image. If the ball is obscured by the player, disappears from the playing field, or the ball cannot be tracked due to too fast speed, the position is corrected again (105c).

The player and ball tracking information thus obtained is three-dimensional coordinates. Thus, the color of the stadium line is set to obtain the trajectory information on the stadium model 106 made in the soccer stadium standard ratio in advance. (103) The present invention automatically extracts the goal post, field, and stadium line to match the game analysis. You get the information. At least four feature points are automatically extracted and displayed on the stadium model. Now, if the analyst sets 107 the feature points to be used among the extracted feature points in order, the present invention automatically obtains the mapping coefficients to the stadium model to have the track information of the player and the ball on the two-dimensional plane. In this way, game analysis information can be obtained and various game analysis is possible.

Unlike existing systems, the present invention provides a soccer game video relayed on a TV without a separate camera to provide scientific game analysis information to attract viewers. It is a device that makes it possible to combine the additional information obtained by converting the trajectory of the ball and the player into practically accurate analysis of the game.

With the information of the stadium, you can grasp the progress of the game such as corner shots, shot shots, free kick shots, etc. The extraction of the goal allows the player's movements to be predicted by the trajectory of the players. It can be applied for measurement, and can provide information such as strategy information through direction of movement and player arrangement, relationship between ball and player, and pass success rate by team classification, and offside decision by final defender detection, and match rules By providing game rules such as free kick distances and penalty circles to unknown people, it is possible to induce interest and enjoy soccer games, and in the case of providing analysis information in real time, various new information such as game prediction information can be induced.

And the visual effect is great by expressing the movement of players and the ball in the stadium model to produce various game information.

Claims (7)

An input device for inputting a video image of a game relayed on a TV; A digital converter for converting the analog video data into digital; A scene change detector for filtering out scenes that are unnecessary for game analysis or to which camera motion analysis techniques cannot be applied; A game information storage unit storing basic analysis information and track information of players and balls in the stadium model derived from the game analysis processor through the image to be filtered and the game analysis processor through the scene change detection unit; Football video automatic analysis device, characterized in that it comprises an output device for outputting it as an image including the game analysis information. A stadium information extracting unit for obtaining basic information for analyzing a football game; A camera motion analysis unit which analyzes the camera motion of pan / tilt / zoom with only the image without using a separate camera by using stadium information extraction information to grasp the location information of the camera; Soccer video automatic analysis device comprising a substantial game analysis processing unit including a stadium model matching unit for calculating a mapping coefficient to the stadium model. A game information storage unit which stores information derived from the game analysis processing unit such as basic analysis information and track information of players and balls in the stadium model through the video to be filtered and the game analysis processing unit filtered through the scene change detection unit; Football video automatic analysis device comprising an output device for outputting the image including the game analysis information. A player region extraction unit for recognizing the positions of the players and classifying the teams using uniform color grouping of the players and player area information; A player overlap solution for separating players when several players overlap each other; A player tracking unit for extracting stadium information and including player position information on the 3D in the player trajectory information on the 2D stadium model; Soccer ball automatic analysis device, characterized in that it comprises a ball tracking unit for inducing the process of extracting the ball through the ball color information, size information, motion information and ball trajectory information. A goal extraction unit for classifying a player's motion related to scoring; A field extraction unit for extracting a field which is a region of interest of the analysis; Soccer video automatic analysis device, characterized in that for analyzing the event having the characteristics of football, including the stadium information extraction unit consisting of a line extraction unit for extracting the stadium line required for model matching. Comparing the performance of the two teams by comparing the number of free kicks, penalty kicks, effective shooting numbers, offside numbers, etc. using the information obtained through the game analysis processing unit; Measuring the performance of an individual player to calculate his / her maximum shooting speed, distance played during game time, and fitness expenditure; Predicting strategic information such as team placement, attack and defense patterns of all players; Football video automatic analysis device that includes game rule information such as free kick distance and penalty circle. Detects the main scenes of the progress of the game, such as corners or free kicks, by determining the cornerstones and free kicks, and determines whether the player's motion is a valid shot by scoring goals. Effective shooting number measuring method; A pass success rate measurement method for determining whether the pass is successful to the same team player when the ball is passed between the player and the team; A method of comparing the performances of both teams, including a method of determining offside lines by measuring the final defender and measuring offside numbers; A method of measuring the fitness expenditure of players inferred by the total distance run during the course of the game by the trajectory of each player; A method of measuring a player's individual performance, including a method of measuring the maximum speed of a shot obtained by the player's movement and the actual moving distance of the ball; Soccer video automatic analysis device and method, including the method of predicting the game strategy obtained by the movement direction and player placement of players for each team.