KR20030064928A - Image processing system and database building method for the object information of soccer game - Google Patents

Abstract

PURPOSE: A soccer game object information video processing system and a database constructing method are provided to extract coordinate values of positions of players and a ball and construct a database of the coordinate values and video data. CONSTITUTION: A soccer game object information video processing system includes a video camera(100), a video processor(200), a monitor(300), and an input unit(400). The video camera takes a picture of a soccer game to generate video frames. The video processor extracts position coordinate values of objects appearing in the video frames and construct a database of the coordinate values and the video frames. The monitor outputs the position coordinate values and video frames. The input unit transmits a control command from a user to the video processor.

Description

IMAGE PROCESSING SYSTEM AND DATABASE BUILDING METHOD FOR THE OBJECT INFORMATION OF SOCCER GAME}

The present invention relates to an object information image processing system and a database construction method of a soccer game, and more particularly, to extract the position coordinate values of players and balls belonging to an object from an image frame of a soccer game photographed through a video camera. The present invention relates to an object information image processing system and a database construction method of a soccer game that can be used for objective analysis of a soccer game by making a database together with image data.

The world's most successful ball game, for example, is probably baseball, basketball, soccer, and football. In addition, these sports events have a common point that they have greatly developed through the professionalization. In particular, in the case of football, not only the development of soccer itself has been achieved by operating a club system or a professional team in each country, but it is also a representative sport that has succeeded in commercialization through the world championship called World Cup.

Determining which country the World Cup takes place in, like determining the Olympic venue, brings great economic benefits to that country. It can be largely divided into advertising revenue through broadcast broadcasts of the game, and tourism revenue from many foreigners visiting the venue to watch the World Cup.

Compared to the Western countries that enjoyed the fruits of commercialization for a long time, other countries that had adopted football lately had to lag behind the development of football. On the other hand, however, there has never been a scientific and unified way to quantitatively record or qualitatively analyze the content of football with a long history.

When recording the contents of a football game with a video camera, there are three main methods of disposing the camera according to the environment around the stadium. In each case, problems that must be solved technically appear differently.

1A is an exemplary view when the angle of the video camera toward the playground according to the prior art is Ø = 0, FIG. 1B is an exemplary view when the angle of the video camera toward the playground according to the prior art is 0 <Ø <90, 1C is an exemplary view when the angle of the image camera toward the playground according to the related art is Ø = 90.

According to this, when the angle of the video camera toward the stadium is Ø = 0, the outdoor training ground or the stadium 20 is not equipped with a spectator seat, and the angle that the video camera 10a can take is parallel to the ground of the playground. In some cases, when the angle toward the playground of the video camera 10a becomes close to horizontal, the appearance of the players 30 overlaps each other, and the distance between the video camera 10a and the players 30 cannot be distinguished. Problems will arise. When the object moves away from the video camera 10a, the size thereof becomes smaller on the screen of the video camera 10a. Therefore, when trying to determine the position only by the size of the height of each player, it is very difficult to distinguish whether or not the player looks small by moving away from the video camera 10a, or because of the short distance from the video camera 10a. Lose. Therefore, even in this case, there is a need for a method that can calculate the position coordinates of each athlete.

When the angle of the video camera toward the stadium becomes 0 <Ø <90, when the audience seat 40 is installed and the video camera 10b is raised to some extent from the floor of the playground, the view of the stadium is square. As a result, when the angle at which the image camera 10b looks at the playground becomes square, the appearance of players moving in the stadium overlaps with each other. In other words, when viewed from the video camera 10b, a part or all of the figure of the player located at the rear starts to be hidden by the player located at the front. Therefore, when the angle toward the playground of the image camera 10b becomes a square, there is a need for a method capable of calculating each position coordinate value by distinguishing athletes overlapping each other.

If the angle of the video camera toward the stadium is Ø = 90, it means that you have a ceiling as an indoor playground. In this case, the video camera 10c installed on the ceiling faces the playground surface in front, and the movements of all the athletes overlap with each other because the video camera 10c becomes an angle of looking at the playground surface vertically. Rarely occurs. In addition, it is quite easy to calculate the position coordinate value P (x, y, t) of each athlete because the shape of the playground is displayed as a square shape. In other words, obtaining the position coordinates of the athletes is equivalent to determining the position of a point on the two-dimensional rectangular coordinate system.

In addition, another problem that needs to be considered in extracting the position coordinate values of players and balls, or objects, in a soccer game is the accuracy of the position change, that is, the resolution of the position coordinate values. The resolution of the position coordinates has a great influence on analyzing the game contents. For example, the resolution of the athletes' position coordinates is significantly different from that of 3 meters and 30 centimeters. If a player can distinguish 30 centimeters of movement from a given position, he will be able to make a very detailed and reliable analysis of defenders' movements and attack breakthroughs in very small areas. However, if the resolution of the position coordinate value is 3 meters, it is impossible to represent this situation in a small area. Therefore, it is important to know how much resolution can be increased for the position coordinate values of each object.

2 is a plan view showing a state of a stadium appearing on the screen of the video camera installed on the ceiling according to the prior art.

As shown, if the width of the stadium 20 is referred to as L _Y and the length L _X is set to approximately 75 meters and 110 meters in the case of the playground of the international standard. The screen of an image camera is made up of dots called pixels, and the size of the screen is given by a two-dimensional grid of these pixels, commonly referred to as an M × N matrix. M indicates the number of pixels in the horizontal direction, and N indicates the number of pixels in the vertical direction.

Assuming that the size of the screen given in FIG. 2 is 640 × 480 and that the magnifications are the same in the horizontal and vertical directions of the screen, the size of the screen occupied by a stadium having a width of 75 meters and a length of 110 meters is about 640 ×. Given by 436. This means that the distance between adjacent pixels in the horizontal and vertical directions of the screen is about 17.2 centimeters each. When the number of pixels constituting the image camera screen is reduced to 320 × 240, the distance between adjacent pixels is increased to about 34.4 centimeters. That is, as the number of pixels increases, the motion of the object can be known in detail, and the resolution of the position coordinate value can be increased.

However, when the image camera facing the playground forms a square, the resolution of the position coordinate value is affected.

Figure 3a is a view showing that the appearance of the playground viewed from the blind spot according to the prior art is caught on the screen of the video camera.

As shown, when a person is standing in the center of the playground, the farther away from the image camera 10b, the smaller the size of the screen is taken and the length of the playground is reduced in proportion to the distance. This becomes more severe as the value of the angle formed between the image camera 10b and the playground surface becomes smaller. For example, between the number of pixels that make up the size of the screen 640 × 480 and the playground side length (L _Y ') located on the far side in the video camera (10b) adjacent if called 320, and, playground length of 110 m pixel The distance corresponds to 34.4 centimeters. In other words, the length per pixel of the near-field side 22 and the far-side side 21 in the image camera 10b is different from each other. This result also appears in the direction of the playground width. As the value of Ø decreases, the width of the playground displayed on the screen decreases significantly. For example, if the number of pixels representing the width L _X 'of the playground displayed on the screen is 109, the distance between adjacent pixels is given as 68.8 centimeters.

When the video cameras are on the ceiling of the stadium, they have the same resolution in the width and length of the playground. However, when the video cameras form a square with the surface of the stadium, they have different resolutions in the width and length of the playground.

When the angle between the image camera and the surface of the playground is square, when the resolution between adjacent pixels is severely different, it is possible to compensate for this by increasing the magnification of the screen.

If the magnification of the playground screen given in Fig. 3a is enlarged twice, the resolution can be increased twice in the width and length directions of the playground. However, the use of this method causes a problem that the sideline, which indirectly determines the size of the playground, disappears from the screen.

FIG. 3B is an enlarged view of the magnification of the video camera of FIG. 3A.

As shown, when the magnification of the image camera 10b is enlarged, resolution may be increased in the width and length directions of the playground, but an area 23 disappears from the screen. When the sideline indicating the length of the playground indirectly goes out of the screen, it becomes difficult to determine the resolution per pixel in the length direction. In addition, if the magnification is further increased, even the sideline 24, which can know the width of the playground, goes out of the screen, and there is no way of knowing the resolution in the width direction of the playground.

In conclusion, if the video camera is square with the playground, the side line can indirectly determine the size of the playground by adjusting the camera's magnification in the direction of increasing the resolution, that is, the distance resolution between pixels that can detect the movement of the object. The problem of getting out of this screen will occur. Therefore, even in such a case, a means for determining the distance resolution between pixels is needed.

Finally, when the image camera is level with the surface of the playground, that is, when Ø = 0, the width of the playground occupying the screen becomes almost straight, making it almost impossible to determine the resolution in the width direction of the playground. In this case, increasing the magnification of the video camera also has no effect. In this state, if the magnification is increased, the players appearing on the screen will be off the screen before the resolution is increased in the width direction of the playground.

The following table summarizes the problems that can occur according to the angle between the surface of the playground and the image camera. Table 1 summarizes the problems according to the angle between the surface of the video camera and the playground.

Angle, Ø problem Ø = 90 It cannot be used outside the stadium. 0 <Ø <90 The resolution, that is, the resolution, is different between pixels in the horizontal and vertical directions of the video camera screen. If you adjust the magnification of the video camera, the sideline, which indirectly determines the size of the playing field, is taken out of the screen. Players sometimes overlap in appearance. Ø = 0 The resolution cannot be determined in the vertical direction of the video camera screen. There are relatively many cases where athletes overlap each other.

In addition, Korean Unexamined Patent Publication No. 2000-0064088 (2000.11.06) discloses a sports video analysis broadcasting system and method. According to this, there is no specific method for how to solve the overlapping problem of the athlete, which occurs according to the angle that the video camera makes with the surface of the playground. In addition, the trajectory according to the movement of the athletes is obtained through the difference between two adjacent temporal images. However, the description of the method has not been made in detail, and the position of each athlete cannot be determined with the still image. That is, the first image is given as a still image, and it is not specified how to determine the initial position of the athletes.

Accordingly, the present invention is to solve the above-mentioned general problems, the purpose of which is to extract the position coordinate values of the players and the ball belonging to the object in the image frame of the soccer game photographed through the image camera with the image data It is to provide an object information image processing system and a database construction method of a football game to be used in the objective analysis of the football game by the database.

Figure 1a is an exemplary view when the angle of the image camera toward the playground according to the prior art is Ø = 0.

Figure 1b is an exemplary view when the angle of the image camera toward the playground according to the prior art is 0 <Ø <90.

Figure 1c is an exemplary view when the angle of the image camera toward the playground according to the prior art is Ø = 90.

Figure 2 is a plan view showing a state of the stadium appearing on the screen of the video camera installed on the ceiling according to the prior art.

Figure 3a is a view showing that the appearance of the playground viewed from the blind spot according to the prior art is caught on the screen of the video camera.

FIG. 3B is an enlarged view of the magnification of the video camera in FIG. 3A; FIG.

4 is a block diagram showing the configuration of an image processing system according to the present invention;

5 is an exemplary view showing a time point of generating image frames simultaneously on the time axis when the number of image frames generated per second is different according to the present invention.

6 is a flowchart illustrating an image processing process according to the present invention.

7 is an exemplary view showing an example of a configuration method of a control panel which is a user interface according to the present invention.

8 is an exemplary diagram illustrating screen division of a monitor used in conjunction with FIG. 7;

9 is a flowchart for editing a video made from two video cameras through a user interface according to the present invention.

10 is an example of use of defining and using a data structure for distinguishing each group of frames according to the present invention.

11A and 11B are flowcharts for calculating position coordinate values of each object of image data belonging to all edited frame groups according to the present invention.

12 is an exemplary view showing a relationship between a playground coordinate system and a screen coordinate system when the image camera according to the present invention forms a square with the surface of the playground.

13 is an exemplary view showing an example of a configuration method of a resolution setting control panel which is a user interface according to the present invention.

14 is an exemplary view showing a data structure in which a result calculated through a resolution setting process and parameters used are stored according to the present invention.

15 is an exemplary view showing a projection of the screen coordinate system according to the present invention to the playground coordinate system.

16 shows an example of a recognized person and estimated position coordinates on a screen coordinate system.

Figure 17a is an exemplary view showing an example of a configuration method of a pattern recognition processor control panel which is a user interface according to the present invention.

17B and 17C are exemplary views illustrating changes in connecting individuals in FIG. 17A.

18 is a diagram illustrating a camera handoff process for automatically recognizing an object moving to an adjacent photographing area according to the present invention.

19 is an exemplary view showing a form of a data structure in which a result of a mode frame group in which a position coordinate value is determined according to the present invention is stored.

20 is a view showing a change in the trajectory that appears when a person jumps in a standing position according to the present invention.

21 is a diagram illustrating a case where crossover between two entities occurs according to the present invention.

FIG. 22 is a diagram illustrating a smooth change in a zigzag trajectory by a curve fitting method. FIG.

FIG. 23 illustrates an example of a user interface for handling errors in FIGS. 20-22.

FIG. 24 illustrates an embodiment of a user interface for analyzing content using information recorded in a database. FIG.

<Description of main parts of drawing>

100: video camera 200: image processing device

201: Digital data input device 202: A / D data converter

203,204: Dual frame buffer 205: Mass data storage device

206: graphic signal generator 207: overlay buffer

208 synchronous pulse generator 209 system clock generator

210: Motion parameter storage 211: Video frame editor

212: Pattern Recognition Processor 213: MPEG Encoder

214: MPEG Decoder 215: User Interface

216 controller 300 monitor

400 input means

Features of the object information image processing system of the football game according to the present invention for achieving the above technical problem,

A video camera for shooting an image of a game to generate an image frame; An image processing apparatus for extracting position coordinate values of objects appearing in the image frame and making a database together with the image frame; A monitor configured to output the position coordinate value and the image frame; And input means for transmitting a user's control command to the image processing apparatus.

The image processing apparatus may include: a synchronization pulse generator configured to give a synchronization signal to the image camera and designate a time at which the image frame is generated; A mass data storage device for storing the image frame; An image frame editor for editing only a part of the image frame in which a game is actually played; A pattern recognition processor configured to sequentially extract position coordinate values of objects from the edited image frame; An MPEG encoder for sequentially compressing the edited image frame and storing the edited image frame in the mass data storage device; An MPEG decoder for restoring the compressed video frame; A graphic signal generator for generating a graphic signal of the restored image frame; A user interface for receiving a control command from the input means; And a controller for controlling the control command and transferring the image frame editor, the pattern recognition processor, and the MPEG encoder and the MPEG decoder.

And a motion parameter memory for temporarily storing parameters generated during compression and decompression of the edited image frame.

In addition, the feature of the object information image processing database construction method of the soccer game according to the present invention,

Storing the edited video frame only by the video frame editor, the portion of which the game is actually played in the mass data storage; Reading the edited image frame from the mass data storage device and setting a resolution; Determining position coordinates of objects in the edited image frame by sequentially extracting and determining a pattern recognition processor; Determining whether an error occurs in the position coordinate values of the objects; If an error has not occurred, determining whether the edited video frame is the last frame; If the edited video frame is the last frame, reading the edited video frame in which no error occurs from the mass data storage device; The edited video frame is compressed by an MPEG encoder and stored in the mass data storage device; Determining whether the edited video frame compressed and stored in the mass data storage device is the last frame; And when the image frame to be compressed and stored is the last frame, storing the position coordinate values of the objects and the compressed image frame in the form of a database in the mass data storage device.

If an error occurs in the step of determining whether the error of the position coordinate values of the objects occurs, comprising the step of correcting through a user interface.

The step of extracting only the part of the game actually played by the video frame editor and storing the edited video frame in the mass data storage device may include: retrieving and setting a video frame corresponding to the start point of the game by playing a reference camera image; Searching and setting an image frame corresponding to an end point of the played game; Displaying a plurality of consecutive front and rear image frames on the left side of the monitor with respect to the image frame corresponding to the starting point; Searching for an image of a camera to be synchronized and setting an image frame close to an image frame corresponding to a start point of the reference camera; Displaying a plurality of consecutive front and rear image frames on the right side of the monitor centering on the image frame adjacent to the image frame corresponding to the start point of the reference camera; Checking whether an image frame indicating a moment closest to the image frame corresponding to the starting point of the reference camera is selected; Determining whether the image frame represents a moment closest to the image frame corresponding to the starting point of the reference camera; And in case of an image frame indicating the closest moment, selecting a synchronization setting for the image of the camera to be synchronized.

In the step of determining whether the image frame indicates the moment closest to the image frame corresponding to the start point of the reference camera, if the image frame indicates the moment closest to the image, the image of the camera to be synchronized is moved back and forth. Moving to search for an image frame representing the closest moment.

In the step of setting the resolution by reading the edited image frame from the mass data storage device, the resolution is set by extracting the contour of the playing field and setting the baseline height to calculate the resolution.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

4 is a block diagram illustrating a configuration of an image processing system according to an exemplary embodiment of the present invention, and FIG. 5 is an exemplary diagram illustrating simultaneous generation of image frames on a time axis when the number of image frames generated per second is different.

As shown in FIG. 4, the configuration of the present invention is largely composed of an image camera 100, an image processing apparatus 200, a monitor 300, and an input means 400.

The video signal generated by the video camera 100 is made in the form of data of an analog video signal or a digital video signal.

The image processing apparatus 200 includes a digital data input device 201, an A / D data converter 202, dual frame buffers 203 and 204, a large data storage device 205, and a graphic signal generator 206. ), Overlay buffer 207, sync pulse generator 208, system clock generator 209, motion parameter storage 210, video frame editor 211, pattern recognition processor 212, And an MPEG encoder 213, an MPEG decoder 214, a user interface 215, and a controller 216.

When an analog video signal generated from the video camera 100 is input to the image processing apparatus 200, the analog video signal is converted into a digital video signal through the A / D data converter 202 to the dual frame buffer 204. Is stored and then transferred back to the mass data storage 205.

That is, while image data is stored in one frame buffer of the dual frame buffer 204, the image data in the other frame buffer is transferred to the mass data storage device 205.

When image data is completely stored in the dual frame buffer 204, the next image data enters the other frame buffer. As such, the control signal for selecting the dual frame buffer is generated in hardware by the A / D data converter 202.

Similarly, the digital video signal input from the video camera 100 is stored in the dual frame buffer 203 through the digital data input device 201.

The image data stored in the dual frame buffer 203 is transferred back to the mass data storage device 205.

The selection of the analog video signal and the digital video signal can be arbitrarily determined.

The video frame editor 211 cuts out only a necessary portion of the video data recorded in the mass data storage device 205 and sets an occurrence time for each frame.

More specifically, the basic role of the image frame editor 211 is to cut out only the portion of the content of the soccer game recorded as a video corresponding to the time when the game was actually played. In other words, only the first half, the second half, or overtime will be drawn out.

If the video data stored in the mass data storage device 205 is inputted from multiple video cameras, the operation of unifying the total number of frames into one while editing the video data shot from each video camera is performed in parallel. .

For example, if the contents of a given soccer game were photographed through two video cameras, the length of the shooting time may be different depending on the person operating the video camera. Some people may start shooting at the beginning of a match, while others may start shooting five minutes before the game begins.

In addition, the shooting time may not only be different, but may or may not be taken for a corresponding time before the end of the first half and the start of the second half.

Therefore, an unnecessary portion of the image data captured by each image camera is cut out to match the entire length of the shooting time.

Once editing is performed to extract only the necessary parts from the video data, a problem may occur when the number of video frames generated per second varies depending on the type of camera. That is, even if the time of recording the content of the game is set equally, if the number of video frames generated per second is different, the total number of video frames generated during a given time varies for each camera.

Therefore, when the frame generation rates of the used cameras are different from each other, the image frame editor 211 determines the total number of frames based on the criteria of the camera generating the image frames at the lowest rate. For example, if an image camera generating 16 frames per second and an image camera generating 32 frames per second are used, the total number of image frames is unified based on 16 frames per second.

As shown in FIG. 5, what is shown below indicates the time axis by dividing 1 second by 16 equal parts. Since the frame of the image frame editor is given to the image camera generating 16 frames per second, the image frame generated from the camera B is synchronized with this time. That is, one frame per second will occur.

In comparison, Camera A, which generates 32 frames per second, is equivalent to generating two frames per second. Therefore, one of two frames is discarded in order to match the number of frame occurrences of the camera B. In this case, the discarded frame is determined as a frame located far from a reference time that occurs once per 1/16 second.

That is, an image frame having an even number of frames is selected and one having an odd number of frames is discarded. However, when two video cameras do not actually create an image frame at exactly the same moment, there may be a slight delay time, Δt, between the image frames produced by the two cameras.

If a person travels at a speed of 21 kilometers per hour, given Δt = 1/32, this means that an error of about 18.2 centimeters can occur through the video frame editing process. That is, a calculation error may exist between 18.2 centimeters between the position coordinate value calculated by the camera A and the position coordinate value of the same object calculated by the camera B.

As a method of solving such an error problem, in the present invention, the synchronization pulse generator 208 generates and supplies a synchronization signal that designates a time for creating a frame by the video camera.

When the synchronization signal generated at the same time is supplied to the cameras connected to the outside of the image processing apparatus 200, it is possible to set the frame generation time, which is equivalent to making Δt to zero. Therefore, it becomes possible to eliminate the error which appears in a position coordinate value.

The system clock generator 209 supplies a frame generation time to the image frame editor 211.

The pattern recognition processor 212 reads the image frame recorded in the mass data storage device 205 to determine the position coordinate values of the objects, and stores the result in the mass data storage device 205.

The MPEG encoder 213 sequentially reads and compresses an image frame stored in the mass data storage 205 and stores the result in the mass data storage 205 again.

The MPEG decoder 214 reads MPEG image data stored in the mass data storage 205 and restores the original moving image.

The motion parameter storage 210 is used to temporarily store parameters generated during MPEG encoding and decoding.

The user interface 215 receives a user's command such as a mouse, a keyboard, a joystick, and the like.

The controller 216 transmits a control command through the user interface 215 to the image frame editor 211, the pattern recognition processor 212, the MPEG encoder 213, and the MPEG decoder 214. do.

The overlay buffer 207 superimposes graphic information on the image information, and the graphic signal generator 206 is used as an output device for showing the image currently being processed to the user.

The sync pulse generator 208 is used for the purpose of specifying the time when the image camera 100 generates an image frame.

6 is a flowchart illustrating an image processing process according to the present invention.

As shown, the step of removing unnecessary parts from the contents of the football game recorded by the image frame editor 110 and extracting and editing only the part in which the game actually proceeds (S10) and storing the edited video frame in a large amount of data Reading from the device 205 to set the resolution (S20), determining the position coordinates of the objects of the edited image frame by the pattern recognition processor 212 (S30), and the position coordinates of the objects. Step S40 of determining whether an error has occurred in calculating the value, if an error occurs, correcting the error through the user interface 215 through the user interface S215, and if an error does not occur, Whether it is a frame (S60); if it is the last frame, reading the edited video frame from the mass data storage device (205) (S70); As the pattern recognition processing is completed for all the image frames, the recorded motion pictures are compressed by the MPEG encoder 213 and stored in the mass data storage device 205 (S80). In step S90, the position coordinate values of the objects calculated by the pattern recognition processor 212 and the image data encoded in MPEG by the MPEG encoder 213, that is, the basic information are stored in the mass data storage device. In step S205, the process is performed.

7 is an exemplary view showing an example of a configuration method of a control panel which is a user interface according to the present invention, FIG. 8 is an exemplary view showing screen division of a monitor used in conjunction with FIG. 7, and FIG. 9 is a user according to the present invention. This is a flowchart for editing video created from two video cameras through the interface.

The control panel, which is a user interface for the image frame editor 211, is largely divided into a part for handling a reference camera image and a part for handling a camera image to synchronize with the reference camera image.

The components constituting the reference camera image include a camera selection combo box 211a, a frame selection slide bar 211b, a play button 211c, a stop button 211d, and a forward frame search. Button 211e, reverse button 211f to search backward frames, edit box 211g showing the current frame number, edit box 211h showing the current frame as time, and skipping steps when searching for frames back and forth Move interval edit box 211i to adjust, frame setting button 211j for specifying the first half start point, frame setting button 211k for specifying the end point of the first half, frame setting button 211m for specifying the second half start point, The frame setting button 211n which designates the end time of the 2nd half, and the edit box 211p which designates the number of frames which generate | occur | produce per second are comprised.

The part for adjusting the camera frame to synchronize with the reference camera has a synchronization setting button 211q that is made according to the length of the frame set in the reference camera.

For example, suppose that the rate of frame generation per second of camera B is 16 and the length of time of the recorded race is 1 hour 55 minutes. It is also assumed that the first half starts near the frame number 300. (If you want to adjust the frame for extension, add a few more buttons.) Adjust the slide bar to change the current frame number and time automatically. Adjust the slide bar so that the current frame number is 300. If the current frame number is 300, press the first half start setting button. Frame number 300 will then appear in the edit box located next to the Settings button. When the start frame is set, the frame and two frames before and after are shown in the left part of the monitor shown in FIG. That is, the image set as the current start frame is shown at the position of B [n], and the previous two frames and the subsequent two frames are respectively B [n-2], B [n-1], B [n + 1], It is shown at the position of B [n + 2].

If you want to adjust the start point forward or backward, use the forward button 211e or the reverse button 211f to move. When the start frame number is changed, press the start setting button 211j again. Next, move the slide bar forward to set the end time of the first half. Since the start of the first half is known through the setting, move to the point where about 45 minutes have been added. Searches forward and backward around the moved frame to determine the frame at which the first half ends. When the frame in which the first half ends is selected, the first half end setting button 211k is pressed. The number of the corresponding frame will appear in the edit box next to it. If the above process is repeated for the second half, it is possible to determine the start frame and the end frame of the second half.

After setting the reference video camera, select the camera A to be synchronized. Once the camera is selected, adjust the slide bar to find the start of the first half. When you get near the starting point, press the Start 1st Half setup button. Then two consecutive frames, front and back with respect to the start frame, are shown on the right side of the monitor. The first half start frame of the reference video camera is displayed on the left side of the monitor by pressing the first half start setting button. Adjust the starting frame of camera A back and forth to find the frame that most closely matches the starting point of the reference camera. When the start frame of the camera A is set, the frame rate per second of the camera A is input, and the synchronization setting button 211q is pressed. Since the length of the frame constituting the first half is determined by the image of the reference camera, when the start point and the number of frames generated per second are determined, the end point of the first half is automatically determined. When the synchronization setting is completed, the first half frame number appears in the edit box. Frame synchronization for the second half is also determined by repeating this process. If a player is replaced during the first half, or if the game is temporarily stopped due to injury of the player, it may be cut out through the video frame editing process.

The above process is summarized as shown in FIG.

As shown in FIG. 9, after reproducing the reference camera image to search for the start point of the game and setting the corresponding frame (S11), and searching and setting the frame corresponding to the end point of the game (S12); Displaying a plurality of consecutive frames in front of and behind the reference camera image on the left side of the monitor (S13), and searching for the camera image to be synchronized to set a frame close to the starting point of the reference camera image. In step S14, displaying a plurality of consecutive frames before and after the start point on the right side of the monitor, in step S15, and selecting a frame representing the moment closest to the image corresponding to the start point of the reference camera. (S16), determining whether it is the closest frame (S17), and when it is the closest frame, for the camera image to be synchronized. Selecting the synchronization setting (S18) and, if it is not the closest frame, step (S19) to search for the frame of the closest moment while moving back and forth the image of the camera to be synchronized.

When the process of a given soccer game goes through the video frame editing process, it is divided into several frame groups. When a match is played normally, there are four frame groups that can occur: 1st half, 2nd half, extra 1st half, 2nd half. However, more frame groups can be created if the player is replaced during the first half, or if the game is temporarily suspended due to injury of the player.

Therefore, a data structure for distinguishing each frame group is defined and used as shown in FIG. 10.

FIG. 10 is a diagram illustrating usage of defining and using a data structure for distinguishing each group of frames according to the present invention. FIG.

As shown, the game ID number is used to distinguish different soccer games and is set to a 4-byte number. If different frame groups have the same race ID number, it means that the frame groups constitute the same race. The frame group start time is expressed in 4 bytes and hexadecimal numbers. The meaning of the frame group start time indicates the data format of the frame group start time.

Byte configuration meaning Explanation 0xHH time H, M, and S have a value between 0 and 6, and T has a value between 0 and 9. For example, if the time is given by 0x01103180, this means 01: 10: 31.80 seconds. 0xMM minute 0xSS second 0xTT A hundredth

The match index is given as a 2-byte (16-bit) number and is used to indicate whether the frame group belongs to the first half, second half, or extra time. The meaning of each bit constituting the match index indicates the match index definition of the frame group as shown in Table 3 below.

Bit position Explanation B15 (MSB) If the bit value is '1', it means that it is a frame group belonging to the first half. B14 If the bit value is '1', it means that the frame group belongs to the second half. B13 If the bit value is '1', it means that the frame group belongs to the first half of extension. B12 If the bit value is '1', it means that the frame group belongs to the second half of the extension. B11 If the bit value is '1', it means that it is a frame group belonging to sudden death. B10 If the bit value is '1', it means that the frame group belongs to the penalty shootout. B9-B7 Image data type. It points to different image formats according to the given values: '0' = Uncompressed, '1' = JPEG, '2' = MPEG. B6-B0 ID number of the video camera used to record the game

When the video frame editing process is completed, the number of video frames shot from all cameras is equally matched, the time at which the frames are generated is synchronized with each other, and all the unnecessary parts are removed. In addition, the total number of video frames constituting the first half and the second half, the game time, and the like are determined according to the above.

The pattern recognition processor calculates the position coordinates of each object with the image data belonging to all the edited frame groups.

11A and 11B are flowcharts for calculating position coordinate values of each object of image data belonging to all edited frame groups according to the present invention.

As shown in the figure, when the video frame editing process is completed, the total number K of video cameras used for shooting a soccer game, the number of frame groups M made through camera k, and the number of frames belonging to the frame group m of camera k. Step (S310) given as N (m), video camera index k = 1, frame group index m = 1 made through camera k, frame index m = 1 belonging to frame group m of camera k, and the screen of camera k Determining distance resolution between pixels, i.e., resolution, on the image (S320), determining k = k + 1 and k <= K (S330), and determining the resolution when k <= K. Return to (S320). if k <= K, then k = 1. Read image frame F (k, m, n) into a mass storage device and then perform pattern recognition processing to assign a unique ID to each recognized object (S340) k = k + 1 and k <= K In step S350, if k <= K, the process returns to step S340. If k <= K, the step of initializing the connection data structure for each object (S360), and the parameters constituting the connection data structure are frame group index, frame index, ID number of the object, and image photographing the object. Camera and index list, location coordinates and properties of the object. Next, a step of designating an image frame to be processed (S370) k = 1, n = n + 1 and a step of determining whether n <= N (S380), and when n <= N, an image frame F (k, m, n) is read from the mass storage device, and then subjected to pattern recognition processing, and checking whether there is an object moved from an adjacent area, and storing the calculated position coordinate values and attributes of the object in the connection data structure (S390) k = The method returns to step S400 of k + 1, k <= K and, if k <= K, the method returns to step S390 of storing the calculated position coordinates and attributes of the object in the connection data structure. If k <= K, the step of designating the next image frame to be processed (S410) k = 1, n = n + 1 and determining whether n <= N (S420), and n <= N In the case, the method returns to step S390 of storing the calculated position coordinate values and the attributes of the object in the connection data structure. If n <= N is not checked (S430), and if there is an error (S440), and if there is an error Compensating the error of the coordinate value through the user interface (S450), and specifying the next frame group to be processed (S460) and determining whether k = 1, m = m + 1 and m <= M ( S470) and the step (S480) is terminated when m <= M. If m <= M, the flow returns to step S340.

In order to calculate the position coordinate value of the object, first, the relationship between the playground coordinate system representing the stadium and the screen coordinate system used by the camera must be defined.

12 is an exemplary diagram illustrating a relationship between a playground coordinate system and a screen coordinate system when the image camera according to the present invention forms a square with the surface of the playground, and both the playground coordinate system and the screen coordinate system are defined as Cartesian coordinate systems.

As shown, the angle Ø and the direction angle between the camera and the surface of the playground By doing so, the playground coordinate system projected on the screen coordinate system does not appear in the rectangular coordinate system. In other words, perspective is applied to the playground coordinate system, which appears to be smaller in distant objects.

The distance resolution between pixels on the screen coordinate system, that is, the resolution capable of detecting the movement of the object, may be calculated by the side line of the playground projected on the screen coordinate system. For example, as shown in FIG. 12, the resolution from the origin of the screen coordinate system to the horizontal direction can be obtained by knowing the number of pixels connected between C3 and C4 and the actual distance in the playground. Point C3 on the screen coordinate system meets the side line S of the playground coordinate system created by connecting points D1 and D3. In this case, the distance between pixels of the line connecting the points D1 and D3, d ₁₃ is Is given by Therefore, the relationship between the points C3 and D3, d ₃₃ and d ₁₃ is given by Equation 1 below.

That is, if the distance between the points D1 and D3 in the playground coordinate system is 75 meters, the distance between the points C3 and D3 can be known. Similarly, we can find the distance between points C4 and D4 and eventually find the distance between C3 and C4. Since the number of pixels occupied on the screen coordinate system by the distance between the points C3 and C4 is given by the number of pixels existing between the points C3 and C4, the distance resolution per pixel can be calculated. In the same way, we can calculate the distance resolution of the horizontal axis h in the vertical direction in the screen coordinate system.

Thus, the distance resolution in the horizontal direction can be obtained from a point on any vertical axis given on the screen coordinate system, which is given by Equation 2 below.

From here Is the resolution at q = 0, Is the resolution at q = h. The resolution in the vertical direction on the screen coordinate system is given regardless of the position of the horizontal axis, and may be calculated from points where the vertical axis located at the origin of the screen coordinate system meets the playground coordinate system.

If the resolution determined by the position and orientation angle of a given camera and the magnification used is not satisfactory, you should increase the resolution by enlarging the magnification of the camera. However, when the magnification of the camera is enlarged, the side line for determining the shape of the playground may be pushed out of the screen, and thus the resolution may not be determined. In order to solve this problem, the present invention uses a method of designating a magnification for determining a resolution before a soccer game is played, and adjusting a desired magnification before the game starts. Through the user interface, the game will adjust the scale before the game begins to search for the shooting stadium frame and determine the resolution.

13 is an exemplary view showing an example of a configuration method of a resolution setting control panel which is a user interface according to the present invention.

As shown, the desired camera is selected through the combo box 215a for camera selection. When a camera is selected, the video recorded through the camera is shown in the frame window 215b. The slide bar 215c at the bottom of the frame window can be used to move to a desired position, and the play button 215d can be used to automatically change the screen. At this time, if the stop button 215 is pressed, the movement of the screen is stopped. When the user wants to move forward or backward one step from the stopped point, the forward button 215f and the reverse button 215g are used. When adjusting the movement interval, the desired value may be input in the edit box 215h. The currently shown frame and the total number of frames appear in the edit box 215i located below the frame window. This function is used to search for image frames for resolution setting captured by the selected camera. When the desired screen is searched, the magnification used for shooting is input by the setting magnification button 215j, the number of pixels in the horizontal direction of the camera is input by the number of pixels in the horizontal direction, and the number of pixels in the vertical direction is input. Is input to the vertical pixel number button 215m. Since not only the contour of the playground but also various objects exist in the captured image, only the contour is extracted by pressing the stadium contour extraction button 215n. At this time, the method of extracting the contour may be an edge detection algorithm or a general image processing algorithm. When the outline of the stadium is extracted, the baseline height is set using the baseline height button 215p. The height of the baseline is given by the number of pixels. Finally, when the resolution setting button 215q is pressed, the resolution can be obtained using the given parameters. When the resolution is calculated, the result is output in the edit boxes of the three resolution buttons 215t, 215u, and 215v.

Entering the enlargement magnification set for shooting a game to the competition magnification button 215r and saving the result of the resolution setting using the button 215s saves all procedures. The above procedure is repeated for all the cameras used.

The results calculated through the resolution setting process and the parameters used are stored in the mass storage device in a data structure as shown in FIG.

14 is an exemplary diagram illustrating a data structure in which a result calculated through a resolution setting process and parameters used are stored according to the present invention.

As shown, the game ID number is used to distinguish different soccer games and is set to a 4-byte number. If different frame groups have the same race ID number, it means that the frame groups constitute the same race.

Just as the resolution of the horizontal and vertical direction of the screen coordinate system can be obtained through the relationship in which the playground coordinate system is mapped to the screen coordinate system, the position coordinate value of the playground coordinate system to which any point on the screen coordinate system is mapped by mapping the screen coordinate system to the playground coordinate system. It can be seen.

15 is an exemplary view illustrating projecting the screen coordinate system according to the present invention into a playground coordinate system.

As shown in FIG. 12, the positions of four points, E ₁ , E ₂ , C ₃ , and C ₄ where the screen coordinate system meets the playground coordinate system may be obtained using Equation 2. From this, it can be seen that O '(x ₁ , y ₁ ) representing the position where the origin of the screen coordinate system is mapped to the playground coordinate system. Also, since the value of x ₂ and y ₂ can be known, the rotation angle between the screen coordinate system and the playground coordinate system Is given by Equation 3 below.

Next, how any pixel on the screen coordinate system, S (p, q), is mapped on the playground coordinate system will be described.

Pixels located at a distance in the horizontal direction and q in the vertical direction from the origin of the screen coordinate system can be obtained from the actual distance by using the resolution in the vertical direction and the horizontal direction of the screen coordinate system. For example, if p = 10 and the resolution in the horizontal direction is 15 centimeters per pixel, it means that it is actually 150 centimeters apart. Since the resolution of the screen coordinate system is already known, given the values of p and q, we know the values of p 'and q' on the playground coordinate system. Therefore, the distance L between O 'and S' is given by Equation 4.

Rotation angle between the screen coordinate system and the playground coordinate system, Since the values of α and β can be obtained using Equations 5 and 6 below.

The position coordinates of any point S (p, q) on the screen coordinate system mapped to the playground coordinate system are given by Equations 7 and 8 as follows.

Using the above equations, we can find the position coordinates in the playground coordinate system for any pixel in the screen coordinate system. That is, the distance from the origin of the playground coordinate system can be obtained. Clearly, knowing the position at which the object captured by the video camera is placed or standing on the screen coordinate system is the same as knowing the position coordinate value on the playground coordinate system where a one-to-one mapping relationship is established. A main role of the pattern recognition processor, which is a component of the present invention, is to recognize an object captured by an image camera on a screen coordinate system, and then map the position to a playground coordinate system to convert the position to a position coordinate value on the playground coordinate system. In addition, the position coordinate value of any object can be obtained with only one still image. The trajectory of the movement of an object is like the sequence of position coordinates calculated in each still image.

As a method of recognizing an object in a pattern recognition processor, image processing algorithms which are generally known may be used, and among them, a morphological pattern recognition algorithm may be used. Morphological pattern recognition algorithm is a method of recognition based on the morphological features of an object. If a person wants to recognize a person within a given image frame, he uses the morphological features of the person as parameters. In other words, a person's figure can be classified into head, torso, arms, and legs, and using this information, an object with human characteristics is found within a given image frame. If you want to recognize the ball, you can use the round circle feature.

When a person is recognized within a given screen coordinate system, the next position where the person stands is determined. The standing position of a person means the position where the foot is placed. This is because the part where the standing person touches the ground becomes a bridge. Using the morphological pattern recognition algorithm, it is possible to estimate the body's torso and leg parts, so that it is possible to estimate where the legs are in contact with the ground. The position where the recognized person's leg is placed becomes the position on the screen coordinate system of the given person, and thus the position coordinate value in the playground coordinate system is determined.

16 illustrates an example of a recognized person and estimated location coordinates on a screen coordinate system.

As shown, when a person is recognized through a pattern recognition algorithm, a circle 212a is automatically drawn in dotted lines on the head and torso of the person. These circles are shown in the form of overlays on the original image and are not written in the image frame. With the circle drawn on the body, the position of the person standing is determined graphically as well. When the position 212b is determined on the screen coordinate system, the actual position coordinate value is calculated by mapping to the playground coordinate system. The information 212c related to the recognized person is shown slightly upwards in a diagonal direction, where the unique ID number, name, and location coordinates in the playground coordinate system are displayed. When a person moves, the graphic information drawn on it moves along with it. If a person is lying on the playground, the position coordinates of the person are similarly assumed to be in the leg portion. When a person moves, the shape of the legs changes, which affects the location coordinates of a given person. This influence should be applied to the morphological pattern recognition algorithm, and in some cases, an error can be made in the position coordinate value.

When the resolution determination process is completed for a given camera, the first image frame belonging to the frame group to be processed is read from the mass storage device and the pattern recognition algorithm described above is applied. When all objects are recognized within a given image frame, a unique ID number is assigned to each recognized object using a user interface as shown in FIG. 17.

17A is an exemplary view illustrating an example of a method of configuring a pattern recognition processor control panel which is a user interface according to the present invention, and FIGS. 17B and 17C are exemplary views illustrating a change when connecting an object in FIG. 17A.

As shown in FIG. 17A, one of the cameras is selected using the camera selection button 212d and a frame group to be processed using the frame group selection button 212e. Press the Go First button 212f to get the first frame of the selected frame group. When the first frame is shown in the frame window, the pattern recognition processing (current frame) button 212g is pressed. Information about the object recognized in the first frame is then displayed in the frame window and the object list.

Objects recognized in the first frame of a given frame group have not been assigned an ID number yet, so they are automatically assigned a temporary ID number beginning with the letter 'X'. For example, if a person named 'Hong Gil Nam' corresponds to an object 'X3', 'X3' is selected through the entity ID combo box 212h for 'Hong Gil Nam'.

Subsequently, as shown in 17b, when 'Hong Gil-Nam' is connected to the object 'X3', the ID number and name of 'Hong Gil-Nam' are displayed on the object 'X3' 212i shown in the frame window. The position coordinate value of 'X3' is displayed on the position coordinate value 212j of 'Hong Gil Nam'.

Likewise, you can link objects in the same way for Hong Gil-dong and Hong Gil-seo. When the above connection is terminated for the first frame of a frame group belonging to a given camera, the same is repeated for the frame group belonging to all used cameras. If the shooting areas of Camera A and Camera B overlap slightly and 'Hong Gil-Nam' is located in the overlapped area, 'Hong Gil-Nam' will be visible on both the first frame of Camera A and Camera B. Then, the position coordinate value of 'Hong Gil Nam' is determined based on the position coordinate value calculated by the camera A and the position coordinate value calculated by the camera B. If the average of the position coordinate values calculated by the two cameras is used as the position coordinate value of 'Hong Gil Nam', the result is given as shown in FIG. 17C.

As shown in FIG. 17C, the position coordinate value of 'Hong Gil Nam' calculated by the frame captured by the camera A is given by (+ 26, -16), and the position coordinate value captured by the camera B is (+ 28, -14). to be. Therefore, the final position coordinate value of 'Hong Gil Nam' is given as (+ 27, -15) which is the average of two coordinate values. If the position coordinate value is not determined, 'ZZZ / ZZZ' is displayed. When the ID numbers for all the objects appearing in the first frame of the entire frame group are finished, the pattern recognition processing (continuous) button 212k is pressed. Then, the next frame is read and the pattern recognition process is automatically and continuously performed. If the pattern recognition process is completed successfully, it moves to the next frame group. When the calculation of the position coordinate value is finished for all frame groups, the pattern recognition process is finished.

If any error occurs while the continuous pattern recognition process is in progress, the pattern recognition process stops at the frame where the error occurs and waits for the user's processing. An error that occurs during the successive pattern recognition process occurs when no position coordinate value of a given object is determined. In other words, no camera can recognize a given object. When an error occurs as such, the frame number in which the error occurs is displayed in the error frame edit box 212m, and the frame is shown in the frame window. The user examines the frame in which the error occurred and investigates why the object was not recognized. The case where the pattern recognition processor does not recognize any object is largely a case where the object is not recognized even though there is an object in the frame and the object is not found in any frame.

If the object is in the frame but is not recognized, the pattern recognition (current frame) button 212g is pressed again to try again. After the object is recognized, the ID number of the object is connected and the pattern recognition processing (continuous) button 212k is pressed to continue.

If the object recognition fails even with the pattern recognition processing (current frame) button 212g, the error correction button 212n is pressed. A cursor appears in the frame window, and when you move the cursor over the object you want to recognize and click, it finds the object in a certain area around the selected point. When the object is recognized, the ID number is connected and then the pattern recognition processing (continuous) button 212k is pressed to continue.

Next, if the object is not found in any frame, it means that the object is out of the camera's shooting area. When the object leaves the shooting area, the position coordinate value calculated last is automatically used as it is.

When an object that has left the shooting area enters the shooting area again, it is recognized as a new object that has not existed before, and thus the error correction procedure described above is performed. When the error correction for the object which reenters the shooting area is completed, the pattern recognition processing procedure is continuously resumed.

When the recognition rate of the pattern recognition algorithm drops, error correction is frequently performed, which reduces the efficiency of the pattern recognition processor. Pattern recognition algorithms can be seen as a way of recognizing an object separately from its surroundings, so as a way to increase the recognition rate, it is to adjust the color of the uniform worn by the players. If the uniform of the uniforms worn by the players is uniformed to blue, and the blue is filtered during the image processing, like the blue screen method commonly used in the movie, only the uniform part of the players can be cut out of the image frame. If the part of the uniform is cut off, it will be much easier to distinguish than before it is cut out, and if the object is recognized in this state, it is possible to improve the recognition rate. Uniform uniform colors, then partly different patterns can be distinguished from the other team.

When an object leaves the shooting area of a given camera and enters the shooting area of an adjacent camera, whenever an adjacent camera knows that the object is out of the shooting area of the previous camera, it will It is possible to prevent the user from performing error correction.

18 is a diagram illustrating a camera handoff process for automatically recognizing an object moving to an adjacent photographing area according to the present invention.

As shown, when an object 30a is transferred from the photographing area of camera A to the photographing area of camera B, three steps are performed.

In step I, the object has not yet appeared in the camera B's shooting area. In stage II, the object appears simultaneously in the camera A and camera B shooting areas. That is, the camera B corresponds to the step of recognizing the newly entered object. When a new object is recognized, Camera B searches for objects belonging to nearby cameras to find out where it came from. The parameter used at this time is the position coordinate value of the recognized object. In other words, by searching for the objects belonging to the surrounding camera shooting area in order to investigate the difference in the position coordinate value. At this time, if the object having the closest position coordinate value is found, the object is determined as the moving object. That is, the entity representing the nearest position coordinate value is determined by the following equation (9).

Here, (x ₀ , y ₀ ) is the position coordinate value of the object newly recognized by the camera B, and J represents the number of objects belonging to the surrounding camera photographing area. That is, the newly recognized entity has the same ID information as the entity having the smallest difference in position coordinate values. When Camera B automatically associates an ID number with a newly recognized object, it informs adjacent cameras that have that object. Then, when the object leaves its own shooting area, it can be detected whether it has moved to the shooting area of the adjacent camera or completely outside the entire shooting area. That is, if no camera recognizes an object that is outside its own shooting area, the object is completely out of the shooting area. The above process is defined as camera handoff. Using the camera handoff procedure, it is possible to continuously track an object once it has been recognized, even if the whole playground is divided into several smaller shot areas.

When the position coordinate values are determined for all frame groups belonging to all cameras used for recording the football game, the result is stored in the mass storage device in the form of the data structure shown in FIG.

19 is an exemplary diagram illustrating a form of a data structure in which a result of a mode frame group in which a position coordinate value is determined according to the present invention is stored.

As shown, all entities recognized during the race store the calculated position coordinate values and camera ID numbers in the coordinate record. The byte length of the coordinate record is recorded in the 'LEN [m]' field and the order in which the remaining information appears is as given in FIG. (X [0], Y [0]) is the position coordinate value of the object in the given frame, and these values are the position coordinate values (X [n], Y [n]) calculated by the cameras that recognize the object. It is obtained from these (where n = 1,, N). Certainly the data structure given in FIG. 19 is for storing information as shown in FIG. 17C and is made up by the number of recognized entities.

When the process of determining the position coordinate value by recognizing the object using the image frame recorded through the image camera is completed, the error correction is performed by examining the change of the calculated position coordinate values, that is, the trajectory. At this time, the errors to be corrected are largely divided into three types.

First, if the video camera is square with the surface of the playground, if a player jumps from the standing position, this may be indicated by the movement of the position coordinate.

That is, since the pattern recognition algorithm estimates the position based on the point where the bridge is located, there is no way to recognize when a player jumps. Therefore, this error is corrected at the end without correcting during the pattern recognition algorithm. This problem does not occur when the camera is installed on the ceiling of the playground. This is because the direction the camera looks parallel to the human body.

Second, if two objects stick together and fall off, a crossover may occur where the trajectory is reversed. If two athletes are fighting in close proximity, the pattern recognition algorithm can confuse the two individuals. In this state, when two objects are separated from each other, there may be a problem of inverting ID numbers connected with position coordinate values.

Third, the distance between the athlete's torso and the ground will vary when the object is standing and running. Therefore, when looking at the position coordinate values calculated by the pattern recognition algorithm when the player is standing and running around as a single trajectory, a change in the zigzag shape can be found. This problem is smoothed by using the curve fitting method, which is one of the graphic processing algorithms within the tolerance of the position coordinate value.

20 is a view showing a change in the trajectory that appears when a person jumps in a standing position according to the present invention.

As shown in the figure, since the image camera is looking at the playground in a square, when the position is calculated based on the leg portion, the difference is Δj from the actual position. If the person who made the jump touches the ground again, it will fall to the same position as before the jump. Δd is the distance between the position before the jump and the position after the jump and falls back to the ground, and you can see how the movement of the person depends on its size. In other words, if you jumped from the standing position, it will be close to Δd → 0. If you jumped while running, Δd> 0. Therefore, by examining the trajectories of the calculated position coordinate values, it is possible to indirectly determine whether or not a given object has jumped and correct them. That is, make Δj zero.

21 is a diagram illustrating a case where crossover between two entities occurs according to the present invention.

As shown, when an object having an ID number of 'X1' and an object having an ID number of 'X2' cross over each other while moving, a phenomenon may occur in which an ID number between two objects is changed. However, due to the limitations of the pattern recognition algorithm, it is very difficult to find a means to detect it. This is because two different objects overlap each other. As such, when a crossover phenomenon occurs, a method of correcting one by one by investigating where crossover occurs after all position calculation is completed is used. In other words, where the trajectories of the two objects cross each other in a cross shape is the subject of the investigation. As a method of preventing such a crossover phenomenon, a plurality of video cameras may be arranged in the same photographing area. In other words, when the cameras are arranged to look at the same shooting region from different angles, even if two objects overlap in one camera, the camera in the other direction increases the probability of distinguishing them.

FIG. 22 is a diagram illustrating smoothly changing a trajectory in a zigzag shape by a curve fitting method.

As shown, the original trajectory, given in a zigzag pattern, is replaced with a new trajectory in a soft form. However, it may also occur in part to be far from the original trajectory, depending on the curve fitting method applied. The curve fitting method generally selects the methods used in the graphic processing algorithm.

FIG. 23 is a diagram illustrating an example of a user interface that processes errors in FIGS. 20 to 22.

As shown, the frame group selection combo box is used to select a frame group whose error is to be corrected. The segment length setting box is used to mark the trace of the object as a chopped fragment on the playground-shaped frame window on the left side of the control panel. If the frame rate is 32 frames per second, and the segment length is 64, this means that the movement trajectory for 2 seconds is displayed.

The error mode selection is to set the type of error to be corrected in the selected frame group. There are three types of errors described above and can be selected individually or all. However, it is usually desirable to select curve fitting at all times, and when curve fitting is selected, it is applied first before compensating for other kinds of errors. In other words, other errors are corrected in the state where the curve fitting is completed. There are two main operating mode choices: one to handle all error corrections automatically, and one to handle each time an error is detected. When operating in the user selection mode, if an error is detected, the user corrects the error using the Calibrate button. After the calibration is completed, movement between segments is done using the Next and Previous buttons. FIG. 23 shows how the three types of errors appear. Even if the trajectory segments of the two objects cross each other, the crossover does not hold when the distance between the two is far apart at a given time. When a crossover occurs and corrections are made, ID numbers that are reversed will be returned. After the error correction process is completed, the corrected position coordinate values are stored in the mass storage device, and the data structure thereof is the same as described with reference to FIG. 19.

After calculating the position coordinates and correcting the error for all the edited frame groups, the memory space required for compressing all the images and storing them in the mass storage device can be saved. Since the images have already been computed, it is not a big problem if the information is lost during the compression process. When compressing and storing the video, JPEG or MPEG methods can be used. In the present invention, images belonging to a frame group are compressed through an MPEG encoder. If you want to restore the compressed images again, use the MPEG decoder.

Images and data structures created by the image processing system of the present invention are all stored in the form of a database and can retrieve desired information whenever necessary. Basic information defined for objectively recording a soccer game, that is, position coordinate values of all objects at a given moment and image frames for recording the objects are all stored in the database. In addition to these basic information, a data structure for storing other information related to a football game can be defined, and the information shown in Table 4 below is included.

Data item Explanation The place where the game was held Points to the country name, city name, stadium name, and address. The time of the match The time of the match is recorded in standard time. Geographical location Indicate the latitude, longitude, and altitude where the competition is held. Human Resource Information for Team A Includes personal information about home team managers, players, coaches and assistants. Human Organization Information for Team B Includes biographical information about the team's manager, players, coaches, and assistants. Match information for Team A players Each team player will record the time he or she participated in the match. Match information for team B players For each player on the away team, record the time of participation. Game result Record the result of the match between the home team and the away team. Team A goal information The player who scored the goal from the home team and the time are recorded. Team B goal information The player who scored the goal from the away team and the time are recorded. Penalty Shootout Results If the outcome of the match is determined by penalty shootout, the result is recorded. Player A substitution information for Team A Record information about players who have been replaced and dismissed from home team. Player substitution information for team B Record information about the away player and the player who is sent off. Match ID Number Record the ID number of the database that recorded the match.

The information recorded in the database is used to analyze the content of the game.

24 is a diagram illustrating an embodiment of a user interface for analyzing contents by using information recorded in a database.

As shown, specifying the race ID number and the frame group reads basic information about the frame from the database. In the user interface shown in FIG. 24, information on the trajectory of each object is graphically displayed, and an image frame is illustrated through a separately installed monitor. At this time, an arbitrary monitor can be designated in order to show the images recording the desired object.

When information about the game is loaded from the database, information about each player, name, ID number, current position coordinate value, etc. is displayed on the user interface.

The trajectory shown in the graphics window can be adjusted in length and specified via the segment settings edit box. The trajectory of each object being played can be controlled by the speed control slide bar. In other words, if you want to play fast, you can move the speed adjustment slide bar to speed it up. In addition, it is possible to repeatedly play only the desired portion separately. For this purpose, the start position setting button and the end position setting button are used.

In addition, it is possible to set the playback stop condition. When a designated player enters a predetermined area, it detects the fact and stops playback. For example, if Hong Gil-dong enters the predetermined area P, playback stops and it is possible to perform detailed trajectory analysis based on the point in time. If the area P is set near the goal area, it can be easily used to analyze the attack pattern of the opponent or the defense pattern. In addition, it is possible to select a desired part by using the partial repeat play function and to store it separately.

In the above described and illustrated with respect to the preferred embodiment of the present invention, the present invention is not limited to the above-described embodiment, the general knowledge in the field of the present invention without departing from the gist of the invention claimed in the claims Anyone with a variety of modifications can be made, as well as such modifications are within the scope of the claims.

As described above, according to the object information image processing system and database construction method of the soccer game of the present invention, by extracting the position coordinate values of the players and the ball belonging to the object in the video frame of the soccer game photographed through the video camera By the database with the image data, there is an effect that can be used for the objective analysis of the football game.

Claims (8)

An image camera 100 for generating an image frame by capturing an image of a game; An image processing apparatus 200 for extracting position coordinate values of objects appearing in the image frame and making a database together with the image frame; A monitor 300 for outputting the position coordinate values and the image frame; And Object information image processing system of a soccer game, characterized in that the input means 400 for transmitting a user's control command to the image processing apparatus (200). The image processing apparatus 200 of claim 1, A sync pulse generator (208) for designating a time at which the video frame is generated by giving a sync signal to the video camera (100); A mass data storage device 205 for storing the image frame; An image frame editor (211) for editing only a portion of the image frame in which a game is actually played; A pattern recognition processor 212 sequentially extracting position coordinate values of objects from the edited image frame; An MPEG encoder (213) for sequentially compressing the edited video frame and storing it in the mass data storage device (205); An MPEG decoder (214) for reconstructing the compressed video frame; A graphic signal generator 206 for generating a graphic signal of the reconstructed image frame; A user interface 215 for receiving a control command from the input means 400; And And a controller 216 for controlling the control command and transferring the image frame editor 211, the pattern recognition processor 212, and the MPEG encoder 213 and the MPEG decoder 214. Object information image processing system of a soccer game. The method of claim 2, And a motion parameter memory (210) for temporarily storing parameters generated during compression and decompression of the edited video frame. A step in which only a portion of the game actually being played is extracted by the image frame editor 110 and the edited image frame is stored in the mass data storage device 205 (S10); Reading the edited image frame from the mass data storage device 205 and setting a resolution (S20); Determining, by the pattern recognition processor 212, position coordinate values of objects in the edited image frame to be sequentially extracted (S30); Determining whether an error of position coordinate values of the objects occurs (S40); If an error does not occur, determining whether the edited video frame is the last frame (S60); If the edited video frame is the last frame, reading the edited video frame in which no error occurs from the mass data storage device (205) (S70); The edited video frame is compressed by an MPEG encoder (213) and stored in the mass data storage device (205) (S80); Determining whether the edited video frame compressed and stored in the mass data storage device 205 is the last frame (S90); And If the image frame to be compressed and stored is the last frame, step S100 of storing the position coordinate values of the objects and the compressed image frame in the mass data storage device 205 in a database form. An object information image processing database construction method for soccer game. The method of claim 4, wherein If an error occurs in step S40 of determining whether an error occurs in the position coordinate values of the objects, correcting through the user interface 215 (S50), the object information image of a soccer game How to build a processing database. 5. The method of claim 4, wherein only the portion of the game that is actually played by the image frame editor 110 is extracted and the edited image frame is stored in the mass data storage device 205 (S10). Retrieving and setting an image frame corresponding to a start point of a game by playing the reference camera image (S11); Searching and setting an image frame corresponding to an end point of the played game (S12); Displaying a plurality of consecutive front and rear image frames on the left side of the monitor 300 centering on the image frame corresponding to the starting point (S13); Setting an image frame close to an image frame corresponding to a starting point of the reference camera by searching for an image of a camera to be synchronized (S14); Displaying a plurality of consecutive front and rear image frames on the right side of the monitor (300) around the image frame close to the image frame corresponding to the start point of the reference camera (S15); Checking whether an image frame indicating a moment closest to the image frame corresponding to the starting point of the reference camera is selected (S16); Determining whether the image frame represents a moment closest to the image frame corresponding to the starting point of the reference camera (S17); And In the case of the image frame indicating the closest moment, the step (S18) of selecting a synchronization setting for the image of the camera to be synchronized, characterized in that the object information image processing database construction method of a soccer game. The method of claim 6, If it is not an image frame indicating the closest moment in step S17, it is determined whether the image frame indicates the moment closest to the image frame corresponding to the start point of the reference camera. And (S19) retrieving an image frame representing the closest moment by moving back and forth. The method of claim 4, wherein In the step S20 of reading the edited image frame from the mass data storage device 205 and setting the resolution, the resolution is set by extracting the contour of the playing field and setting the baseline height to calculate the resolution. An object information image processing database construction method for a soccer game.