JP2011164668A - Image processor and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To present valid information to a user who has a strong interest in sports in a service for presenting information obtained by analyzing a sport video. <P>SOLUTION: About a video section set by a user, a number of player calculation means of a formation data generation part calculates the average number of players Np(i,j) on the basis of player layout data for each of preliminarily set divided areas (i,j). A motion vector calculation means calculates an average motion vector Vp(i,j) on the basis of the player layout data. Then, a formation data generation means generates formation data including the average number of players Np(i,j) and the average motion vector Vp(i,j) for each video section. The formation data are displayed at a display unit, so that it is possible for a user to recognize the formation of players globally catching a whole field. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an image processing apparatus and a program, and more particularly, to an image processing field for analyzing a sports video.

  Conventionally, development of image processing technology has been developed to analyze and analyze sports video to realize classification and search based on player actions (eg, shoot, free kick), team formation, team tactics, etc. Reference 1, 2, 3).

  Among such image processing techniques, a technique for automatically tracking a sports player is known. For example, the method of Non-Patent Document 3 extracts lines by Hough transform and generates line fitting by wire frame deformation when generating arrangement data of athletes. However, with this method, the calculation cost of Hough transform and wire frame deformation is high, and for example, a calculation time of 90 minutes is required for 3 seconds of video processing. In view of this, a method has been proposed in which line and camera angles are specified by discrete search for the purpose of reducing calculation costs, and sports player arrangement data is generated (see Non-Patent Document 4).

  In general, in the method of automatically tracking a sports player, it is necessary to consider camera switching, a rapid change in camera movement, etc., and therefore tracking a player for only about several tens of frames of video data. However, the player arrangement data is generated while initializing tracking every several tens of frames. That is, player arrangement data including the player position and the player motion vector is generated every several tens of frames.

T. Watanabe et.al, “A soccer field tracking method with wire frame model from tv images”, ICIP 2004, pp.III-1633-1636. Shimawaki, et al., “Analysis of long-time soccer broadcast video for scene search system”, Information Processing Research Report, 2004-CVIM-144, pp.125-132. Nakagawa, et al., “Automatic Game Analysis of Soccer Video”, Information Processing Research Report, 2001-DPS-106, pp.193-198 Takahiro Mochizuki, Masato Fujii and 3 others, “Fast soccer player placement method by discrete search of lines and camera angles”, FIT2008 7th Information Science and Technology Forum, I-045, 3rd volume, p. 289-290

  By the way, there is a service that presents live broadcast video or archive video of sports using a platform that can be distributed by WEB. With this service, the user can view sports videos. However, for users who have a strong interest in sports, such as commentators and critics who analyze sports, it is not sufficient to simply view sports videos. Therefore, in order to respond to the demands of users who have a strong interest in sports, not only sports videos but also useful information (for example, team formation data, search results for scenes similar to a specific formation) are presented. Service is sought.

  In order to realize such a service, for example, it is necessary to convert a player's formation into a data that captures the entire field globally in a long video segment. The conventional method generates player arrangement data in a video section of about several tens of frames (several seconds), does not target a long video section of minutes, and does not convert a player's formation into data. .

  Therefore, the present invention has been made to solve the above-mentioned problems, and its purpose is useful for users who have a strong interest in sports in a service that presents information obtained by analyzing sports video. An object is to provide an image processing apparatus and a program capable of presenting information.

  In order to achieve the above object, the present invention comprises a storage means for storing a sports video in which a sports game is photographed, and player arrangement data including data on the position and movement of the player at predetermined time intervals in the sports video. In the image processing apparatus for generating formation data indicating the number of players and the movement of the players for each divided area obtained by dividing the field in which the game is played into a predetermined number based on the player placement data, the player placement is stored from the storage unit. A player number calculation means for reading out data and calculating the number of players for each of the divided areas for each predetermined video section based on the player position data included in the player placement data, and the player placement data from the storage means For each of the divided regions for each of the predetermined video sections based on the movement data of the players included in the player arrangement data. Based on the motion vector calculation means for calculating the motion vector of the player, the number of players calculated by the number of players calculation means, and the motion vector calculated by the motion vector calculation means, for each predetermined video section, It is characterized by comprising formation data generation means for generating formation data including the number of players and motion vectors for each divided area and storing them in the storage means.

  In addition, the image processing apparatus of the present invention reads the formation data from the storage means in which the formation data is stored, and according to the number of players included in the formation data for each of the divided areas of the predetermined video section. Player number display calculating means for calculating player number display data, motion vector display calculating means for calculating motion vector display data corresponding to a motion vector included in the formation data for each of the divided areas of the video section, Display that displays the number-of-players display data calculated by the number-of-players display calculation means and the motion vector display data calculated by the motion vector display calculation means in a predetermined form for each of the divided areas into which the field is divided. Means.

  The image processing apparatus according to the present invention further includes, from the storage unit, the first formation data for the sports video for the first video section and the same sports as the sports video for the second video section. Second formation data for a sports video in which another game is photographed is read out, and the number of players for each of the divided areas in the first video section included in the first formation data and the second formation data Calculate the difference between the number of players for each of the divided areas in the second video section included, and calculate the number of players similarity indicating the degree of similarity of the number of players between the formation data Means for each divided area in the first video section included in the first formation data. A motion vector that calculates a difference between the vector and a motion vector for each of the divided regions in the second video section included in the second formation data, and indicates a degree of similarity of the motion vectors between the two formation data Based on the motion vector similarity calculating means for calculating the similarity, the number of players similarity calculated by the player number similarity calculating means, and the motion vector similarity calculated by the motion vector similarity calculating means, And a similarity calculation means for calculating a similarity between the first formation data in the first video section and the second formation data in the second video section.

  Furthermore, the image processing program of the present invention causes a computer to function as the image processing apparatus.

  As described above, according to the present invention, the formation data including the motion vector indicating the number of players and the movement of the players is obtained for each divided region obtained by dividing the field for the video section at a certain time based on the player arrangement data. Generated. Further, according to the present invention, the number of players and the motion vector included in the formation data are displayed on the screen for each divided area obtained by dividing the field. Further, according to the present invention, the similarity between the first formation data for the sports video for the predetermined video section and the second formation data for the sports video of another game for the predetermined video section is calculated. Calculated. This makes it possible to present useful information to a user who has a strong interest in sports in a service that presents information obtained by analyzing sports videos.

It is a figure which shows the hardware constitutions of the image processing apparatus by embodiment of this invention. It is a block diagram which shows the function structure of the control part with which the image processing apparatus was equipped. It is a block diagram which shows the structure of a formation data generation part. It is a block diagram which shows the structure of a graphical presentation part. It is a block diagram which shows the structure of a similarity calculation part. It is a flowchart which shows the process of a formation data generation part. It is a flowchart which shows the process of a graphical presentation part. It is a flowchart which shows the process of a similarity calculation part. It is a figure explaining the player arrangement data contained in a video section. It is a figure explaining the example of a division | segmentation of a field. It is a figure explaining the process which produces | generates formation data from player arrangement | positioning data. It is a figure explaining player arrangement | positioning data. It is a figure explaining formation data. It is a figure which shows the example of a graphical presentation screen of formation data. It is a figure which shows the example of a display screen.

Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the drawings.
[Image processing device]
FIG. 1 is a diagram illustrating a hardware configuration of an image processing apparatus 1 according to an embodiment of the present invention. As the hardware configuration of the image processing apparatus 1, a normal computer can be used. The image processing apparatus 1 includes a CPU 101, a storage unit 102 including a ROM and a RAM for storing various programs and data, a storage device (hard disk device) 103 for storing various programs and data, and a network such as the Internet. A network interface 104 that transmits and receives data and the like, an operation / input interface 105 that controls the input of data in accordance with user operations on the operation unit 108 such as a mouse and a keyboard, display data that prompts the user to perform key operations, The display interface 106 controls the output of video data, formation data, and the like to the display device 109, and these components are connected to each other via a system bus 107.

  The storage device 103 stores an OS (operating system) and various application programs for providing basic functions of the image processing apparatus 1, video data 126, player arrangement data 127, formation data 128, and the like. Yes. These programs and data are read from the storage device 103 to the RAM of the storage unit 102 by the CPU 101 when the image processing apparatus 1 performs processing. FIG. 1 shows a state in which an operating system and various application programs are read from the storage device 103 to the storage unit 102. The storage unit 102 includes an operating system 120, a formation data generation program 121, and a graphical presentation. A program 122 and a similarity calculation program 123 are stored. Application programs such as the formation data generation program 121 are stored in a storage medium such as a magnetic disk (floppy (registered trademark) disk, hard disk, etc.), optical disk (CD-ROM, DVD, etc.), semiconductor memory, and distributed. Can be downloaded from an external server connected to the network to the storage device 103 via the network interface 104.

  Here, the operating system 120 is read out from the storage device 103 and executed by the CPU 101, and as a basic function of the image processing device 1, the storage unit 102, the storage device 103, the network interface 104, the operation / input interface 105, and the like. The display interface 106 is managed. Then, in a state where the operating system 120 is executed by the CPU 101, the above-described formation data generation program 121, the graphical presentation program 122, the similarity calculation program 123, and the like are executed.

  The control unit 110 includes a CPU 101 and a storage unit 102, and the CPU 101 reads out and executes various programs stored in the storage unit 102 and the storage device 103, thereby performing overall control of the entire image processing apparatus 1. As described above, in the image processing apparatus 1, the control unit 110 performs various processes according to the formation data generation program 121, the graphical presentation program 122, the similarity calculation program 123, and the like with the hardware configuration illustrated in FIG. 1.

  Hereinafter, the video data 126 will be described as video data in which a soccer game is shot. Further, it is assumed that the player arrangement data 127 is generated by performing image processing on the video data 126 by the method of Non-Patent Document 4 described above, for example. Details of the player arrangement data 127 and the formation data 128 will be described later.

  FIG. 2 is a block diagram illustrating a functional configuration of the control unit 110 included in the image processing apparatus 1. The control unit 110 illustrated in FIG. 1 uses the formation data generation program 121, the graphical presentation program 122, and the similarity calculation program 123. The functional structure at the time of performing various processes is shown. The control unit 110 includes a formation data generation unit 10, a graphical presentation unit 20, and a similarity calculation unit 30, and the storage unit 102 stores video data 126, player arrangement data 127, and formation data 128. . The video data 126, the player arrangement data 127, and the formation data 128 correspond to the respective data stored in the storage device 103 shown in FIG. 1, and FIG. A state in which data is read to the storage unit 102 is shown. The formation data 128 is stored in the storage device 103 by the formation data generation unit 10 via the storage unit 102.

  The formation data generation unit 10 reads the player arrangement data 127 from the storage unit 102, and in the video section set by the user, for each division area set in advance, the average number of players and the average for each team based on the player arrangement data The motion vector is calculated, formation data including the average number of players and the average motion vector is generated, and stored as the formation data 128 in the storage unit 102. Details of the formation data generation unit 10 will be described later.

  The graphical presentation unit 20 reads out the video data 126 and the formation data 128 from the storage unit 102, displays the video data 126 on the screen, and is set in advance in the video section set by the user according to the time synchronized with the video data 126. For each divided area, formation data including the average number of players and the average motion vector is displayed on the screen for each team. Further, the graphical presentation unit 20 inputs the similarity calculation result from the similarity calculation unit 30, reads out the video data 126 and the formation data 128 with high similarity from the storage unit 102, and displays them on the screen. Details of the graphical presentation unit 20 will be described later.

  The similarity calculation unit 30 reads the formation data 128 to be a reference (search query) and the formation data 128 to be searched for similarity calculation from the storage unit 102, calculates the number of players similarity and the motion vector similarity, The similarity between the two formation data is calculated. In addition, the similarity calculation unit 30 uses information for specifying formation data to be a reference (search query), information for specifying formation data to be searched for similarity calculation, and the similarity as a similarity calculation result. Is output to the graphical presentation unit 20. Details of the similarity calculation unit 30 will be described later.

[Formation data generator]
First, the formation data generation unit 10 shown in FIG. 2 will be described in detail. FIG. 3 is a block diagram illustrating a configuration of the formation data generation unit 10. The formation data generation unit 10 includes a player number calculation unit 11, a motion vector calculation unit 12, and a formation data generation unit 13. As described above, the formation data generation unit 10 calculates the average number of players and the average motion vector based on the player arrangement data for each preset divided region in the video section set by the user, Formation data including an average motion vector is generated.

  The number-of-players calculation means 11 reads the player arrangement data 127 from the storage unit 102, and in the video section set by the user, for each division area set in advance, the average number of players for each team (in the division area of the video section). The average number of players existing at the time corresponding to one player arrangement data) is calculated and output to the formation data generation means 13.

  The motion vector calculation means 12 reads the player arrangement data 127 from the storage unit 102, and in the video section set by the user, for each division area set in advance, the average motion vector for each team (in the division area of the video section). The motion vector of a representative player is calculated and output to the formation data generation means 13.

  The formation data generation means 13 receives the average number of players for each team in the divided area of the predetermined video section from the player number calculation means 11, and from the motion vector calculation means 12 for each team in the divided area of the predetermined video section. An average motion vector is input, formation data including the average number of players and the average motion vector is generated, and stored as formation data 128 in the storage unit 102.

(Formation data generator / processing)
Next, the process of the formation data generation unit 10 shown in FIG. 3 will be described in detail. FIG. 6 is a flowchart showing the processing of the formation data generation unit 10. The formation data generation unit 10 reads the player arrangement data 127 of the video data set (selected) by the user (video data viewed by the user) (step S601), and becomes a formation data generation unit according to the user's key operation. The unit time Tu time data is input (step S602). Thereby, formation data is generated every unit time Tu.

  FIG. 12 is a diagram for explaining the player arrangement data 127. The player arrangement data 127 is composed of data of one game, and a game ID (gameID) indicating an identifier for identifying a soccer game as video data, a team name (t1, t2), and a player arrangement of a plurality of records. Consists of data. The player arrangement data of one record is data for an image of a predetermined number of frames. The start frame No, the end frame No, the number of players of the team t1, the position coordinates and motion vectors for each player in the team t1, the number of players of the team t2, It consists of position coordinates and motion vectors for each player in team t2. In the example of FIG. 12, the player arrangement data of the first record is the number of players of team t1, the number of players of team t2, and the number of players of team t2 for the image of start frame No = 6488 and end frame No = 6545. First player position coordinates (51, 2) and motion vector (−8.097778, −9.498276), second player position coordinates (39, 85) and motion vector (−11.470154) in team t2. , -8.21049). The position coordinate indicates the position of the x and y coordinates when the field is represented by a plane, and the motion vector indicates the direction and magnitude of the motion when the field is represented by the x and y plane.

  Returning to FIG. 6, the formation data generation unit 10 divides the soccer field into Dx × Dy areas set in advance, and the divided areas (i, j) = (1, 1) to ( Dx, Dy) are set (step S603).

  FIG. 10 is a diagram for explaining an example of field division, and shows an example of division where Dx = Dy = 5. As shown in FIG. 10, the formation data generation unit 10 divides a soccer field into 25 regions, and sets divided regions (1, 1) (1, 2) to (5, 5) in the field. .

  Returning to FIG. 6, the formation data generating unit 10 sets Ts = 0 as the initial start time of the video section (step S604). Then, as will be described later, the formation data generation unit 10 performs the processing from step S605 to step S611 from the start to the end of the video, and the unit time Tu (the video section defined by the start time Ts and the end time Ts + Tu). The average number of players and the average motion vector are calculated for each divided region (i, j).

  The formation data generation unit 10 sets (i, j) = (1, 1) as the initial divided region (step S605). Then, as will be described later, the formation data generation unit 10 performs the processing from step S606 to step S609 for all the divided regions (i, j), and calculates the average number of players and the average motion vector.

ここで、平均選手数Ｎｐ（ｉ，ｊ）は、映像区間Ｔｓ〜Ｔｓ＋Ｔｕにおいて、分割領域（ｉ，ｊ）に存在していた、１つの選手配置データあたりの平均選手数を示す。Ｎｓｕは、映像区間Ｔｓ〜Ｔｓ＋Ｔｕに含まれる選手配置データの数を示す。ｎ_ｐ ^（ｋ）（ｉ，ｊ）は、映像区間Ｔｓ〜Ｔｓ＋Ｔｕに含まれるＮｓｕ個の選手配置データのうちのｋ番目の選手配置データにおける、領域（ｉ，ｊ）内の選手数を示す。 The number-of-players calculation means 11 of the formation data generation unit 10 extracts player placement data included in the video sections Ts to Ts + Tu from the player placement data 127, and calculates the average number of players Np (i, j) by the following formula. It calculates for every (step S606).

Here, the average number of players Np (i, j) indicates the average number of players per player arrangement data that existed in the divided area (i, j) in the video section Ts to Ts + Tu. Nsu indicates the number of player arrangement data included in the video section Ts to Ts + Tu. n _p ^(k) (i, j) indicates the number of players in the area (i, j) in the k-th player placement data among the Nsu player placement data included in the video section Ts to Ts + Tu.

  FIG. 9 is a diagram for explaining the Nsu player arrangement data included in the video sections Ts to Ts + Tu. The video section Ts to Ts + Tu (the video section from the start time Ts to the end time Tu) includes Nsu player placement data, and the Nsu player placement data included in this video section is the player. The numbers are extracted from the player arrangement data 127 by the number calculation means 11 and the motion vector calculation means 12.

ここで、平均動きベクトルＶｐ（ｉ，ｊ）は、映像区間Ｔｓ〜Ｔｓ＋Ｔｕの分割領域（ｉ，ｊ）に存在していた選手のうち、Ｍ方向動きヒストグラムにより度数（選手数）が最大となる方向の動きベクトルを有する選手について、その選手の平均動きベクトル（分割領域（ｉ，ｊ）における代表的な選手の動きベクトル）を示す。ｎ_ＰＭＶは、映像区間Ｔｓ〜Ｔｓ＋Ｔｕに含まれるＮｓｕ個の選手配置データにおける分割領域（ｉ，ｊ）の動きベクトルから求めた、Ｍ方向ヒストグラムの最大度数（最大選手数）を示す。Ｖ_ＰＭＶは、ｎ_ＰＭＶ（最大度数）の方向に属する選手の動きベクトルについて、要素毎に合計した動きベクトルを示す。 Returning to FIG. 6, the motion vector calculation means 12 extracts player placement data included in the video section Ts to Ts + Tu from the player placement data 127, and calculates the average motion vector Vp (i, j) by the following formula. It is calculated every time (step S607).

Here, the average motion vector Vp (i, j) has the highest frequency (number of players) by the M-direction motion histogram among the players that existed in the divided region (i, j) of the video section Ts to Ts + Tu. For a player having a direction motion vector, an average motion vector of the player (a motion vector of a representative player in the divided region (i, j)) is shown. n _PMV indicates the maximum frequency (maximum number of players) of the M-direction histogram obtained from the motion vector of the divided region (i, j) in the Nsu player arrangement data included in the video section Ts to Ts + Tu. V _PMV represents a motion vector obtained by summing up each element regarding the motion vector of the player belonging to the direction of n _PMV (maximum frequency).

Specifically, the motion vector calculation means 12 uses the Nsu player arrangement data included in the video sections Ts to Ts + Tu, and calculates the M-direction histogram (for example, M =) from the motion vector of the divided region (i, j). A graph showing the relationship between direction and frequency as 6 directions is generated, and the maximum frequency among the frequencies (number of players) corresponding to each direction is obtained as n _PMV . Further, the motion vector calculation means 12 extracts the motion vector of the player belonging to the direction of the maximum frequency n _PMV , _adds up the elements of each motion vector, and obtains the total motion vector V _PMV . Then, the motion vector calculation means 12 divides the total motion vector V _PMV by the maximum frequency n _PMV by the above formula to obtain the average motion vector Vp (i, j).

  In this way, for the divided region (i, j) in the video section Ts to Ts + Tu, the average number of players Np (i, j) is calculated for each team by the number-of-players calculation means 11, and the average motion is calculated by the motion vector calculation means 12. A vector Vp (i, j) is calculated. The average number of players Np (i, j) and the average motion vector Vp (i, j) for each video section Ts to Ts + Tu are stored in the storage unit 102 (not shown).

  The formation data generation unit 10 determines whether or not the average number of players Np (i, j) and the average motion vector Vp (i, j) have been calculated for all the divided regions (i, j) (step S608).

  When the formation data generation unit 10 determines in step S608 that the average number of players Np (i, j) and the average motion vector Vp (i, j) are not calculated for all the divided regions (i, j) ( Step S608: N), i = i + 1 or j = j + 1 is calculated, a new divided area (i, j) is set (Step S609), the process proceeds to Step S606, and a new divided area (i, j) is set. Process.

  On the other hand, when the formation data generation unit 10 determines in step S608 that the average number of players Np (i, j) and the average motion vector Vp (i, j) have been calculated for all the divided regions (i, j) ( Step S608: Y), Ts = Ts + Tu is calculated, the video start time Ts is advanced by Tu, and a new start time Ts is set in order to define a new video section (step S610). Then, the formation data generation unit 10 determines whether or not the new start time Ts exceeds the video end time (step S611).

  When the formation data generation unit 10 determines in step S611 that the new start time Ts does not exceed the end time of the video (step S611: N), the formation data generation unit 10 proceeds to step S605, and the player number calculation unit 11 and the motion vector. The calculating means 12 calculates the average number of players Np (i, j) and the average motion vector Vp (i, j) for each divided region (i, j) in the new video section Ts to Ts + Tu.

  On the other hand, when the formation data generation unit 10 determines in step S611 that the new start time Ts exceeds the end time of the video (step S611: Y), the formation data generation unit 13 determines the number-of-players calculation unit 11. The average number of players Np (i, j) for each divided area (i, j) in the video section Ts to Ts + Tu is input from the motion vector calculation means 12 and the divided area (i, j) in the video section Ts to Ts + Tu. The average motion vector Vp (i, j) for each is inputted, formation data is generated, and the formation data 128 is stored in the storage device 103 via the storage unit 102 (step S612). Thus, the process ends. In this case, the average number of players Np (i, j) and the average motion vector Vp (i, j) for each video section Ts to Ts + Tu are read from the storage unit 102.

  FIG. 11 is a diagram illustrating a process of generating formation data from player arrangement data. As shown in FIG. 11A, when the unit time Tu of the video section is 5 minutes, formation data for each team (team A, team B) is generated from Nsu player arrangement data included in 5 minutes. The Also, as shown in FIG. 11B, when the unit time Tu of the video section is 15 minutes, formation data for each team (team A, team B) is obtained from the Nsu player placement data included in 15 minutes. Generated. That is, the formation data is generated for each unit Tu and for each team.

  FIG. 13 is a diagram for explaining the formation data 128. The formation data 128 is composed of data of one game, and includes a game ID (gameID) indicating an identifier for specifying a soccer game video as video data, a start time Ts and an end time Ts + Tu (time (min)) of the video section. ), Team name, average number of players Np (i, j) and average motion vector Vp (i, j) for each divided area (i, j) as one record, and corresponding to a plurality of video sections and each team Consists of multiple records. In the example of FIG. 13, a in the first record is game video ID = 62, video sections 0 to 5, team = average player number Np (i, j) and average motion for each divided area (i, j) in Japan. It is composed of vector Vp (i, j), and b in the second record is the same number of players per division area (i, j) in the same game video ID = 62, the same video section 0-5, and team = Brazil Np (i, j) and the average motion vector Vp (i, j), and c in the third record is the same as game video ID = 62, video section 5-10, team = divided area in Japan It is composed of an average number of players Np (i, j) and an average motion vector Vp (i, j) for each (i, j). Such data in record units exists for each team and for each video section up to the video end time.

  As described above, according to the image processing device 1 according to the embodiment of the present invention, the formation data generation unit 10 determines the divided regions (i, j) in the predetermined video sections Ts to Ts + Tu based on the player arrangement data 127. Then, the average number of players Np (i, j) of the players existing in the divided area (i, j) of the video section Ts to Ts + Tu is calculated, and the divided area (i, j) of the video section Ts to Ts + Tu is calculated. The average motion vector Vp (i, j) of a representative player at is calculated and formation data is generated. By displaying this formation data on the display 109, the user can grasp the formation of the player who globally captures the entire field. In addition to providing sports videos to users who have a strong interest in sports in a service that uses a platform that can deliver live sports videos or archives videos via WEB, it also analyzes team tactics, etc. It is possible to present useful information.

[Graphical presentation section]
Next, the graphical presentation unit 20 shown in FIG. 2 will be described in detail. FIG. 4 is a block diagram showing a configuration of the graphical presentation unit 20. The graphical presentation unit 20 includes a player number display calculation unit 21, a motion vector display calculation unit 22, and a graphical presentation unit (display unit) 23. As described above, the graphical presentation unit 20 displays the video data on the screen and displays the formation data for each team on the divided area in the predetermined video section according to the time synchronized with the video data. Also, the similarity calculation result is input from the similarity calculation unit 30, and video data and formation data with high similarity are displayed on the screen.

  The player number display calculation means 21 reads the formation data 128 from the storage unit 102, and displays the divided area (i, j) in the video section Ts to Ts + Tu from the average player number Np (i, j) according to the number. In order to digitize the form, the player number display data is calculated for each team and output to the graphical presentation means 23.

  The motion vector display calculation unit 22 reads the formation data 128 from the storage unit 102, and for the divided region (i, j) in the video section Ts to Ts + Tu, the motion vector display calculation unit 22 corresponds to the motion vector from the average motion vector Vp (i, j). In order to digitize the display form, motion vector display data consisting of direction and size is calculated for each team and output to the graphical presentation means 23.

  The graphical presentation unit 23 reads out the video data 126 from the storage unit 102 and displays the video data 126 on the display unit 109, and also displays the number-of-players display data of the divided area (i, j) in the video section Ts to Ts + Tu from the player number display calculation unit 21. Is input from the motion vector display calculation means 22 and the motion vector display data of the divided area (i, j) in the video section Ts to Ts + Tu is input, and the player number display data and the motion vector display at the time synchronized with the video data 126 are input. The data is displayed graphically on the display 109. Further, the graphical presentation unit 23 receives the similarity calculation result from the similarity calculation unit 30, and the video calculated by the video data 126 with high similarity, the number-of-players display calculation unit 21, and the motion vector display calculation unit 22. The player number display data and the motion vector display data of the data 126 are graphically displayed on the display 109.

(Graphical presentation / processing)
Next, the process of the graphical presentation unit 20 shown in FIG. 4 will be described in detail. FIG. 7 is a flowchart showing the processing of the graphical presentation unit 20. The graphical presentation unit 20 reads out the video data 126 viewed by the user from the storage unit 102 and displays it on the screen, and reads out the formation data 128 of the video data 126 from the storage unit 102 (step S701).

  The graphical presentation unit 20 sets an initial start time Ts = 0 synchronized with the video data 126 (step S702), and the average number of players Np (i, j) in the video section Ts to Ts + Tu (0 to 5) from the formation data 128. Then, the average motion vector Vp (i, j) is extracted (step S703). In the example of the formation data 128 shown in FIG. 13, the video sections corresponding to the start time of the video data 126 are the video sections 0 to 5, and when team = Japan, the average number of players Np (i , J) and the average motion vector Vp (i, j) are extracted, and when the team = Brazil, the average number of players Np (i, j) and the average motion vector Vp (i, j) are extracted from the second record b. Is done. Then, as will be described later, the graphical presentation unit 20 performs processing from step S703 to step S711 from the start to the end of the video, and the video time interval Tu (the video determined by the start time Ts and the end time Ts + Tu). The average number of players Np (i, j) and the average motion vector Vp (i, j) are graphically displayed on the screen for each divided area (i, j) in (section).

  The graphical presentation unit 20 sets (i, j) = (1, 1) as the initial divided region (step S704). And the graphical presentation part 20 performs the process from step S705 to step S708 about all the division | segmentation areas (i, j) so that it may mention later, and it displays the player for screen display from the average number of players Np (i, j). Numerical display data is calculated, and motion vector display data for screen display is calculated from the average motion vector Vp (i, j).

The player number display calculation means 21 of the graphical presentation unit 20 displays the RGB value (255, 255, C (i, j)) indicating the color when the average player number Np (i, j) is displayed on the screen in a blue density color. ), The number-of-players display data C (i, j) is calculated for each team by the following formula (step S705).
C (i, j) = 255-255 × (Np (i, j) / Nmax)
Here, when C (i, j) becomes smaller than 0, it is replaced with 0. Nmax represents a preset constant.

When the average motion vector Vp (i, j) is displayed on the screen by the arrow, the motion vector display calculation means 22 sets the unit vector of the average motion vector Vp (i, j) as the direction of the arrow, and uses the following formula: The length of the arrow is calculated, and motion vector display data including the direction of the arrow and the length of the arrow is obtained for each team (step S706).
Arrow length = | Vp (i, j) | × R _fld
Here, R _fld indicates a scale ratio between the field assumed image to be graphically displayed and the coordinate space field of the original formation data 128. This scale ratio is obtained using the scale of the field assumption image to be graphically displayed as the numerator and the scale of the coordinate space field of the formation data 128 as the denominator.

  In this way, with respect to the divided region (i, j) in the video section Ts to Ts + Tu, the player number display calculation means 21 calculates the player number display data of the average player number Np (i, j) for each team, and moves The vector display calculation means 22 calculates motion vector display data of the average motion vector Vp (i, j) for each team. The number-of-players display data and the motion vector display data of the divided area (i, j) in the video section Ts to Ts + Tu are stored in the storage unit 102 (not shown).

  The graphical presentation unit 20 determines whether or not the player number display data and the motion vector display data have been calculated for all the divided regions (i, j) (step S707).

  When the graphical presentation unit 20 determines in step S707 that the player number display data and the motion vector display data are not calculated for all the divided regions (i, j) (step S707: N), i = i + 1 or j = J + 1 is calculated, a new divided area (i, j) is set (step S708), the process proceeds to step S705, and the new divided area (i, j) is processed.

  On the other hand, when the graphical presentation unit 20 determines in step S707 that the player number display data and the motion vector display data have been calculated for all the divided regions (i, j) (step S707: Y), the graphical presentation unit 23 Then, the color of the divided area (i, j) in the video section Ts to Ts + Tu is displayed in a blue density color based on the player number display data of the divided area (i, j) (step S709). As a result, the user can see that the area displayed in dark blue has a large number of players, and the area displayed in light blue represents the number of players from the color of the divided area (i, j) in the field displayed on the display 109. I can grasp that there are few. Further, the graphical presentation means 23 displays the arrows in the divided area (i, j) in the video section Ts to Ts + Tu in the direction and length based on the motion vector display data of the divided area (i, j) ( Step S709). Thereby, the user can grasp the movement of the representative player from the direction and length of the arrow in the divided area (i, j) in the field displayed on the display 109. In this case, the player number display data and motion vector display data of the divided region (i, j) in the video section Ts to Ts + Tu are read from the storage unit 102 (not shown).

  FIG. 14 is a diagram illustrating an example of a graphical presentation screen for formation data. As shown in FIG. 14, the graphical presentation means 23 generates a field presentation image for graphical presentation to be displayed on the display 109, and divides the divided areas (i, j) = (1, 1) to (Dx, Dy). Set the corresponding area. Then, the graphical presentation means 23 displays the average number of players Np (i, j) of the players that existed in the divided area (i, j) of the video section Ts to Ts + Tu for each team in a color corresponding to the number. Color. Further, the graphical presentation means 23 draws the average motion vector Vp (i, j) of a representative player in the divided area (i, j) of the video section Ts to Ts + Tu for each team in the direction and length of the arrow. To do.

  Returning to FIG. 7, the graphical presentation unit 20 updates the start time Ts = Ts + Tu synchronized with the video data 126, and sets a new start time Ts (step S710). Then, the graphical presentation unit 20 determines whether or not the new start time Ts that defines a new video segment exceeds the video end time (step S711).

  If the graphical presentation unit 20 determines in step S711 that the new start time Ts does not exceed the video end time (step S711: N), the process proceeds to step S703, and for the new video section Ts to Ts + Tu, For each divided area (i, j), the number-of-players display data and the motion vector display data are calculated and displayed on the screen.

  On the other hand, if the graphical presentation unit 20 determines in step S711 that the new start time Ts exceeds the end time of the video (step S711: Y), the process ends.

  The graphical presentation means 23 receives the similarity calculation result from the similarity calculator 30 and displays video data and formation data with high similarity on the screen. This process will be described later.

  As described above, according to the image processing apparatus 1 according to the embodiment of the present invention, the graphical presentation unit 20 displays the video data 126 on the screen and divides the video sections Ts to Ts + Tu at the time synchronized with the video data 126. For each region (i, j), the number-of-players display data of the average number of players Np (i, j) included in the formation data 128 and the motion vector display data of the average motion vector Vp (i, j) are calculated graphically. Display on the screen. Thereby, the user can grasp | ascertain the formation of the player who caught the whole field globally. In addition to providing sports videos to users who have a strong interest in sports in a service that uses a platform that can deliver live sports videos or archives videos via WEB, it also analyzes team tactics, etc. It is possible to present useful information.

[Similarity calculator]
Next, the similarity calculation unit 30 shown in FIG. 2 will be described in detail. FIG. 5 is a block diagram illustrating a configuration of the similarity calculation unit 30. The similarity calculation unit 30 includes a player number similarity calculation unit 31, a motion vector similarity calculation unit 32, and a similarity calculation unit 33. As described above, the similarity calculation unit 30 calculates the number-of-players similarity and the motion vector similarity between the formation data to be a reference (search query) and the formation data to be searched for similarity calculation. Calculate the similarity between the formation data.

  The number-of-players similarity calculation means 31 reads the formation data 128 (FDQ) serving as a reference (search query) and the formation data 128 (FDT) serving as a search target for similarity calculation from the storage unit 102, and predetermined data in the formation data FDQ. The player number similarity Spn (Q, T) is calculated based on the average number of players Np (i, j) in the video section and the average number of players Np (i, j) in the predetermined video section in the formation data FDT. To do. Here, the predetermined video section indicates one of the video sections included in the formation data FDQ and FDT, and may be a section set in advance by the user. In addition, the time length of the predetermined video section in the formation data FDQ may be different from the time length of the predetermined video section in the formation data FDT. Then, the player number similarity calculating unit 31 outputs the player number similarity Spn (Q, T) to the similarity calculating unit 33.

  Similar to the number-of-players similarity calculation unit 31, the motion vector similarity calculation unit 32 reads the formation data FDQ and the formation data FDT from the storage unit 102, and averages the motion vector Vp (i, i, in a predetermined video section in the formation data FDQ. j) and the average motion vector Vp (i, j) of a predetermined video section in the formation data FDT, the motion vector similarity Smv (Q, T) is calculated. Then, the motion vector similarity calculation unit 32 outputs the motion vector similarity Smv (Q, T) to the similarity calculation unit 33.

  The similarity calculation means 33 inputs the player number similarity Spn (Q, T) from the player number similarity calculation means 31 and the motion vector similarity Smv (Q, T) from the motion vector similarity calculation means 32. Based on the number-of-players similarity Spn (Q, T) and the motion vector similarity Smv (Q, T), the similarity between the formation data FDQ of the predetermined video section and the formation data FDT of the predetermined video section S (Q, T) is calculated, the similarity S (Q, T) is displayed on the display 109, information for specifying the formation data FDQ and its video section, and the formation data FDT and its video section are specified. Information and the similarity S (Q, T) are output to the graphical presentation unit 20.

  The similarity calculation unit 30 may input the formation data FDQ and information for specifying the video section from the graphical presentation unit 20. In this case, the player number similarity calculating unit 31 and the motion vector similarity calculating unit 32 read the formation data FDQ from the storage unit 102 based on these pieces of information, and determine the player number similarity Spn (Q, T) and the motion vector. The similarity Smv (Q, T) is calculated. Further, the similarity calculation means 33 does not output the formation data FDQ and the information for specifying the video section to the graphical presentation unit 20, and the information for specifying the formation data FDT and the video section, and the similarity S Only (Q, T) is output to the graphical presentation unit 20.

(Similarity calculator / processing)
Next, the process of the similarity calculation unit 30 shown in FIG. 5 will be described in detail. FIG. 8 is a flowchart showing the processing of the similarity calculation unit 30. The similarity calculation unit 30 reads from the storage unit 102 formation data 128 (FDQ) serving as a reference (search query) and formation data 128 (FDT) serving as a search target for similarity calculation (step S801).

ここで、Ｃｐｎは、選手数類似度Ｓｐｎ（Ｑ，Ｔ）を０〜１の範囲の値にするために、予め設定された定数である。Ｄｘ，Ｄｙは、分割領域（ｉ，ｊ）におけるｉ，ｊの最大値を示す。 Players number similarity calculation unit 31, from the formation data FDQ, extracts the average number players given image segment ^{Np (Q) (i, j} ), from the formation data FDT, the average number players given image segment Np ^{( T)} Extract (i, j). Then, the number-of-players similarity calculation means 31 calculates the difference in the average number of players Np (i, j) for each divided region (i, j) (step S802), and the formation data FDQ and the formation data are calculated according to the following equations. The number-of-players similarity Spn (Q, T) indicating the similarity regarding the player distribution with the FDT is calculated (step S803).

Here, Cpn is a constant set in advance in order to set the number-of-players similarity Spn (Q, T) to a value in the range of 0 to 1. Dx and Dy indicate the maximum values of i and j in the divided area (i, j).

Similar to the player number similarity calculation means 31, the motion vector similarity calculation means 32 extracts the average motion vector Vp ^(Q) (i, j) of a predetermined video section from the formation data FDQ, and uses the formation data FDT. Then, an average motion vector Vp ^(T) (i, j) of a predetermined video section is extracted. Then, the motion vector similarity calculating unit 32 calculates a difference between the average motion vectors Vp (i, j) for each divided region (i, j) (step S804), and the formation data FDQ and the formation data are calculated according to the following equations. A motion vector similarity Smv (Q, T) indicating the similarity with respect to the movement of the player with the FDT is calculated (step S805).

また、ｄ_Ｅ（Ｖｐ^（Ｑ）（ｉ，ｊ），Ｖｐ^（Ｔ）（ｉ，ｊ））は、平均動きベクトルＶｐ^（Ｑ）（ｉ，ｊ）と平均動きベクトルＶｐ^（Ｔ）（ｉ，ｊ）との間のユークリッド距離を示し、Ｃｍｖは、動きベクトル類似度Ｓｍｖ（Ｑ，Ｔ）を０〜１の範囲の値にするために、予め設定された定数を示す。 Here, w _ij indicates a weight for strongly reflecting the motion vector of the divided area (i, j) in the formation data FDQ where the motion of the player is large in the motion vector similarity Smv (Q, T). Calculated by the formula.

Also, d _E (Vp ^(Q) (i, j), Vp ^(T) (i, j)) is an average motion vector Vp ^(Q) (i, j) and an average motion vector Vp ^(T) (i, j). jm) represents a Euclidean distance from the j), and Cmv represents a constant set in advance in order to set the motion vector similarity Smv (Q, T) to a value in the range of 0 to 1.

  The number-of-players similarity Spn (Q, T) and the motion vector similarity Smv (Q, T) between the formation data FDQ of the predetermined video section and the formation data FDT of the predetermined video section are stored in the storage unit 102. (Not shown).

Based on the number-of-players similarity Spn (Q, T), the motion vector similarity Smv (Q, T), and the preset weight λ, the similarity calculation means 33 calculates the similarity S ′ ( Q, T) is calculated.
S ′ (Q, T) = λSpn (Q, T) + (1−λ) Smv (Q, T)
Here, λ is a weight (value in the range of 0 to 1) that determines the degree of reflection of the number-of-players similarity Spn (Q, T) and the motion vector similarity Smv (Q, T) with respect to the similarity S (Q, T). ).

Then, the similarity calculation unit 33 calculates the similarity S (Q, T) between the formation data FDQ for the predetermined video section and the formation data FDT for the predetermined video section (step S806), and the process is terminated. To do.
S (Q, T) = MAX (S ′ (Q, T), S ′ (Q, T ^t ))
Here, MAX (A, B) indicates the larger value of A and B. T ^t is the formation data in which the arrangement of the divided areas (i, j), the average number of players Np (i, j) and the average motion vector Vp (i, j) of the formation data FDT are point-symmetric at the center of the field. Indicates. When calculating the similarity S (Q, T), the similarity S ′ (Q, T ^t ) between the formation data FDQ and the formation data symmetric with respect to the formation data FDT at the center of the field is also obtained. The calculation is to take into account the attack in the opposite direction (the concept of “left side” and “right side” is also “point-symmetric” because the top, bottom, left and right are reversed).

  Further, the similarity calculation unit 30 changes the video section in the formation data FDT, and the similarity S (Q, Q) between the formation data FDQ of the predetermined video section and the formation data FDT of the new video section after the change. T) is calculated. For example, when Tu = 5, the similarity calculation unit 30 first calculates the formation data FDQ of a predetermined video section and the formation data FDT of the video sections 0 to 5 in the video sections 0 to 5 of the formation data FDT. The similarity S (Q, T) between the formation data FDQ of the predetermined video section and the formation data FDT of the video sections 5 to 10 is calculated. Then, the degree of similarity S (Q, T) between the formation data FDT of all the video sections Ts to Ts + Tu (5) is calculated. Similarly, for the formation data FDT of the other video data 126, the similarity S (Q, T) between the formation data FDQ of a predetermined video section and the formation data FDT of all the video sections is calculated.

  As described above, the process of the similarity calculation unit 30 allows the user to acquire information on the formation of another team similar to the predetermined video section and the predetermined team formation in the video data 126. Specifically, the user acquires a plurality of other similar formation data FDT together with the similarity S (Q, T) for the video section and team formation data FDQ set for a certain video data 126. Can do.

  As described above, according to the image processing apparatus 1 according to the embodiment of the present invention, the similarity calculation unit 30 includes the formation data FDQ to be a reference (search query) and the formation data FDT to be searched for similarity calculation. The player's similarity Spn (Q, T) and motion vector similarity Smv (Q, T) are calculated, and the similarity S (Q, T) between the two formation data is calculated. This makes it possible to search for a scene similar to the video specified by the user, based on the similarity of the formation data of the players who globally capture the entire field. In addition to providing sports videos to users who have a strong interest in sports in a service that uses a platform that can deliver live sports videos or archives videos via WEB, it also analyzes team tactics, etc. It is possible to present useful information.

[Screen display example]
Next, an example of a screen displayed on the display unit 109 of the image processing apparatus 1 illustrated in FIG. 1 will be described. FIG. 15 is a diagram illustrating an example of a screen displayed on the display device 109. The upper part of the screen is an area in which video data or the like (reference video) set (selected) by the user is displayed. The lower part of the screen is a video of a formation similar to the formation of the video data displayed at the upper part of the screen. This is an area where data (search video) is displayed.

  The graphical presentation unit 20 sets the video data 126 by the user's key operation, sets the team (Switzerland) in the area b, sets the time unit (5 minutes) in the area c, When the time (first half 25-30 minutes) is set in the area and these setting data are input, the graphical presentation means 23 extracts frames corresponding to the video sections 25 to 30 from the video data 126, and the area a. To display. Then, the player number display calculation means 21 extracts the average player number Np (i, j) of the Swiss team in the video sections 25 to 30 from the formation data 128 of the video data 126, and calculates the player number display data. . The motion vector display calculation means 22 extracts the average motion vector Vp (i, j) of the Swiss team in the video sections 25 to 30 from the formation data 128 of the video data 126, and calculates the motion vector display data. . Then, the graphical presentation means 23 represents the number-of-players display data in color for each divided area (i, j), represents the motion vector display data for each divided area (i, j), and represents in the video sections 25-30. Displayed in area e as Swiss team formation data.

  The formation data generation unit 10 reads the player arrangement data 127 of the video data 126 set by the user, generates the formation data 128 based on the player arrangement data 127 by the process shown in FIG. 20 may calculate the number-of-players display data and the motion vector display data using the formation data 128 generated by the formation data generation unit 10 and display it on the screen.

  Accordingly, the user can grasp the entire field globally and grasp the formation of Swiss players in the time zone while viewing the video data 126 of the first half 25 to 30 minutes set by key operation. . Further, the user can analyze not only the sports video but also the team tactics.

  When the video data 126 to be searched for similarity calculation is set by the user's key operation and the start of search processing is input to the graphical presentation unit 20, the graphical presentation unit 23 sets the search target for similarity calculation as the search target. Information for specifying the video data 126 to be used, information for specifying the video data 126 to be displayed displayed at the top of the screen, and information for specifying the team (Switzerland) and the video section (25 to 30) Is output to the similarity calculation unit 30. The similarity calculation unit 30 inputs these pieces of information from the graphical presentation unit 20.

  The player number similarity calculation means 31 reads the average number of players Np (i, j) of the Swiss team in the video sections 25 to 30 in the formation data 128 (FDQ) of the reference video data 126, and the similarity Of the formation data 128 (FDT) of the video data 126 to be searched for calculation, the average number of players Np (i, j) of a predetermined team in the video sections 0 to 5 is read out. And the number-of-players similarity calculation means 31 calculates the number of players similarity. Further, the motion vector similarity calculating means 32 reads the average motion vector Vp (i, j) of the Swiss team in the video sections 25 to 30 in the formation data FDQ, and the video section 0 in the formation data FDT. The average motion vector Vp (i, j) of the predetermined team in ˜5 is read out. Then, the motion vector similarity calculating unit 32 calculates a motion vector similarity.

  Then, the similarity calculation unit 33 calculates the similarity S (Q, T) between the formation data FDQ and the formation data FDT based on the number of players similarity and the motion vector similarity. The similarity calculation unit 30 also calculates the similarity S (Q, T) for other video sections in the reference video data 126 and for other reference video data 126. That is, the similarity S (Q, T) of all the video sections in the reference video data 126 is calculated. The similarity calculation unit 33 sets the similarity S (Q, T), information for specifying the video data 126 to be searched for similarity calculation, and information for specifying the team and the video section as a set. The degree search result is output to the graphical presentation means 23 of the graphical presentation unit 20.

  When the graphical presentation unit 23 of the graphical presentation unit 20 inputs the similarity search result from the similarity calculation unit 33 of the similarity calculation unit 30, the similarity calculation is performed in descending order of the similarity S (Q, T). The information such as the video data 126, the team, and the video section to be searched is displayed in the area h.

  When one video data 126 is selected from the plurality of video data 126 displayed in the area h by the user's operation, the graphical presentation unit 23 displays the selected video data 126 in the area f. Further, the graphical presentation means 23 causes the player number display calculation means 21 and the motion vector display calculation means 22 to calculate the player number display data and the motion vector display data for the selected team and video section of the video data 126, and the player. The number display data is represented by a color for each divided area (i, j), and the motion vector display data is represented by an arrow for each divided area (i, j) and displayed in the area g.

  Accordingly, the user can grasp the formation while viewing the video data 126 of the first half 25 to 30 minutes set by the key operation, and further, the video data 126 of another formation similar to the formation. Can be viewed, and other similar formations can be grasped. Thus, the user can globally view the entire field and compare similar formations. That is, the user can analyze not only watching sports videos but also comparing team tactics with those of other teams.

  The present invention has been described with reference to the embodiment. However, the present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the technical idea thereof. For example, as shown in FIG. 14, when the formation data is graphically displayed, the graphical presentation unit 20 is colored with a color corresponding to the average number of players Np (i, j), and the average motion vector Vp (i, j ) In accordance with the direction and length of the arrow. However, the present invention is not limited to the display of the formation data, but may be displayed in another form. Good. For example, for each divided area (i, j), a bar graph is displayed according to the average number of players Np (i, j), or a gradation is displayed according to the direction and size of the average motion vector Vp (i, j). You may make it do.

  Moreover, although the said embodiment demonstrated the soccer video data as an example as the video data 126, this invention is not limited to soccer video data, The video data 126 is video data of other sports. Also good.

DESCRIPTION OF SYMBOLS 1 Image processing apparatus 10 Formation data generation part 11 Number of players calculation means 12 Motion vector calculation means 13 Formation data generation means 20 Graphical presentation part 21 Number of players display calculation means 22 Motion vector display calculation means 23 Graphical presentation means 30 Similarity degree calculation part 31 Number-of-players similarity calculation means 32 Motion vector similarity calculation means 33 Similarity calculation means 101 CPU
102 storage unit 103 storage device 104 network interface 105 operation / input interface 106 display interface 107 system bus 108 operation unit 109 display unit 110 control unit 120 operating system 121 formation data generation program 122 graphical presentation program 123 similarity calculation program 126 video data 127 Player placement data 128 Formation data

Claims (4)

A sports image in which a sports game is photographed, and storage means for storing player arrangement data including data on the positions and movements of the players at predetermined intervals in the sports video are provided, and the game is based on the player arrangement data. In the image processing apparatus that generates formation data indicating the number of players and the movement of the players for each divided region obtained by dividing the field to be divided into a predetermined number,
Player number reading means for reading player arrangement data from the storage means, and calculating the number of players for each of the divided areas for each predetermined video section based on the position data of the players included in the player arrangement data;
Motion vector calculation means for reading player arrangement data from the storage means and calculating player motion vectors for each of the divided regions for each of the predetermined video sections based on player movement data included in the player arrangement data. When,
Formation data including the number of players and the motion vector for each divided region for each predetermined video section based on the number of players calculated by the number-of-players calculation means and the motion vector calculated by the motion vector calculation means. Formation data generation means for generating and storing in the storage means;
An image processing apparatus comprising: The formation data is read from the storage means storing the formation data of claim 1, and the player number display data corresponding to the number of players included in the formation data is calculated for each of the divided areas of the predetermined video section. Player number display calculation means,
Motion vector display calculating means for calculating motion vector display data corresponding to a motion vector included in the formation data for each of the divided regions of the video section;
The number-of-players display data calculated by the number-of-players display calculation means and the motion vector display data calculated by the motion vector display calculation means are displayed in a predetermined form for each of the divided areas into which the field is divided. Display means;
An image processing apparatus comprising: The image processing apparatus according to claim 1.
Furthermore, from the storage means, the first formation data for the sports video for the first video section, and the sports video for which another game of the same sport as the sports video is shot for the second video section. The second formation data is read out, the number of players for each of the divided areas in the first video section included in the first formation data, and the second video section in the second video data included in the second formation data. A player number similarity calculating means for calculating a difference between the number of players for each divided region and calculating a player number similarity indicating a degree of similarity between the number of players between both formation data;
A motion vector for each of the divided regions in the first video section included in the first formation data, and a motion vector for each of the divided regions in the second video section included in the second formation data. A motion vector similarity calculating means for calculating a motion vector similarity indicating a degree of similarity of motion vectors between both formation data,
Based on the player number similarity calculated by the player number similarity calculating means and the motion vector similarity calculated by the motion vector similarity calculating means, first formation data in the first video section; Similarity calculating means for calculating the similarity between the second formation data in the second video section;
An image processing apparatus comprising:   An image processing program for causing a computer to function as the image processing apparatus according to any one of claims 1 to 3.

Images