WO2020235339A1 - Play analyzing device, and play analyzing method - Google Patents

Play analyzing device, and play analyzing method Download PDF

Info

Publication number
WO2020235339A1
WO2020235339A1 PCT/JP2020/018462 JP2020018462W WO2020235339A1 WO 2020235339 A1 WO2020235339 A1 WO 2020235339A1 JP 2020018462 W JP2020018462 W JP 2020018462W WO 2020235339 A1 WO2020235339 A1 WO 2020235339A1
Authority
WO
WIPO (PCT)
Prior art keywords
rally
play
section
information
ball
Prior art date
Application number
PCT/JP2020/018462
Other languages
French (fr)
Japanese (ja)
Inventor
優麻 片山
純子 上田
井村　康治
Original Assignee
パナソニックＩｐマネジメント株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by パナソニックＩｐマネジメント株式会社 filed Critical パナソニックＩｐマネジメント株式会社
Publication of WO2020235339A1 publication Critical patent/WO2020235339A1/en

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B69/00—Training appliances or apparatus for special sports
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B71/00—Games or sports accessories not covered in groups A63B1/00 - A63B69/00
    • A63B71/06—Indicating or scoring devices for games or players, or for other sports activities
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion

Definitions

  • This disclosure relates to a sports play analyzer and a play analysis method.
  • Data Volley has been known as volleyball analysis software. Data Volley is used by analysts who are familiar with the software, who, based on subjective judgment, enter data into Data Volley that indicates the status of the players on the team.
  • Patent Document 1 discloses a scouting system with improved operability of analysts. Specifically, in Patent Document 1, the position information of each player is acquired by using image analysis, and the input player situation, each position information of each player, and a still image of a desired scene are associated with each other. It is disclosed that a database is constructed in a storage device to display each position of each player, a still image of a desired scene, and a player situation. According to the present invention, it is possible to solve the problem of Data Volley software that training of a scorer (analyst) is required and the data accuracy is low because the position information of each player is input by visual subjective judgment.
  • the conventional volleyball analysis software sequentially captures the images of the receive scenes for the serve, inputs the skill evaluation of each player, and aggregates and displays the number of receive times and the receive success rate of each player.
  • the user team manager, etc. analyzes the strengths and weaknesses of the team by looking at the aggregated results.
  • An object of the present disclosure is to provide a play analyzer and a method capable of easily acquiring a video of a rally section for each score from a play video.
  • the play analysis device is a play analysis device that analyzes the play of a ball game, and includes a referee motion detection unit that detects an motion pattern of a referee of the ball game using a play video, and the play. It is provided with a rally section determination unit that determines the start of the rally section based on the image and determines the end of the rally section based on the operation pattern of the referee.
  • the play analysis method is a play analysis method for analyzing the play of a ball game, in which the motion pattern of the referee of the ball game is detected using the play video, and the rally section is based on the play video. The start is determined, and the end of the rally section is determined based on the movement pattern of the referee.
  • the video of the rally section for each score can be easily obtained from the play video.
  • the play analysis system 1 is a system that analyzes a play video of a ball game and reproduces a play video based on the analysis result.
  • the moving body used in the ball game is typically a ball, but may be a shuttle or the like used for badminton.
  • volleyball which is one of the ball games, will be described as an example.
  • the play analysis system 1 can be applied to all ball games such as table tennis, tennis, beach volleyball, and badminton. That is, the play analysis system 1 is a ball game in which points or wins or losses are determined by actions by a team or an individual composed of a plurality of people, and can be applied to any ball game in which a mobile rally is performed. Is.
  • the play analysis system 1 includes a plurality of cameras 3 (3A, 3B, 3C, 3D), a display device 4, an input device 5, and a play analysis device 100.
  • the plurality of cameras 3 are installed at different positions. For example, as shown in FIG. 1, each camera 3 is installed at a position where the court 10 can be photographed from a high place with different viewpoints (angles of view). Although the number of cameras 3 is four in FIG. 1, the present embodiment is not limited to this, and the number of cameras 3 may be any number as long as it is two or more. By using two or more cameras 3, the three-dimensional position of the ball can be calculated.
  • Each camera 3 is communicably connected to the play analyzer 100 via wire or wireless. Each camera 3 captures the situation during the ball game and generates a captured image. Then, each camera 3 generates a moving image frame 201 (see FIG. 2) from a plurality of captured images. Then, each camera 3 transmits the moving image frame 201 to the play analyzer 100.
  • the moving image frame 201 has a plurality of captured images, for example, MP4, H.A. 264, H. It may be compressed based on a standard such as 265 or Motion JPEG.
  • the display device 4 is communicably connected to the play analysis device 100 via wire or wireless, and displays an image output from the play analysis device 100.
  • the display device 4 is, for example, a liquid crystal display or an organic EL display.
  • the input device 5 is communicably connected to the play analysis device 100 via wire or wireless, receives an operation from the user, and transmits the operation information to the play analysis device 100.
  • the input device 5 is, for example, a keyboard, a mouse, a microphone and / or a touch panel.
  • the input device 5 and the display device 4 may be integrated devices.
  • the play analysis device 100 identifies a player who has performed an action on the ball by using a moving image frame taken by each camera 3.
  • the actions taken on the ball include serve, reception, dig, toss, attack, and block.
  • a player who has performed an action on the ball may be referred to as an "actor”.
  • the play analyzer 100 detects the rally section of volleyball using the moving image frame.
  • the center point of the surface of the coat 10 is the origin O
  • the axis parallel to the surface of the coat 10 and parallel to the net 11 is the X axis
  • the axis perpendicular to the net 11 is the Y axis
  • the axis perpendicular to the surface of the coat 10 is the Z axis.
  • the X-axis is in the forward direction away from the referee 12 and to the referee 12.
  • the approaching direction is the negative direction.
  • the Y-axis has a positive direction in the left direction and a negative direction in the right direction when viewed from the referee 12.
  • the Z-axis has a positive direction away from the surface of the coat 10. That is, the Z-axis coordinate z corresponds to the height from the surface of the coat 10.
  • the linesmen 14A judges the line on the positive direction side of the Y-axis
  • the linesmen 14B judges the line on the negative direction side.
  • the linesmen 14C judges the line on the positive direction side of the X axis among the lines of the two courts 10 parallel to the Y axis
  • the linesmen 14D judges the line on the negative direction side.
  • the linesmen 14 moves the flag 15 that they own and informs the players and the like of the judgment result.
  • the play analysis device 100 includes a video receiving unit 101, a moving body locus detection unit 102, an action determination unit 103, a number recognition unit 104, a referee motion detection unit 105, a rally section determination unit 106, and a result output unit 107.
  • the UI control unit 108, the video control unit 109, the play information control unit 110, the score information control unit 111, and the storage unit 112 are provided.
  • the video receiving unit 101 receives the moving image frame 201 transmitted from each of the cameras 3A to 3D and stores it in the storage unit 112. The video receiving unit 101 discards frames that are not associated with the rally section information.
  • the moving body locus detection unit 102 applies, for example, the method disclosed in Non-Patent Document 1 to a plurality of moving image frames 201 stored in the storage unit 112, and applies the method disclosed in Non-Patent Document 1 to the time when the moving body locus 201 was shot (hereinafter, “” The three-dimensional position (x, y, z) and velocity of the ball at the "frame time") are calculated. Then, the moving body locus detection unit 102 generates the ball locus information 202 in association with the frame time, the three-dimensional position of the ball, and the moving speed, and stores the ball locus information 202 in the storage unit 112.
  • the moving body locus detection unit 102 If the moving body locus detection unit 102 cannot detect the ball, the moving body locus detection unit 102 generates ball locus information 202 indicating that fact and stores it in the storage unit 112. That is, the moving body locus detection unit 102 detects the movement pattern of the ball and stores the ball locus information 202 related to the detection result in the storage unit 112.
  • the action determination unit 103 determines whether or not an action has occurred on the ball from the ball trajectory information 202. When it is determined that an action has occurred, the action determination unit 103 generates action information 203 by associating the frame time in which the action occurred (hereinafter referred to as "action frame time") with the type of the action in which the action occurred. It is stored in the storage unit 112. The details of the action determination unit 103 will be described later.
  • the number recognition unit 104 recognizes the uniform number of the actor from the plurality of moving image frames 201 stored in the storage unit 112. Then, the number recognition unit 104 generates the actor information 204 by associating the action frame time with the recognized uniform number of the actor, and stores the actor information 204 in the storage unit 112. The details of the number recognition unit 104 will be described later.
  • the referee motion detection unit 105 detects the movement of the flag 15 of the linesmen 14 as the motion pattern of the referee from the moving image frame 201 used to generate the ball trajectory information 202. Then, the referee motion detection unit 105 determines whether or not the movement of the flag 15 indicates the end of the rally section, and generates a flag motion flag indicating the determination result. The details of the referee motion detection unit 105 will be described later.
  • the rally section determination unit 106 determines the start of the rally section based on the trajectory pattern of the ball, and determines the end of the rally section based on the flag operation flag and the operation of the input device 5 from the user. Then, the rally section determination unit 106 generates the rally section information 205 by associating the frame time associated with the ball locus information 202 and the flag operation flag with the information indicating the start or end of the rally section, and stores the storage unit 106. Store in 112. Further, when the rally section determination unit 106 cannot determine the start and end of the rally section, it determines that the rally is in progress.
  • the rally section determination unit 106 stores the rally section information 205 including the information indicating that the rally is in progress in the storage unit 112 instead of the information indicating the start or end of the rally section. The details of the rally section determination unit 106 will be described later.
  • the result output unit 107 generates play information 206 by associating the ball trajectory information 202, the action information 203, the actor information 204, and the rally section information 205, and stores the play information 206 in the storage unit 112. It is desirable that the play information 206 is managed in units of matches or sets, and it is preferable that the play information 206 includes information regarding the match section (match start, end) and set section (set start, end) by the user's operation. From this point of view, in the present embodiment, the result output unit 107 registers the end information indicating the end of the set or the game in the play information 206 based on the operation of the input device 5 from the user.
  • the action frame time T and the action type “attack” are associated with each other
  • the actor information 204 the action frame time T and the actor uniform number “14” are associated with each other.
  • the result output unit 107 When the action frame time T and the fact that the rally is in progress are associated with each other in the information 205, the result output unit 107 generates the play information 206 as shown in FIG. That is, as shown in FIG. 3, the result output unit 107 indicates that the action type “attack”, the uniform number “14” of the actor, and the rally are in progress with respect to the frame time T of the ball trajectory information 202.
  • the associated play information 206 is generated.
  • the frame time T, in rally, uniform number player "14" are three-dimensional position (x T, y T, z T ), It can be known that the "attack” of the speed " ST (km / h)" is performed.
  • the result output unit 107 generates play information 206 associated with the start of the rally section with respect to the frame time Tt1 of the ball trajectory information 202.
  • the result output unit 107 generates play information 206 associated with the end of the rally section with respect to the frame time T + t2 of the ball trajectory information 202.
  • the result output unit 107 generates play information 206 in which the end of the rally section and the end of the set are associated with the frame time T + t3 of the ball locus information 202.
  • the result output unit 107 generates play information 206 in which the end of the rally section and the end of the match are associated with the frame time T + t4 of the ball trajectory information 202.
  • the UI control unit 108 generates the UI 2000 as shown in FIG. 4 and displays it on the display device 4. Further, the UI control unit 108 receives an operation for the UI 2000 through the input device 5.
  • the video control unit 109 displays the ball game play video 2100 on the UI2000.
  • the play video 2100 may be a real-time video being shot by the cameras 3A to 3D, or may be a recorded video of past play. Further, the video control unit 109 displays a play video 2100 of a scene based on the operation of the input device 5 from the user. The details of the image control unit 109 will be described later.
  • the play information control unit 110 arranges the contents of the play information 206 generated during one rally in chronological order and displays it on the UI 2000 as a play information list 2200.
  • the details of the play information list 2200 will be described later.
  • the score information control unit 111 arranges the contents of the score information in chronological order and displays the score information list 2300 on the UI2000.
  • One score information includes the score of one's own team after one rally and the score of the opponent's team. Further, one or more play information 206 generated during the rally is associated with one score information. The details of the score information list 2300 will be described later.
  • the action determination unit 103 calculates the trajectory of the ball from the three-dimensional position and velocity of the ball for each frame time included in the ball trajectory information 202. Then, when the calculated trajectory change of the ball matches a predetermined condition, the action determination unit 103 determines that an action for the ball has occurred. When the action determination unit 103 determines that an action has occurred, the action determination unit 103 generates action information 203 in association with the action frame time and the type of the action that has occurred, and stores the action information 203 in the storage unit 112.
  • the action determination unit 103 may determine the action type based on the change in the trajectory of the ball, the three-dimensional position and speed of the ball, the rules of sports, and the like.
  • Volleyball action types include serve, reception (receive against serve), dig (receive against attack), toss, attack, and block.
  • the trajectory of the ball detected for the first time from the start of analysis has a movement component in the Y-axis direction (the long side direction of the court shown in FIG. 1), and the velocity component of the ball in the Y-axis direction is If it is within the predetermined range, the action type is determined to be "serve".
  • the action determination unit 103 when the trajectory of the ball straddles the coordinates where the net 11 exists on the Y axis after “serving", and the change in the three-dimensional position of the ball changes from descending to ascending (that is,). , When the change in coordinates in the Z-axis direction turns positive), the action type is determined to be “reception”. According to the rules of volleyball, the action to receive "serve” is “reception", so by making a judgment based on such rules, "reception” and “dig” can be distinguished.
  • the action determination unit 103 determines another action type according to the determination criteria according to the rules of sports.
  • the occurrence of the action is determined using the three-dimensional information, but the occurrence of the action may be determined using the two-dimensional or one-dimensional information. For example, when determining the occurrence of "serve" under the above-mentioned conditions, it is possible to determine from only the analysis start time and the information in the Y-axis direction without using the information in the X-axis direction and the Z-axis direction.
  • the number recognition unit 104 selects a video frame corresponding to the action frame time included in the action information 203 and a video frame corresponding to a frame time in the vicinity of the action frame time as an action frame. Select as. Next, the number recognition unit 104 detects an area where a player (person) is photographed (hereinafter referred to as a “player area”) from the action frame. For example, the number recognition unit 104 detects a player area from within the detection area by using a player detection model (neural network) in which a plurality of player images during competition are trained by deep learning. Then, the number recognition unit 104 recognizes the uniform number of the actor from the action frame.
  • a player detection model neural network
  • the number recognition unit 104 recognizes the uniform number of an actor from within the detection area by using a model (neural network) for detecting uniform numbers obtained by learning numerical images of uniforms of a plurality of athletes by deep learning.
  • the number recognition unit 104 generates the actor information 204 by associating the action frame time with the recognized uniform number of the actor, and stores it in the storage unit 112.
  • the referee motion detection unit 105 calculates the three-dimensional position and velocity of the flag 15 at the frame time from the moving image frame 201 used to generate the ball trajectory information 202. Next, the referee motion detection unit 105 determines whether or not the movement of the flag 15 indicates the end of the rally based on their three-dimensional positions and moving speeds. An example of the movement of the flag 15 indicating the end of the rally will be described later. Then, the referee motion detection unit 105 generates a flag motion flag indicating the determination result of the end of the rally.
  • the frame time associated with the flag operation flag by the referee operation detection unit 105 may be any time from the start to the end of the movement of the flag 15 indicating the end of the rally. For example, the flag 15 is in an operating state from a stationary state. It may be the time of the timing changed to.
  • Example of flag movement indicating the end of the rally Next, an example of the movement of the flag 15 indicating the end of the rally will be described with reference to FIGS. 5A to 5D.
  • the referee motion detection unit 105 determines that the rally has been completed according to the criteria according to the sports rules when the referee's movements and the like meet the predetermined conditions.
  • FIG. 5A shows a ball-out operation pattern in which the attacking ball falls to the outside of the court 10.
  • the linesmen 14 raises the flag 15 lowered as shown in image P1 as shown in image P2.
  • FIG. 5B shows a blockout operation pattern in which the attacking ball hits the blocker and then falls to the floor.
  • the linesmen 14 moves the flag 15 lowered as shown in the image P1 so that the bar is horizontal at the height of the face as shown in the image P3.
  • FIG. 5C shows an operation pattern at the time of a foul such as a ball hitting an antenna.
  • the linesmen 14 swings the flag 15 lowered as shown in the image P1 to the left and right on the head as shown in the image P4.
  • FIG. 5D shows a ball-in motion pattern in which the attacking ball falls inside the court 10.
  • the linesmen 14 moves the flag 15 lowered as shown in image P1 to the outside of the body as shown in image P5, and then moves it so as to point to the floor in front of the body.
  • the rally section determination unit 106 determines that the ball has been thrown up, which is the start operation of the serve, based on the ball trajectory information 202, it determines that the rally has started at the frame time associated with the ball trajectory information 202. To do. Then, the rally section determination unit 106 generates the rally section information 205 by associating the frame time of the rally start with the information indicating the start of the rally section, and stores it in the storage unit 112. The frame time at the start of the rally may be any time from the start to the end of the throwing of the ball. The rally section determination unit may determine that the rally section has started based on the determination result of "serve" of the action determination unit 103 without using the ball trajectory information 202.
  • the rally section determination unit 106 determines that the rally has ended at the frame time associated with this flag operation flag. Then, the rally section determination unit 106 stores the rally section information 205 in which the frame time of the rally end and the information indicating the end of the rally section are associated with each other in the storage unit 112.
  • the rally section determination unit 106 determines that the moving speed of the ball is less than the threshold speed for a certain frame period based on the ball trajectory information 202, the rally section ends at the frame time associated with the ball trajectory information 202. It is judged that it has been done. Then, the rally section determination unit 106 stores the rally section information 205 in which the frame time when the moving speed becomes less than the threshold speed and the information indicating the end of the rally section are associated with each other in the storage unit 112. For example, when there is a foul, it is no longer necessary to continue the rally, and as shown in FIG. 6A, a player who intends to toss or receive may catch the ball B and stop playing. At this time, the linesmen 14 may not judge.
  • the rally section determination unit 106 can determine the end of the rally section based on the moving speed of the ball.
  • the threshold speed may be set to, for example, a speed slower than the speed at which the player walks with the ball or the speed at which the player throws the ball at the player who hits the serve next.
  • the rally section determination unit 106 determines that the three-dimensional position of the ball is out of the effective range of the game based on the ball trajectory information 202, the rally section is said to have ended at the frame time associated with the ball trajectory information 202. judge.
  • the position outside the effective range of the game is, for example, a viewing seat.
  • the rally section determination unit 106 stores the rally section information 205 in which the frame time when the ball enters the spectator's seat and the information indicating the end of the rally section are associated with each other in the storage unit 112. For example, as shown in FIG. 6B, when the ball B goes out of the effective range of the match due to a receive miss, the rally ends, but the linesmen 14 may not judge.
  • the moving speed may not be less than the threshold speed, but the rally section determination unit 106 can determine the end of the rally section based on the three-dimensional position of the ball.
  • the rally section determination unit 106 determines based on the ball trajectory information 202 that the time during which the ball cannot be detected continues for the threshold time or longer, it determines that the rally section has ended at the frame time associated with the ball trajectory information 202. .. Then, the rally section determination unit 106 stores the rally section information 205 in which the frame time when the time during which the ball cannot be detected becomes equal to or longer than the threshold time and the information indicating the end of the rally section are associated with each other in the storage unit 112. For example, when there is a foul, when a player trying to toss or receive catches the ball and moves to the service area as it is, it is blocked by other players and the ball cannot be photographed with the camera 3.
  • the moving body locus detection unit 102 may not be able to detect the ball.
  • the end of the rally section cannot be determined based on whether the moving speed of the ball is less than the threshold speed or whether the ball is out of the effective range of the game, but the rally section determination unit 106 determines the ball.
  • the end of the rally section can be determined based on the undetectable time.
  • the threshold time may be set based on, for example, the maximum number of continuous frames in which ball shielding can occur during the rally.
  • the rally section determination unit 106 determines that the end of the rally section is registered based on the operation of the input device 5 from the user, the rally section information 205 that associates the frame time of the rally end with the information indicating the end of the rally section. Is stored in the storage unit 112.
  • the case where the user performs the registration operation for the end of the rally section is, for example, the case where the user determines from the content of the video that the rally has ended.
  • the rally section determination unit 106 when the rally section determination unit 106 cannot determine the start and end of the rally section and does not recognize the operation to register the end of the rally section from the user, it is associated with the flag operation flag. It is determined that the rally is being performed at the frame time. Then, the rally section determination unit 106 stores the rally section information 205 in which the frame time during the rally and the information indicating the rally are associated with each other in the storage unit 112.
  • the play information list 2200 displays the contents of the play information 206 generated during one rally in chronological order. As shown in FIG. 4, the play information list 2200 has a play button 2201, a team 2202, a player number 2203, and an action 2204 for each play information 206.
  • the play button 2201 is a button for playing the play video 2100 related to the play information 206 to which the play button 2201 is associated.
  • Team 2202 indicates whether the player who performed action 2204 is his own team or the opponent's team. For example, "a" represents your team and “e” represents your opponent's team. Further, the team 2202 is a button, and when the team 2202 is pressed, the play information control unit 110 displays “a” and "e” buttons for correction.
  • the player number 2203 indicates the number (uniform number) of the player who performed the action 2204. Further, the player number 2203 is a button, and the play information control unit 110 displays a button for correcting the player number when the player number 2203 is pressed.
  • the button for correcting the player number may be a button for inputting a number. Alternatively, the button for correcting the player number may be a list of player numbers registered in the team.
  • Action 2204 indicates the type of action. For example, “S” is a serve, “R” is a reception, “E” is a toss, “A” is an attack, “D” is a dig, and “B” is a block. Further, the action 2204 is a button, and when the action 2204 is pressed, the play information control unit 110 "S", “R”, “E”, “A”, “D” for correction. And the "B” button is displayed.
  • the first line of the play information list 2200 shown in FIG. 4 indicates that the player with the uniform number "6" of his team ("a") has performed the reception "R".
  • the score information list 2300 displays the contents of the score information corresponding to each rally in one set in chronological order. As shown in FIG. 4, the score information list 2300 has a play button 2301, a score 2302, and a check box 2303 for each score information.
  • the play button 2301 is a button for playing the play video 2100 related to the score information associated with the play button 2301.
  • Score 2302 indicates the score of one's own team and the opponent's team after the rally. Further, the score 2302 can be selected (clicked), and when the score 2302 is selected, the score information control unit 111 uses the play information list 2200 associated with the rally (score information) of the score 2302. Is displayed.
  • Check box 2303 is for the user to turn on and off the check for the rally on a score-by-score basis. For example, in a real-time situation such as during a match, even if an error is found in the contents of the play information list 2200 (team 2202, player number 2203, action 2204), the next rally will start immediately, so a certain rally
  • the user can identify that the contents of the play information list 2200 are in the provisional state by selecting the check box 2303 corresponding to the rally (input of identification information) in case of wanting to re-verify (correct) later. To do so. This allows the user to later re-verify the rally with the check box 2303 selected.
  • the score information control unit 111 limits and displays only the rallies in which the check box 2303 is selected in response to the instruction from the user, so that the user can efficiently perform the correction work.
  • the number of check boxes 2303 is not limited to one.
  • the video control unit 109 plays the play video 2100 in the vicinity of the frame time of the play information 206 corresponding to the play button 2201. As a result, the user can easily confirm the play of the player corresponding to the play information 206.
  • the video control unit 109 plays the play video 2100 of the rally corresponding to the play button 2301.
  • the video control unit 109 identifies the rally associated with the score of the pressed play button 2301.
  • the video control unit 109 identifies the play information 206 including the rally section information 205 indicating the start of the specified rally section, and recognizes the frame time of the specified play information 206 as the start time of the rally section.
  • the video control unit 109 identifies the play information 206 including the rally section information 205 indicating the end of the rally section, and recognizes the frame time of the specified play information 206 as the end time of the rally.
  • the video control unit 109 acquires the moving image frame 201 between the start time and the end time of the rally section, and reproduces the play video 2100 of the rally section. That is, the video control unit 109 extracts the play video 2100 of the rally section from the play video 2100 according to the determination result of the rally section determination unit 106.
  • the video receiving unit 101 receives the moving image frame 201 transmitted from the cameras 3A to 3D and stores it in the storage unit 112 (S1).
  • the mobile locus detection unit 102 acquires a plurality (predetermined number) of moving image frames 201 stored in the storage unit 112 (S2).
  • the moving body locus detection unit 102 determines whether or not the ball can be detected from the moving image frame 201 (S3).
  • the video receiving unit 101 discards the moving image frame 201 used for ball detection from the storage unit 112 (S4) and returns to S1. ..
  • the moving body locus detection unit 102 calculates the three-dimensional position and the moving speed of the ball at each frame time, and generates the ball locus information 202 (S5).
  • the rally section determination unit 106 determines whether or not the rally section has been started based on the ball locus information 202 (S6).
  • the rally section determination unit 106 determines that the rally section has not started (S6: NO)
  • the ball trajectory information 202 is discarded (S7). After that, the processing of S4 is performed, and the process returns to S1.
  • the rally section determination unit 106 determines that the rally section has started (S6: YES)
  • the rally section determination unit 106 generates rally section information 205 indicating the start of the rally section (S8).
  • the result output unit 107 determines whether or not the play information 206 indicating the start of the rally section has been generated (S9).
  • the result output unit 107 determines that the play information 206 indicating the start of the rally section has not been generated (S9: NO)
  • the result output unit 107 generates the play information 206 as shown in FIG. 9 (S10).
  • the processing of S10 is performed after the determination of NO in S9 in this way, the result output unit 107 generates play information 206 including ball trajectory information 202 and rally section information 205.
  • the video receiving unit 101 performs the same process as in S1.
  • the moving image frames 201 from the cameras 3A to 3D are stored in the storage unit 112 (S11).
  • the moving body locus detection unit 102 performs the same processing as in S2 and S5. That is, the moving body locus detection unit 102 acquires a predetermined number of moving image frames 201 from the storage unit 112 (S12) and generates the ball locus information 202 (S13).
  • the action determination unit 103 determines whether or not an action has occurred on the ball from the ball trajectory information 202 (S14). When the action determination unit 103 determines that the action has not occurred (S14: NO), the result output unit 107 performs the process of S10 (the process of generating the play information 206) as shown in FIG. When the process of S10 is performed after the determination of NO in S14 is performed in this way, the result output unit 107 generates play information 206 including only the ball trajectory information 202. On the other hand, as shown in FIG. 8, when the action determination unit 103 determines that an action has occurred (S14: YES), the action determination unit 103 generates action information 203 (S15). Next, the number recognition unit 104 generates the actor information 204 (S16).
  • the referee motion detection unit 105 detects the movement of the flag 15 of the line umpire 14 based on the moving image frame 201 used to generate the ball trajectory information 202, and the flag motion relating to whether or not the movement indicates the end of the rally section. Generate a flag (S17).
  • the rally section determination unit 106 performs the end determination process of the rally section (S18). The details of the end determination process of the rally section will be described later.
  • the rally section determination unit 106 determines whether or not the end of the rally could be determined in the end determination process of the rally section (S19).
  • the rally section determination unit 106 can determine the end of the rally section (S19: YES)
  • the rally section determination unit 106 generates rally section information 205 indicating the end of the rally section (S20).
  • the result output unit 107 performs the processing of S10 (the processing of generating the play information 206).
  • the processing of S20 is performed after the determination of YES in S19 in this way, the result output unit 107 generates play information 206 including ball trajectory information 202 and rally section information 205.
  • the rally section determination unit 106 determines whether or not there has been a rally section end registration operation from the user (S21). When it is determined that the end registration operation has been performed (S21: YES), the rally section determination unit 106 performs the process of S20 (the process of generating the rally section information 205 indicating the end of the rally section). Then, the result output unit 107 performs the processing of S10 (the processing of generating the play information 206). When the processing of S10 is performed after the determination of YES in S21 in this way, the result output unit 107 generates play information 206 including the respective information 202 and 205, as in the case of the determination of YES in S19.
  • the rally section determination unit 106 when it is determined that the rally section end registration operation has not been performed (S21: NO), the rally section determination unit 106 generates rally section information 205 indicating that the rally is in progress (S22). Then, the result output unit 107 performs the processing of S10 (the processing of generating the play information 206). When the processing of S10 is performed after the processing of S22 in this way, the result output unit 107 generates play information 206 including ball trajectory information 202, action information 203, actor information 204, and rally section information 205.
  • the result output unit 107 determines whether or not the rally section has ended (S23).
  • the result output unit 107 performs the process of S9 when it is determined that the rally section has not ended (S23: NO), that is, when the play information 206 generated in S10 indicates the start of the rally section or the middle of the rally.
  • the result output unit 107 determines whether or not the user has registered the end of the match (S24).
  • the result output unit 107 determines that the match end registration operation has been performed (S24: YES)
  • the result output unit 107 registers the match end in the play information 206 (S25). Then, the play information generation process ends.
  • the result output unit 107 determines whether or not there has been a set end registration operation from the user (S26). When the result output unit 107 determines that the set end registration operation has been performed (S26: YES), the result output unit 107 registers the set end in the play information 206 (S27). Then, it returns to S1. On the other hand, when the result output unit 107 determines that the set end registration operation has not been performed (S26: NO), the result output unit 107 returns to S1 without performing the process of S27.
  • the rally section determination unit 106 determines whether or not the flag operation flag indicates the end of the rally (S31), as shown in FIG. When the rally section determination unit 106 determines that the flag operation flag indicates the end of the rally (S31: YES), the rally section determination unit 106 determines that the rally section has ended (S32). Then, the rally section end determination process is completed, and the flow proceeds to S19 shown in FIG.
  • the rally section determination unit 106 determines whether or not the moving speed of the ball is less than the threshold speed based on the ball trajectory information 202 (S33). .. When the rally section determination unit 106 determines that the moving speed of the ball is less than the threshold speed (S33: YES), the rally section determination unit 106 proceeds to S32.
  • the rally section determination unit 106 determines whether or not the position of the ball is out of the effective range of the game based on the ball trajectory information 202 (S33: NO). S34). When the rally section determination unit 106 determines that the position of the ball is out of the effective range of the match (S34: YES), the rally section determination unit 106 proceeds to S32.
  • the rally section determination unit 106 determines whether or not the time during which the ball cannot be detected continues for the threshold time or longer based on the ball trajectory information 202. (S35). When the rally section determination unit 106 determines that the time during which the ball cannot be detected continues for the threshold time or longer (S35: YES), the process proceeds to S32.
  • the rally section determination unit 106 determines that the time during which the ball cannot be detected has not continued for the threshold time or longer (S35: NO), the rally section end determination process ends. Then, the flow proceeds to S19 shown in FIG.
  • the play analyzer 100 uses the play video 2100 to detect the operation pattern of the linesmen 14. Next, the play analyzer 100 determines the start of the rally section based on the play video 2100, and determines the end of the rally section based on the operation pattern of the linesmen 14.
  • the play analyzer 100 can easily acquire the play video 2100 of the rally section for each score from the play video 2100 based on the determination result of the start and end of the rally section. Further, in volleyball, it is difficult to determine from the play video 2100 that the ball has fallen to the floor, and if the play is continued even after the ball has fallen to the floor, this continued play may be determined to be in the rally. There is. However, since the play analyzer 100 determines the end of the rally section based on the operation pattern of the linesmen 14, it can appropriately determine the end of the rally even in a situation where it is difficult to determine whether or not the ball has fallen to the floor.
  • the play analyzer 100 determines the end of the rally section based on the operation pattern of the flag 15 of the linesmen 14.
  • the play analyzer 100 can easily determine the end of the rally section based on the flag 15 whose appearance is conspicuous. Further, since the play analyzer 100 detects the movement patterns of the flags 15 of the four line referees 14, the determination accuracy is improved as compared with the case of detecting the movement patterns of the referee 12 and the assistant referee 13 who are one person each. Can be improved.
  • the play analysis device 100 may perform a process of generating play information 206 after the shooting is completed.
  • the play analyzer 100 may determine the end of the rally section instead of the linesmen 14 or together with the linesmen 14 based on the operation pattern of at least one of the referee 12 or the assistant referee 13.
  • the play analyzer 100 may determine the end of the rally section based on the movement amount of the flag 15 instead of the movement speed of the flag 15 or together with the movement speed.
  • the play analyzer 100 may determine the end of the rally section based on the amount of movement of the ball instead of or together with at least one of the moving speed and position of the ball.
  • the play analyzer 100 may determine the start of the rally section instead of the trajectory pattern of the ball or based on the motion pattern of at least one of the referee 12 or the assistant referee 13 together with the trajectory pattern of the ball.
  • the play information 206 may not include the end information, or may include start information indicating the start of at least one of the set and the match in addition to or instead of the end information.
  • the end information may indicate only the end of the set or only the end of the match.
  • the play analysis device 100 determines whether or not there is an end registration operation of the rally section (S21), determines whether or not the moving speed of the ball is less than the threshold speed (S33), and the position of the ball is a match. Even if at least one of the process of determining whether or not the ball is out of the effective range (S34) and the process of determining whether or not the time during which the ball cannot be detected continues for the threshold time or longer (S35) is not performed. Good.
  • the play analyzer 100 determines the start of the rally section based on the ball detection, but as shown in FIG. 12, the play analyzer 100 determines the start of the rally section based on the action detection of the serve. May be good.
  • the play analyzer 100 performs the processing of S5 (the processing of generating the ball locus information 202) after the processing of S1 and S2.
  • the action determination unit 103 determines whether or not the serve action has occurred (S41). Whether or not the action of this serve has occurred may be determined based on the trajectory of the ball in the ball trajectory information 202, or may be determined based on the movement of the player in the moving image frame.
  • the play analyzer 100 performs the processes of S7 and S4.
  • the rally section determination unit 106 performs the process of S8 (the process of generating the rally section information 205 indicating the start of the rally section). .. Then, the play analyzer 100 performs the same processing as S15 and S16 (action information 203 generation processing (S42) and actor information 204 generation processing (S43)), and then performs the processing of S9 shown in FIG. ..
  • the generated play information 206 indicating the start of the rally section includes the ball trajectory information 202, the action information 203 indicating that the rally section is serving, the actor information 204, and the rally section information 205.
  • FIG. 13 is a diagram showing a hardware configuration of a computer that realizes the functions of each device by a program.
  • the computer 3100 includes an input device 3101 such as a keyboard or mouse and a touch pad, an output device 3102 such as a display or a speaker, a CPU (Central Processing Unit) 3103, a GPU (Graphics Processing Unit) 3104, and a ROM (Read Only Memory) 3105.
  • Read information from a recording medium such as RAM (RandomAccessMemory) 3106, hard disk device or storage device 3107 such as SSD (SolidStateDrive), DVD-ROM (DigitalVersatileDiskReadOnlyMemory) or USB (UniversalSerialBus) memory. It includes a reading device 3108 and a transmitting / receiving device 3109 that communicates via a network, and each unit is connected by a bus 3110.
  • the reading device 3108 reads the program from the recording medium on which the program for realizing the function of each of the above devices is recorded, and stores the program in the storage device 3107.
  • the transmission / reception device 3109 communicates with the server device connected to the network, and stores the program downloaded from the server device for realizing the function of each device in the storage device 3107.
  • the CPU 3103 copies the program stored in the storage device 3107 to the RAM 3106, and sequentially reads and executes the instructions included in the program from the RAM 3106, whereby the functions of the above devices are realized.
  • LSI is an integrated circuit. These may be individually integrated into one chip, or may be integrated into one chip so as to include a part or all of them. Although it is referred to as LSI here, it may be referred to as IC, system LSI, super LSI, or ultra LSI depending on the degree of integration.
  • the method of making an integrated circuit is not limited to LSI, and may be realized by a dedicated circuit or a general-purpose processor.
  • An FPGA Field Programmable Gate Array
  • a reconfigurable processor that can reconfigure the connection and settings of the circuit cells inside the LSI may be used.
  • One aspect of the disclosure is useful for systems for analyzing sports.

Abstract

A play analyzing device 100 is provided with: a referee motion detecting unit 105 which employs a play video to detect a motion pattern of a referee of a ball game; and a rally segment determining unit 106 which determines the start of a rally segment on the basis of the play video, and determines the end of the rally segment on the basis of the motion pattern of the referee.

Description

プレイ分析装置、及び、プレイ分析方法Play analyzer and play analysis method
　本開示は、スポーツのプレイ分析装置、及び、プレイ分析方法に関する。 This disclosure relates to a sports play analyzer and a play analysis method.
　従来より、バレーボール分析ソフトウェアとして「Data Volley」が知られている。Data Volleyはこのソフトウェアに精通したアナリストによって使用され、当該アナリストが、主観的判断に基づいて、チームの選手の状況を示すデータをData Volleyに入力する。 Conventionally, "Data Volley" has been known as volleyball analysis software. Data Volley is used by analysts who are familiar with the software, who, based on subjective judgment, enter data into Data Volley that indicates the status of the players on the team.
　特許文献１には、アナリストの操作性を向上させたスカウティングシステムが開示されている。具体的には、特許文献１には、画像解析を用いて各選手の位置情報を取得し、入力された選手状況と、各選手のそれぞれの位置情報と、所望のシーンの静止画像とを関連付けて記憶装置にデータベースを構築し、各選手の各位置、所望のシーンの静止画像、選手状況を表示することが開示されている。この発明により、スコアラー（アナリスト）のトレーニングが必要であり、各選手の位置情報が視覚による主観的判断により入力されているためにデータ精度が低いというData Volleyのソフトウェアの問題点を解決できる。 Patent Document 1 discloses a scouting system with improved operability of analysts. Specifically, in Patent Document 1, the position information of each player is acquired by using image analysis, and the input player situation, each position information of each player, and a still image of a desired scene are associated with each other. It is disclosed that a database is constructed in a storage device to display each position of each player, a still image of a desired scene, and a player situation. According to the present invention, it is possible to solve the problem of Data Volley software that training of a scorer (analyst) is required and the data accuracy is low because the position information of each player is input by visual subjective judgment.
　また、従来のバレーボール分析ソフトウェアは、サーブに対するレシーブのシーンを撮影した映像を順次取り込み、各選手の技能評価を入力し、各選手のレシーブ回数及びレシーブ成功率等を集計及び表示する。ユーザ（チームの監督等）は、この集計結果を見て、チームの強み及び弱みを分析する。 In addition, the conventional volleyball analysis software sequentially captures the images of the receive scenes for the serve, inputs the skill evaluation of each player, and aggregates and displays the number of receive times and the receive success rate of each player. The user (team manager, etc.) analyzes the strengths and weaknesses of the team by looking at the aggregated results.
特開２００４－３５１０９７号公報Japanese Unexamined Patent Publication No. 2004-351907
　上述のような従来の技術では、例えば、サーブレシーブの瞬間の画像を順次取り込み、各選手の技能評価を入力することができるが、得点ごとのラリー区間の映像を取得する仕組みについては考慮されていない。 In the conventional technique as described above, for example, images at the moment of serve receive can be sequentially captured and the skill evaluation of each player can be input, but the mechanism for acquiring the image of the rally section for each score is considered. Absent.
　本開示の目的は、プレイ映像から得点ごとのラリー区間の映像を容易に取得できるプレイ分析装置及び方法を提供することにある。 An object of the present disclosure is to provide a play analyzer and a method capable of easily acquiring a video of a rally section for each score from a play video.
　本開示の一態様に係るプレイ分析装置は、球技のプレイを分析するプレイ分析装置であって、プレイ映像を用いて、前記球技の審判員の動作パターンを検出する審判動作検出部と、前記プレイ映像に基づきラリー区間の開始を判定すると共に、前記審判員の動作パターンに基づき前記ラリー区間の終了を判定するラリー区間判定部と、を備える。 The play analysis device according to one aspect of the present disclosure is a play analysis device that analyzes the play of a ball game, and includes a referee motion detection unit that detects an motion pattern of a referee of the ball game using a play video, and the play. It is provided with a rally section determination unit that determines the start of the rally section based on the image and determines the end of the rally section based on the operation pattern of the referee.
　本開示の一態様に係るプレイ分析方法は、球技のプレイを分析するプレイ分析方法であって、プレイ映像を用いて前記球技の審判員の動作パターンを検出し、前記プレイ映像に基づきラリー区間の開始を判定すると共に、前記審判員の動作パターンに基づき前記ラリー区間の終了を判定する。 The play analysis method according to one aspect of the present disclosure is a play analysis method for analyzing the play of a ball game, in which the motion pattern of the referee of the ball game is detected using the play video, and the rally section is based on the play video. The start is determined, and the end of the rally section is determined based on the movement pattern of the referee.
　なお、これらの包括的又は具体的な態様は、システム、装置、方法、集積回路、コンピュータプログラム、又は、記録媒体で実現されてもよく、システム、装置、方法、集積回路、コンピュータプログラムおよび記録媒体の任意な組み合わせで実現されてもよい。 Note that these comprehensive or specific embodiments may be realized in a system, device, method, integrated circuit, computer program, or recording medium, and the system, device, method, integrated circuit, computer program, and recording medium. It may be realized by any combination of.
　本開示によれば、プレイ映像から得点ごとのラリー区間の映像を容易に取得できる。 According to the present disclosure, the video of the rally section for each score can be easily obtained from the play video.
実施の形態に係るプレイ分析システムの概要を示す図である。It is a figure which shows the outline of the play analysis system which concerns on embodiment. 実施の形態に係るプレイ分析装置の構成の例を示す図である。It is a figure which shows the example of the structure of the play analyzer which concerns on embodiment. 実施の形態に係るプレイ情報の例を示す図である。It is a figure which shows the example of the play information which concerns on embodiment. 実施の形態に係るプレイ分析装置が生成するＵＩ（User Interface）の例を示す図である。It is a figure which shows the example of the UI (User Interface) generated by the play analysis apparatus which concerns on embodiment. 実施の形態に係るボールアウトの旗の動作パターンを示す図である。It is a figure which shows the operation pattern of the ball-out flag which concerns on embodiment. 実施の形態に係るブロックアウトの旗の動作パターンを示す図である。It is a figure which shows the operation pattern of the block-out flag which concerns on embodiment. 実施の形態に係る反則時の旗の動作パターンを示す図である。It is a figure which shows the operation pattern of the flag at the time of the offense which concerns on embodiment. 実施の形態に係るボールインの旗の動作パターンを示す図である。It is a figure which shows the operation pattern of the ball-in flag which concerns on embodiment. 実施の形態に係るボールの移動速度に基づきラリー区間が終了したと判定する場合のプレイ状態を示す図である。It is a figure which shows the play state when it is determined that the rally section is finished based on the moving speed of the ball which concerns on embodiment. 実施の形態に係るボールの３次元位置に基づきラリー区間が終了したと判定する場合のプレイ状態を示す図である。It is a figure which shows the play state when it is determined that the rally section is finished based on the three-dimensional position of the ball which concerns on embodiment. 実施の形態に係るプレイ情報生成処理を示すフローチャートである。It is a flowchart which shows the play information generation processing which concerns on embodiment. 実施の形態に係るプレイ情報生成処理を示すフローチャートであり、図７のフローチャートの続きの処理を示す。It is a flowchart which shows the play information generation processing which concerns on embodiment, and shows the continuation processing of the flowchart of FIG. 実施の形態に係るプレイ情報生成処理を示すフローチャートであり、図８のフローチャートの続きの処理を示す。It is a flowchart which shows the play information generation processing which concerns on embodiment, and shows the continuation processing of the flowchart of FIG. 実施の形態に係るプレイ情報生成処理を示すフローチャートであり、図９のフローチャートの続きの処理を示す。It is a flowchart which shows the play information generation processing which concerns on embodiment, and shows the continuation processing of the flowchart of FIG. 実施の形態のラリー区間終了判定処理を示すフローチャートである。It is a flowchart which shows the rally section end determination processing of embodiment. 変形例に係るプレイ情報生成処理を示すフローチャートである。It is a flowchart which shows the play information generation processing which concerns on a modification. 本開示の実施の形態に係るハードウェア構成の例を示す図である。It is a figure which shows the example of the hardware configuration which concerns on embodiment of this disclosure.
　以下、図面を適宜参照して、本発明の実施の形態について、詳細に説明する。但し、必要以上に詳細な説明は省略する場合がある。例えば、既によく知られた事項の詳細説明や実質的に同一の構成に対する重複説明を省略する場合がある。これは、以下の説明が不必要に冗長になるのを避け、当業者の理解を容易にするためである。 Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings as appropriate. However, more detailed explanation than necessary may be omitted. For example, detailed explanations of already well-known matters and duplicate explanations for substantially the same configuration may be omitted. This is to avoid unnecessary redundancy of the following description and to facilitate the understanding of those skilled in the art.
　なお、添付図面および以下の説明は、当業者が本開示を十分に理解するために提供されるのであって、これらにより特許請求の範囲に記載の主題を限定することは意図されていない。 It should be noted that the accompanying drawings and the following description are provided for those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims.
（実施の形態）
＜プレイ分析システム＞
　まず、図１を参照しながら、一実施の形態に係るプレイ分析システムの概要について説明する。 (Embodiment)
<Play analysis system>
First, the outline of the play analysis system according to the embodiment will be described with reference to FIG.
　プレイ分析システム１は、球技を撮影したプレイ映像を分析し、当該分析結果に基づくプレイ映像を再生するシステムである。球技に用いられる移動体は、典型的にはボールであるが、バドミントンに用いられるシャトル等であってもよい。本実施の形態では、球技の１つであるバレーボールを例に説明する。ただし、プレイ分析システム１は、卓球、テニス、ビーチバレー、又はバドミントン等の球技全般に適用可能である。すなわち、プレイ分析システム１は、複数人から構成されるチームや個人がアクションを行うことにより点数又は勝敗が決定される球技であって、移動体のラリーが行われるどのような球技にも適用可能である。 The play analysis system 1 is a system that analyzes a play video of a ball game and reproduces a play video based on the analysis result. The moving body used in the ball game is typically a ball, but may be a shuttle or the like used for badminton. In this embodiment, volleyball, which is one of the ball games, will be described as an example. However, the play analysis system 1 can be applied to all ball games such as table tennis, tennis, beach volleyball, and badminton. That is, the play analysis system 1 is a ball game in which points or wins or losses are determined by actions by a team or an individual composed of a plurality of people, and can be applied to any ball game in which a mobile rally is performed. Is.
　プレイ分析システム１は、複数のカメラ３（３Ａ、３Ｂ、３Ｃ、３Ｄ）と、表示装置４と、入力装置５と、プレイ分析装置１００と、を備える。 The play analysis system 1 includes a plurality of cameras 3 (3A, 3B, 3C, 3D), a display device 4, an input device 5, and a play analysis device 100.
　複数のカメラ３は、それぞれ異なる位置に設置される。例えば、図１に示すように、各カメラ３は、高所からコート１０を異なる視点（画角）で撮影できる位置に設置される。なお、図１ではカメラ３が４台であるが、本実施の形態はこれに限られず、カメラ３は、２台以上であれば何台でもよい。２台以上のカメラ３を用いることにより、ボールの３次元位置を算出できる。 The plurality of cameras 3 are installed at different positions. For example, as shown in FIG. 1, each camera 3 is installed at a position where the court 10 can be photographed from a high place with different viewpoints (angles of view). Although the number of cameras 3 is four in FIG. 1, the present embodiment is not limited to this, and the number of cameras 3 may be any number as long as it is two or more. By using two or more cameras 3, the three-dimensional position of the ball can be calculated.
　各カメラ３は、プレイ分析装置１００と有線又は無線を介して通信可能に接続されている。各カメラ３は、球技中の状況を撮影し、撮影画像を生成する。そして、各カメラ３は、複数の撮影画像から動画フレーム２０１（図２参照）を生成する。そして、各カメラ３は、動画フレーム２０１をプレイ分析装置１００へ送信する。動画フレーム２０１は、複数の撮影画像が、例えば、ＭＰ４、Ｈ．２６４、Ｈ．２６５、又は、Ｍｏｔｉｏｎ　ＪＰＥＧ等の規格に基づいて圧縮されたものであってよい。 Each camera 3 is communicably connected to the play analyzer 100 via wire or wireless. Each camera 3 captures the situation during the ball game and generates a captured image. Then, each camera 3 generates a moving image frame 201 (see FIG. 2) from a plurality of captured images. Then, each camera 3 transmits the moving image frame 201 to the play analyzer 100. The moving image frame 201 has a plurality of captured images, for example, MP4, H.A. 264, H. It may be compressed based on a standard such as 265 or Motion JPEG.
　表示装置４は、プレイ分析装置１００と有線又は無線を介して通信可能に接続されており、プレイ分析装置１００から出力される画像を表示する。表示装置４は、例えば、液晶ディスプレイ又は有機ＥＬディスプレイ等である。 The display device 4 is communicably connected to the play analysis device 100 via wire or wireless, and displays an image output from the play analysis device 100. The display device 4 is, for example, a liquid crystal display or an organic EL display.
　入力装置５は、プレイ分析装置１００と有線又は無線を介して通信可能に接続されており、ユーザからの操作を受け付け、その操作情報をプレイ分析装置１００へ送信する。入力装置５は、例えば、キーボード、マウス、マイク及び／又はタッチパネル等である。なお、入力装置５及び表示装置４は一体型の装置であってもよい。 The input device 5 is communicably connected to the play analysis device 100 via wire or wireless, receives an operation from the user, and transmits the operation information to the play analysis device 100. The input device 5 is, for example, a keyboard, a mouse, a microphone and / or a touch panel. The input device 5 and the display device 4 may be integrated devices.
　プレイ分析装置１００は、各カメラ３が撮影した動画フレームを用いて、ボールに対してアクションを行った選手を特定する。バレーボールの場合、ボールに対して行われるアクションには、サーブ、レセプション、ディグ、トス、アタック、及びブロック等がある。以下、ボールに対してアクションを行った選手を、「アクター」と表現する場合がある。また、プレイ分析装置１００は、動画フレームを用いて、バレーボールのラリー区間を検出する。 The play analysis device 100 identifies a player who has performed an action on the ball by using a moving image frame taken by each camera 3. In the case of volleyball, the actions taken on the ball include serve, reception, dig, toss, attack, and block. Hereinafter, a player who has performed an action on the ball may be referred to as an "actor". In addition, the play analyzer 100 detects the rally section of volleyball using the moving image frame.
　なお、本実施の形態では、図１に示すように、コート１０の面の中心点を原点Ｏとし、コート１０の面と平行かつネット１１と平行な軸をＸ軸、コート１０の面と平行かつネット１１と垂直な軸をＹ軸、コート１０の面と垂直な軸をＺ軸とする。図１に示すように、主審１２、副審１３、４人の線審１４（１４Ａ，１４Ｂ，１４Ｃ，１４Ｄ）が位置している場合、Ｘ軸は、主審１２から離れる方向を正方向、主審１２に近づく方向を負方向とする。Ｙ軸は、主審１２から見て左方向を正方向、右方向を負方向とする。Ｚ軸は、コート１０の面から離れる方向を正方向とする。つまりＺ軸の座標ｚは、コート１０の面からの高さに相当する。なお、Ｘ軸に平行な２本のコート１０のラインのうちＹ軸の正方向側のラインを線審１４Ａがジャッジし、負方向側のラインを線審１４Ｂがジャッジする。また、Ｙ軸に平行な２本のコート１０のラインのうちＸ軸の正方向側のラインを線審１４Ｃがジャッジし、負方向側のラインを線審１４Ｄがジャッジする。線審１４は、自身が所持する旗１５を動かして、ジャッジ結果を選手等に伝える。 In the present embodiment, as shown in FIG. 1, the center point of the surface of the coat 10 is the origin O, the axis parallel to the surface of the coat 10 and parallel to the net 11 is the X axis, and parallel to the surface of the coat 10. The axis perpendicular to the net 11 is the Y axis, and the axis perpendicular to the surface of the coat 10 is the Z axis. As shown in FIG. 1, when the referee 12, the assistant referee 13, and the four line referees 14 (14A, 14B, 14C, 14D) are located, the X-axis is in the forward direction away from the referee 12 and to the referee 12. The approaching direction is the negative direction. The Y-axis has a positive direction in the left direction and a negative direction in the right direction when viewed from the referee 12. The Z-axis has a positive direction away from the surface of the coat 10. That is, the Z-axis coordinate z corresponds to the height from the surface of the coat 10. Of the two court 10 lines parallel to the X-axis, the linesmen 14A judges the line on the positive direction side of the Y-axis, and the linesmen 14B judges the line on the negative direction side. Further, the linesmen 14C judges the line on the positive direction side of the X axis among the lines of the two courts 10 parallel to the Y axis, and the linesmen 14D judges the line on the negative direction side. The linesmen 14 moves the flag 15 that they own and informs the players and the like of the judgment result.
＜プレイ分析装置＞
　次に、図２を参照しながら、プレイ分析装置１００について説明する。 <Play analyzer>
Next, the play analyzer 100 will be described with reference to FIG.
　プレイ分析装置１００は、映像受信部１０１と、移動体軌跡検出部１０２と、アクション判定部１０３と、番号認識部１０４と、審判動作検出部１０５と、ラリー区間判定部１０６と、結果出力部１０７と、ＵＩ制御部１０８と、映像制御部１０９と、プレイ情報制御部１１０と、スコア情報制御部１１１と、記憶部１１２とを備える。 The play analysis device 100 includes a video receiving unit 101, a moving body locus detection unit 102, an action determination unit 103, a number recognition unit 104, a referee motion detection unit 105, a rally section determination unit 106, and a result output unit 107. The UI control unit 108, the video control unit 109, the play information control unit 110, the score information control unit 111, and the storage unit 112 are provided.
　映像受信部１０１は、各カメラ３Ａ～３Ｄから送信される動画フレーム２０１を受信し、記憶部１１２に格納する。映像受信部１０１は、ラリー区間情報に対応付けられていないフレームについては破棄する。 The video receiving unit 101 receives the moving image frame 201 transmitted from each of the cameras 3A to 3D and stores it in the storage unit 112. The video receiving unit 101 discards frames that are not associated with the rally section information.
　移動体軌跡検出部１０２は、記憶部１１２に格納された複数の動画フレーム２０１に対して、例えば非特許文献１に開示されている手法を適用し、動画フレーム２０１が撮影された時刻（以下「フレーム時刻」という）におけるボールの３次元位置（ｘ，ｙ，ｚ）及び速度を算出する。そして、移動体軌跡検出部１０２は、フレーム時刻と、ボールの３次元位置及び移動速度とを対応付けてボール軌跡情報２０２を生成し、記憶部１１２に格納する。また、移動体軌跡検出部１０２は、ボールを検出できなかった場合、その旨を表わすボール軌跡情報２０２を生成し、記憶部１１２に格納する。つまり、移動体軌跡検出部１０２は、ボールの移動パターンを検出して、その検出結果に関するボール軌跡情報２０２を記憶部１１２に格納する。 The moving body locus detection unit 102 applies, for example, the method disclosed in Non-Patent Document 1 to a plurality of moving image frames 201 stored in the storage unit 112, and applies the method disclosed in Non-Patent Document 1 to the time when the moving body locus 201 was shot (hereinafter, “” The three-dimensional position (x, y, z) and velocity of the ball at the "frame time") are calculated. Then, the moving body locus detection unit 102 generates the ball locus information 202 in association with the frame time, the three-dimensional position of the ball, and the moving speed, and stores the ball locus information 202 in the storage unit 112. If the moving body locus detection unit 102 cannot detect the ball, the moving body locus detection unit 102 generates ball locus information 202 indicating that fact and stores it in the storage unit 112. That is, the moving body locus detection unit 102 detects the movement pattern of the ball and stores the ball locus information 202 related to the detection result in the storage unit 112.
　アクション判定部１０３は、ボール軌跡情報２０２から、ボールに対するアクションが発生したか否かを判定する。アクション判定部１０３は、アクションが発生したと判定した場合、アクションが発生したフレーム時刻（以下「アクションフレーム時刻」という）と、その発生したアクションの種別とを対応付けてアクション情報２０３を生成し、記憶部１１２に格納する。なお、アクション判定部１０３の詳細については後述する。 The action determination unit 103 determines whether or not an action has occurred on the ball from the ball trajectory information 202. When it is determined that an action has occurred, the action determination unit 103 generates action information 203 by associating the frame time in which the action occurred (hereinafter referred to as "action frame time") with the type of the action in which the action occurred. It is stored in the storage unit 112. The details of the action determination unit 103 will be described later.
　番号認識部１０４は、記憶部１１２に格納されている複数の動画フレーム２０１から、アクターのユニフォーム番号を認識する。そして、番号認識部１０４は、アクションフレーム時刻と、その認識したアクターのユニフォーム番号とを対応付けてアクター情報２０４を生成し、記憶部１１２に格納する。なお、番号認識部１０４の詳細については後述する。 The number recognition unit 104 recognizes the uniform number of the actor from the plurality of moving image frames 201 stored in the storage unit 112. Then, the number recognition unit 104 generates the actor information 204 by associating the action frame time with the recognized uniform number of the actor, and stores the actor information 204 in the storage unit 112. The details of the number recognition unit 104 will be described later.
　審判動作検出部１０５は、ボール軌跡情報２０２の生成に用いられた動画フレーム２０１から、線審１４の旗１５の動きを審判員の動作パターンとして検出する。そして、審判動作検出部１０５は、旗１５の動きがラリー区間の終了を示すか否かを判定し、その判定結果を示す旗動作フラグを生成する。なお、審判動作検出部１０５の詳細については後述する。 The referee motion detection unit 105 detects the movement of the flag 15 of the linesmen 14 as the motion pattern of the referee from the moving image frame 201 used to generate the ball trajectory information 202. Then, the referee motion detection unit 105 determines whether or not the movement of the flag 15 indicates the end of the rally section, and generates a flag motion flag indicating the determination result. The details of the referee motion detection unit 105 will be described later.
　ラリー区間判定部１０６は、ボールの軌跡パターンに基づきラリー区間の開始を判定すると共に、旗動作フラグやユーザからの入力装置５の操作に基づきラリー区間の終了を判定する。そして、ラリー区間判定部１０６は、ボール軌跡情報２０２や旗動作フラグに対応付けられたフレーム時刻と、ラリー区間の開始又は終了を示す情報とを対応付けてラリー区間情報２０５を生成し、記憶部１１２に格納する。また、ラリー区間判定部１０６は、ラリー区間の開始及び終了を判定できなかった場合、ラリー中であると判定する。そして、ラリー区間判定部１０６は、ラリー区間の開始又は終了を示す情報の代わりに、ラリー中を示す情報を含むラリー区間情報２０５を記憶部１１２に格納する。なお、ラリー区間判定部１０６の詳細については後述する。 The rally section determination unit 106 determines the start of the rally section based on the trajectory pattern of the ball, and determines the end of the rally section based on the flag operation flag and the operation of the input device 5 from the user. Then, the rally section determination unit 106 generates the rally section information 205 by associating the frame time associated with the ball locus information 202 and the flag operation flag with the information indicating the start or end of the rally section, and stores the storage unit 106. Store in 112. Further, when the rally section determination unit 106 cannot determine the start and end of the rally section, it determines that the rally is in progress. Then, the rally section determination unit 106 stores the rally section information 205 including the information indicating that the rally is in progress in the storage unit 112 instead of the information indicating the start or end of the rally section. The details of the rally section determination unit 106 will be described later.
　結果出力部１０７は、ボール軌跡情報２０２と、アクション情報２０３と、アクター情報２０４と、ラリー区間情報２０５とを対応付けて、プレイ情報２０６を生成し、記憶部１１２に格納する。なお、プレイ情報２０６は、試合またはセット単位で管理することが望ましく、ユーザの操作により、試合区間（試合開始、終了）やセット区間（セット開始、終了）に関する情報を含むようにするとよい。このような観点から、本実施の形態では、結果出力部１０７は、ユーザからの入力装置５の操作に基づき、セット又は試合の終了を示す終了情報をプレイ情報２０６に登録する。 The result output unit 107 generates play information 206 by associating the ball trajectory information 202, the action information 203, the actor information 204, and the rally section information 205, and stores the play information 206 in the storage unit 112. It is desirable that the play information 206 is managed in units of matches or sets, and it is preferable that the play information 206 includes information regarding the match section (match start, end) and set section (set start, end) by the user's operation. From this point of view, in the present embodiment, the result output unit 107 registers the end information indicating the end of the set or the game in the play information 206 based on the operation of the input device 5 from the user.
　例えば、アクション情報２０３においてアクションフレーム時刻Ｔとアクション種別「アタック」とが対応付けられており、アクター情報２０４においてアクションフレーム時刻Ｔとアクターのユニフォーム番号「１４」とが対応付けられており、ラリー区間情報２０５においてアクションフレーム時刻Ｔとラリー中であることとが対応付けられている場合、結果出力部１０７は、図３に示すようなプレイ情報２０６を生成する。すなわち、結果出力部１０７は、図３に示すように、ボール軌跡情報２０２のフレーム時刻Ｔに対して、アクション種別「アタック」と、アクターのユニフォーム番号「１４」と、ラリー中であることとを対応付けたプレイ情報２０６を生成する。これにより、ユーザ又は他の装置は、プレイ情報２０６を用いて、フレーム時刻Ｔにおいて、ラリー中に、ユニフォーム番号「１４」の選手（アクター）が、３次元位置（ｘＴ，ｙＴ，ｚＴ）のボールに対して、速度「ＳＴ（ｋｍ／ｈ）」の「アタック」を行ったことを知ることができる。 For example, in the action information 203, the action frame time T and the action type “attack” are associated with each other, and in the actor information 204, the action frame time T and the actor uniform number “14” are associated with each other. When the action frame time T and the fact that the rally is in progress are associated with each other in the information 205, the result output unit 107 generates the play information 206 as shown in FIG. That is, as shown in FIG. 3, the result output unit 107 indicates that the action type “attack”, the uniform number “14” of the actor, and the rally are in progress with respect to the frame time T of the ball trajectory information 202. The associated play information 206 is generated. Thus, a user or other device, using the play information 206, the frame time T, in rally, uniform number player "14" (actors) are three-dimensional position (x T, y T, z T ), It can be known that the "attack" of the speed " ST (km / h)" is performed.
　また、例えば、結果出力部１０７は、ボール軌跡情報２０２のフレーム時刻Ｔ－ｔ１に対して、ラリー区間開始であることを対応付けたプレイ情報２０６を生成する。結果出力部１０７は、ボール軌跡情報２０２のフレーム時刻Ｔ＋ｔ２に対して、ラリー区間終了であることを対応付けたプレイ情報２０６を生成する。結果出力部１０７は、ボール軌跡情報２０２のフレーム時刻Ｔ＋ｔ３に対して、ラリー区間終了であることと、セット終了であることとを対応付けたプレイ情報２０６を生成する。結果出力部１０７は、ボール軌跡情報２０２のフレーム時刻Ｔ＋ｔ４に対して、ラリー区間終了であることと、試合終了であることとを対応付けたプレイ情報２０６を生成する。 Further, for example, the result output unit 107 generates play information 206 associated with the start of the rally section with respect to the frame time Tt1 of the ball trajectory information 202. The result output unit 107 generates play information 206 associated with the end of the rally section with respect to the frame time T + t2 of the ball trajectory information 202. The result output unit 107 generates play information 206 in which the end of the rally section and the end of the set are associated with the frame time T + t3 of the ball locus information 202. The result output unit 107 generates play information 206 in which the end of the rally section and the end of the match are associated with the frame time T + t4 of the ball trajectory information 202.
　ＵＩ制御部１０８は、図４に示すようなＵＩ２０００を生成し、表示装置４に表示する。また、ＵＩ制御部１０８は、入力装置５を通じて、ＵＩ２０００に対する操作を受け付ける。 The UI control unit 108 generates the UI 2000 as shown in FIG. 4 and displays it on the display device 4. Further, the UI control unit 108 receives an operation for the UI 2000 through the input device 5.
　映像制御部１０９は、ＵＩ２０００上に、球技のプレイ映像２１００を表示する。プレイ映像２１００は、カメラ３Ａ～３Ｄが撮影中のリアルタイム映像であってもよいし、過去のプレイの録画映像であってもよい。また、映像制御部１０９は、ユーザからの入力装置５の操作に基づく場面のプレイ映像２１００を表示する。なお、映像制御部１０９の詳細については後述する。 The video control unit 109 displays the ball game play video 2100 on the UI2000. The play video 2100 may be a real-time video being shot by the cameras 3A to 3D, or may be a recorded video of past play. Further, the video control unit 109 displays a play video 2100 of a scene based on the operation of the input device 5 from the user. The details of the image control unit 109 will be described later.
　プレイ情報制御部１１０は、図４に示すように、１ラリー中に生成されたプレイ情報２０６の内容を時系列に並べ、プレイ情報リスト２２００としてＵＩ２０００上に表示する。なお、プレイ情報リスト２２００の詳細については後述する。 As shown in FIG. 4, the play information control unit 110 arranges the contents of the play information 206 generated during one rally in chronological order and displays it on the UI 2000 as a play information list 2200. The details of the play information list 2200 will be described later.
　スコア情報制御部１１１は、スコア情報の内容を時系列に並べ、スコア情報リスト２３００としてＵＩ２０００上に表示する。１つのスコア情報には、１ラリー後の自分のチームのスコアと相手のチームのスコアとが含まれている。また、１つのスコア情報には、当該ラリー中に生成された１以上のプレイ情報２０６が対応付けられている。なお、スコア情報リスト２３００の詳細については後述する。 The score information control unit 111 arranges the contents of the score information in chronological order and displays the score information list 2300 on the UI2000. One score information includes the score of one's own team after one rally and the score of the opponent's team. Further, one or more play information 206 generated during the rally is associated with one score information. The details of the score information list 2300 will be described later.
＜アクション判定部の詳細＞
　次に、アクション判定部１０３の詳細について説明する。 <Details of action judgment unit>
Next, the details of the action determination unit 103 will be described.
　アクション判定部１０３は、ボール軌跡情報２０２に含まれるフレーム時刻毎のボールの３次元位置及び速度からボールの軌跡を算出する。そして、アクション判定部１０３は、その算出したボールの軌跡変化が所定の条件に合致した場合、ボールに対するアクションが発生したと判定する。アクション判定部１０３は、アクションが発生したと判定した場合、アクションフレーム時刻と、その発生したアクションの種別とを対応付けてアクション情報２０３を生成し、記憶部１１２に格納する。 The action determination unit 103 calculates the trajectory of the ball from the three-dimensional position and velocity of the ball for each frame time included in the ball trajectory information 202. Then, when the calculated trajectory change of the ball matches a predetermined condition, the action determination unit 103 determines that an action for the ball has occurred. When the action determination unit 103 determines that an action has occurred, the action determination unit 103 generates action information 203 in association with the action frame time and the type of the action that has occurred, and stores the action information 203 in the storage unit 112.
　なお、アクション判定部１０３は、ボールの軌跡変化、ボールの３次元位置及び速度、並びに、スポーツのルール等に基づいて、アクション種別を判定してもよい。バレーボールのアクション種別には、サーブ、レセプション（サーブに対するレシーブ）、ディグ（アタックに対するレシーブ）、トス、アタック、及びブロック等がある。例えば、アクション判定部１０３は、解析開始から初めて検出されたボールの軌跡がＹ軸方向（図１に示すコートの長辺方向）の移動成分を有し、当該Ｙ軸方向のボールの速度成分が所定の範囲内である場合、アクション種別を「サーブ」と判定する。他の例として、アクション判定部１０３は、「サーブ」後にボールの軌跡がＹ軸においてネット１１の存在する座標を跨ぎ、かつ、ボールの３次元位置の変化が下降から上昇に転じた場合（すなわち、Ｚ軸方向の座標の変化がプラスに転じた場合）、アクション種別を「レセプション」と判定する。バレーボールのルール上、「サーブ」を受けるアクションは「レセプション」であるので、このようなルールに基づく判定を行うことにより、「レセプション」と「ディグ」を区別できる。 Note that the action determination unit 103 may determine the action type based on the change in the trajectory of the ball, the three-dimensional position and speed of the ball, the rules of sports, and the like. Volleyball action types include serve, reception (receive against serve), dig (receive against attack), toss, attack, and block. For example, in the action determination unit 103, the trajectory of the ball detected for the first time from the start of analysis has a movement component in the Y-axis direction (the long side direction of the court shown in FIG. 1), and the velocity component of the ball in the Y-axis direction is If it is within the predetermined range, the action type is determined to be "serve". As another example, in the action determination unit 103, when the trajectory of the ball straddles the coordinates where the net 11 exists on the Y axis after "serving", and the change in the three-dimensional position of the ball changes from descending to ascending (that is,). , When the change in coordinates in the Z-axis direction turns positive), the action type is determined to be "reception". According to the rules of volleyball, the action to receive "serve" is "reception", so by making a judgment based on such rules, "reception" and "dig" can be distinguished.
　同様に、アクション判定部１０３は、ボールの軌跡変化や選手の動き等が所定の条件に合致する場合に、スポーツのルールに準じた判定基準によって他のアクション種別を判定する。 Similarly, when the change in the trajectory of the ball, the movement of the player, or the like meets the predetermined conditions, the action determination unit 103 determines another action type according to the determination criteria according to the rules of sports.
　なお、上記では、３次元の情報を用いてアクションの発生を判定しているが、２次元又は１次元の情報を用いてアクションの発生を判定しても良い。例えば、上述した条件で「サーブ」の発生を判定する場合、Ｘ軸方向とＺ軸方向の情報を用いずに、解析開始の時刻とＹ軸方向の情報のみから判定可能である。 In the above, the occurrence of the action is determined using the three-dimensional information, but the occurrence of the action may be determined using the two-dimensional or one-dimensional information. For example, when determining the occurrence of "serve" under the above-mentioned conditions, it is possible to determine from only the analysis start time and the information in the Y-axis direction without using the information in the X-axis direction and the Z-axis direction.
＜番号認識部の詳細＞
　次に、番号認識部１０４の詳細について説明する。 <Details of number recognition unit>
Next, the details of the number recognition unit 104 will be described.
　番号認識部１０４は、複数の動画フレーム２０１の中から、アクション情報２０３に含まれるアクションフレーム時刻に対応する動画フレームと、当該アクションフレーム時刻の近傍のフレーム時刻に対応する動画フレームとを、アクションフレームとして選択する。次に、番号認識部１０４は、アクションフレームから選手（人物）が撮影されている領域（以下「選手領域」という）を検出する。例えば、番号認識部１０４は、競技中の複数の選手画像をディープラーニングによって学習させた選手検出用のモデル（ニューラルネットワーク）を用いて、検出領域内から選手領域を検出する。そして、番号認識部１０４は、アクションフレームから、アクターのユニフォーム番号を認識する。例えば、番号認識部１０４は、複数の選手のユニフォームの数字画像をディープラーニングによって学習させたユニフォーム番号検出用のモデル（ニューラルネットワーク）を用いて、検出領域内から、アクターのユニフォーム番号を認識する。次に、番号認識部１０４は、アクションフレーム時刻と、認識したアクターのユニフォーム番号とを対応付けてアクター情報２０４を生成し、記憶部１１２に格納する。 From the plurality of video frames 201, the number recognition unit 104 selects a video frame corresponding to the action frame time included in the action information 203 and a video frame corresponding to a frame time in the vicinity of the action frame time as an action frame. Select as. Next, the number recognition unit 104 detects an area where a player (person) is photographed (hereinafter referred to as a “player area”) from the action frame. For example, the number recognition unit 104 detects a player area from within the detection area by using a player detection model (neural network) in which a plurality of player images during competition are trained by deep learning. Then, the number recognition unit 104 recognizes the uniform number of the actor from the action frame. For example, the number recognition unit 104 recognizes the uniform number of an actor from within the detection area by using a model (neural network) for detecting uniform numbers obtained by learning numerical images of uniforms of a plurality of athletes by deep learning. Next, the number recognition unit 104 generates the actor information 204 by associating the action frame time with the recognized uniform number of the actor, and stores it in the storage unit 112.
＜審判動作検出部の詳細＞
　次に、審判動作検出部１０５の詳細について説明する。 <Details of referee motion detection unit>
Next, the details of the referee motion detection unit 105 will be described.
　審判動作検出部１０５は、ボール軌跡情報２０２の生成に用いられた動画フレーム２０１から、フレーム時刻における旗１５の３次元位置及び速度を算出する。次に、審判動作検出部１０５は、それらの３次元位置及び移動速度に基づき旗１５の動きがラリーの終了を示すか否かを判定する。なお、ラリー終了を示す旗１５の動きの一例については後述する。そして、審判動作検出部１０５は、ラリー終了の判定結果を示す旗動作フラグを生成する。審判動作検出部１０５が旗動作フラグに対応付けるフレーム時刻は、ラリー終了を示す旗１５の動きが開始されて終了するまでのいずれの時刻であってもよく、例えば、旗１５が静止状態から動作状態に変化したタイミングの時刻であってもよい。 The referee motion detection unit 105 calculates the three-dimensional position and velocity of the flag 15 at the frame time from the moving image frame 201 used to generate the ball trajectory information 202. Next, the referee motion detection unit 105 determines whether or not the movement of the flag 15 indicates the end of the rally based on their three-dimensional positions and moving speeds. An example of the movement of the flag 15 indicating the end of the rally will be described later. Then, the referee motion detection unit 105 generates a flag motion flag indicating the determination result of the end of the rally. The frame time associated with the flag operation flag by the referee operation detection unit 105 may be any time from the start to the end of the movement of the flag 15 indicating the end of the rally. For example, the flag 15 is in an operating state from a stationary state. It may be the time of the timing changed to.
＜ラリー終了を示す旗の動きの一例＞
　次に、図５Ａ～図５Ｄを参照して、ラリー終了を示す旗１５の動きの一例について説明する。同様に、審判動作検出部１０５は、審判員の動き等が所定の条件に合致する場合に、スポーツのルールに準じた判定基準によって、ラリーが終了したと判定する。 <Example of flag movement indicating the end of the rally>
Next, an example of the movement of the flag 15 indicating the end of the rally will be described with reference to FIGS. 5A to 5D. Similarly, the referee motion detection unit 105 determines that the rally has been completed according to the criteria according to the sports rules when the referee's movements and the like meet the predetermined conditions.
　図５Ａは、攻撃のボールがコート１０の外側に落ちたボールアウトの動作パターンを示す。ボールアウトのとき、線審１４は、画像Ｐ１のように下げている旗１５を、画像Ｐ２のように上げる。 FIG. 5A shows a ball-out operation pattern in which the attacking ball falls to the outside of the court 10. At the time of ball-out, the linesmen 14 raises the flag 15 lowered as shown in image P1 as shown in image P2.
　図５Ｂは、攻撃のボールがブロッカーに当たってから床に落ちたブロックアウトの動作パターンを示す。ブロックアウトのとき、線審１４は、画像Ｐ１のように下げている旗１５を、画像Ｐ３のように棒が顔の高さで水平になるように動かす。 FIG. 5B shows a blockout operation pattern in which the attacking ball hits the blocker and then falls to the floor. At the time of blockout, the linesmen 14 moves the flag 15 lowered as shown in the image P1 so that the bar is horizontal at the height of the face as shown in the image P3.
　図５Ｃは、ボールがアンテナに当たったりする等の反則時の動作パターンを示す。反則のとき、線審１４は、画像Ｐ１のように下げている旗１５を、画像Ｐ４のように頭の上で左右に振る。 FIG. 5C shows an operation pattern at the time of a foul such as a ball hitting an antenna. At the time of foul, the linesmen 14 swings the flag 15 lowered as shown in the image P1 to the left and right on the head as shown in the image P4.
　図５Ｄは、攻撃のボールがコート１０の内側に落ちたボールインの動作パターンを示す。ボールインのとき、線審１４は、画像Ｐ１のように下げている旗１５を、画像Ｐ５のように体の外側に移動させた後、体の正面において床を指すように動かす。 FIG. 5D shows a ball-in motion pattern in which the attacking ball falls inside the court 10. At the time of ball-in, the linesmen 14 moves the flag 15 lowered as shown in image P1 to the outside of the body as shown in image P5, and then moves it so as to point to the floor in front of the body.
＜ラリー区間判定部の詳細＞
　次に、ラリー区間判定部１０６の詳細について説明する。 <Details of the rally section judgment unit>
Next, the details of the rally section determination unit 106 will be described.
　ラリー区間判定部１０６は、ボール軌跡情報２０２に基づきサーブの開始動作であるボールの投げ上げが行われたと判定した場合、このボール軌跡情報２０２に対応付けられたフレーム時刻にラリーが開始されたと判定する。そして、ラリー区間判定部１０６は、ラリー開始のフレーム時刻と、ラリー区間開始を示す情報とを対応付けてラリー区間情報２０５を生成し、記憶部１１２に格納する。ラリー開始のフレーム時刻は、ボールの投げ上げが開始されて終了するまでのいずれの時刻であってもよい。なお、ラリー区間判定部は、ボール軌跡情報２０２を用いずに、アクション判定部１０３の「サーブ」という判定結果に基づきラリー区間が開始されたと判定してもよい。 When the rally section determination unit 106 determines that the ball has been thrown up, which is the start operation of the serve, based on the ball trajectory information 202, it determines that the rally has started at the frame time associated with the ball trajectory information 202. To do. Then, the rally section determination unit 106 generates the rally section information 205 by associating the frame time of the rally start with the information indicating the start of the rally section, and stores it in the storage unit 112. The frame time at the start of the rally may be any time from the start to the end of the throwing of the ball. The rally section determination unit may determine that the rally section has started based on the determination result of "serve" of the action determination unit 103 without using the ball trajectory information 202.
　ラリー区間判定部１０６は、審判動作検出部１０５で生成された旗動作フラグがラリー終了を示すものである場合、この旗動作フラグに対応付けられたフレーム時刻にラリーが終了したと判定する。そして、ラリー区間判定部１０６は、ラリー終了のフレーム時刻と、ラリー区間終了を示す情報とを対応付けたラリー区間情報２０５を記憶部１１２に格納する。 When the flag operation flag generated by the referee operation detection unit 105 indicates the end of the rally, the rally section determination unit 106 determines that the rally has ended at the frame time associated with this flag operation flag. Then, the rally section determination unit 106 stores the rally section information 205 in which the frame time of the rally end and the information indicating the end of the rally section are associated with each other in the storage unit 112.
　ラリー区間判定部１０６は、ボール軌跡情報２０２に基づきボールの移動速度が、一定フレーム間、閾速度未満であると判定した場合、このボール軌跡情報２０２に対応付けられたフレーム時刻にラリー区間が終了したと判定する。そして、ラリー区間判定部１０６は、移動速度が閾速度未満になったときのフレーム時刻と、ラリー区間終了を示す情報とを対応付けたラリー区間情報２０５を記憶部１１２に格納する。例えば、反則があったとき、ラリーを継続する必要がなくなり、図６Ａに示すように、トスやレシーブをしようとする選手がボールＢをキャッチしてプレイを止める場合がある。そして、このとき、線審１４がジャッジをしないことがある。このような状況になった場合でも、ラリー区間判定部１０６は、ボールの移動速度に基づきラリー区間の終了を判定できる。なお、閾速度は、例えば、選手がボールを持って歩く速度や、次にサーブを打つ選手にボールを投げるときの速度よりも遅い速度に設定すればよい。 When the rally section determination unit 106 determines that the moving speed of the ball is less than the threshold speed for a certain frame period based on the ball trajectory information 202, the rally section ends at the frame time associated with the ball trajectory information 202. It is judged that it has been done. Then, the rally section determination unit 106 stores the rally section information 205 in which the frame time when the moving speed becomes less than the threshold speed and the information indicating the end of the rally section are associated with each other in the storage unit 112. For example, when there is a foul, it is no longer necessary to continue the rally, and as shown in FIG. 6A, a player who intends to toss or receive may catch the ball B and stop playing. At this time, the linesmen 14 may not judge. Even in such a situation, the rally section determination unit 106 can determine the end of the rally section based on the moving speed of the ball. The threshold speed may be set to, for example, a speed slower than the speed at which the player walks with the ball or the speed at which the player throws the ball at the player who hits the serve next.
　ラリー区間判定部１０６は、ボール軌跡情報２０２に基づきボールの３次元位置が試合の有効範囲外であると判定した場合、このボール軌跡情報２０２に対応付けられたフレーム時刻にラリー区間が終了したと判定する。なお、試合の有効範囲外の位置とは、例えば、観覧席である。そして、ラリー区間判定部１０６は、ボールが観客席に入ったときのフレーム時刻と、ラリー区間終了を示す情報とを対応付けたラリー区間情報２０５を記憶部１１２に格納する。例えば、図６Ｂに示すように、レシーブミスでボールＢが試合の有効範囲外に出たとき、ラリーは終了するが、線審１４がジャッジをしないことがある。ボールが試合の有効範囲外に出るときは、移動速度が閾速度未満になっていない場合があるが、ラリー区間判定部１０６は、ボールの３次元位置に基づきラリー区間の終了を判定できる。 When the rally section determination unit 106 determines that the three-dimensional position of the ball is out of the effective range of the game based on the ball trajectory information 202, the rally section is said to have ended at the frame time associated with the ball trajectory information 202. judge. The position outside the effective range of the game is, for example, a viewing seat. Then, the rally section determination unit 106 stores the rally section information 205 in which the frame time when the ball enters the spectator's seat and the information indicating the end of the rally section are associated with each other in the storage unit 112. For example, as shown in FIG. 6B, when the ball B goes out of the effective range of the match due to a receive miss, the rally ends, but the linesmen 14 may not judge. When the ball goes out of the effective range of the game, the moving speed may not be less than the threshold speed, but the rally section determination unit 106 can determine the end of the rally section based on the three-dimensional position of the ball.
　ラリー区間判定部１０６は、ボール軌跡情報２０２に基づきボールを検出できない時間が閾時間以上継続したと判定した場合、このボール軌跡情報２０２に対応付けられたフレーム時刻にラリー区間が終了したと判定する。そして、ラリー区間判定部１０６は、ボールを検出できない時間が閾時間以上になったときのフレーム時刻と、ラリー区間終了を示す情報とを対応付けたラリー区間情報２０５を記憶部１１２に格納する。例えば、反則があったとき、トスやレシーブをしようとする選手がボールをキャッチして、そのままサービスエリアに移動する際に、他の選手に遮られてボールがカメラ３で撮影できなくなってしまい、移動体軌跡検出部１０２がボールを検出できない場合がある。この場合、ボールの移動速度が閾速度未満になったか否かやボールが試合の有効範囲外に出たか否か、に基づきラリー区間の終了を判定できないが、ラリー区間判定部１０６は、ボールを検出できない時間に基づきラリー区間の終了を判定できる。なお、閾時間は、例えば、ラリー間にボールの遮蔽が発生しうる最大連続フレーム数に基づき設定すればよい。 When the rally section determination unit 106 determines based on the ball trajectory information 202 that the time during which the ball cannot be detected continues for the threshold time or longer, it determines that the rally section has ended at the frame time associated with the ball trajectory information 202. .. Then, the rally section determination unit 106 stores the rally section information 205 in which the frame time when the time during which the ball cannot be detected becomes equal to or longer than the threshold time and the information indicating the end of the rally section are associated with each other in the storage unit 112. For example, when there is a foul, when a player trying to toss or receive catches the ball and moves to the service area as it is, it is blocked by other players and the ball cannot be photographed with the camera 3. The moving body locus detection unit 102 may not be able to detect the ball. In this case, the end of the rally section cannot be determined based on whether the moving speed of the ball is less than the threshold speed or whether the ball is out of the effective range of the game, but the rally section determination unit 106 determines the ball. The end of the rally section can be determined based on the undetectable time. The threshold time may be set based on, for example, the maximum number of continuous frames in which ball shielding can occur during the rally.
　ラリー区間判定部１０６は、ユーザからの入力装置５の操作に基づきラリー区間の終了を登録すると判定した場合、ラリー終了のフレーム時刻と、ラリー区間終了を示す情報とを対応付けたラリー区間情報２０５を記憶部１１２に格納する。ユーザがラリー区間終了の登録操作を行う場合とは、例えば、ユーザが映像の内容からラリーが終了したと判定した場合である。 When the rally section determination unit 106 determines that the end of the rally section is registered based on the operation of the input device 5 from the user, the rally section information 205 that associates the frame time of the rally end with the information indicating the end of the rally section. Is stored in the storage unit 112. The case where the user performs the registration operation for the end of the rally section is, for example, the case where the user determines from the content of the video that the rally has ended.
　また、ラリー区間判定部１０６は、ラリー区間の開始及び終了を判定できず、かつ、ユーザからのラリー区間の終了を登録する旨の操作を認識しなかった場合、旗動作フラグに対応付けられたフレーム時刻にラリーが行われていると判定する。そして、ラリー区間判定部１０６は、ラリー中のフレーム時刻と、ラリー中を示す情報とを対応付けたラリー区間情報２０５を記憶部１１２に格納する。 Further, when the rally section determination unit 106 cannot determine the start and end of the rally section and does not recognize the operation to register the end of the rally section from the user, it is associated with the flag operation flag. It is determined that the rally is being performed at the frame time. Then, the rally section determination unit 106 stores the rally section information 205 in which the frame time during the rally and the information indicating the rally are associated with each other in the storage unit 112.
＜プレイ情報リスト＞
　次に、プレイ情報リスト２２００について説明する。 <Play information list>
Next, the play information list 2200 will be described.
　プレイ情報リスト２２００は、上述のとおり、１ラリー中に生成されたプレイ情報２０６の内容を時系列に並べて表示したものである。プレイ情報リスト２２００は、図４に示すように、プレイ情報２０６毎に、再生ボタン２２０１と、チーム２２０２と、選手番号２２０３と、アクション２２０４とを有する。 As described above, the play information list 2200 displays the contents of the play information 206 generated during one rally in chronological order. As shown in FIG. 4, the play information list 2200 has a play button 2201, a team 2202, a player number 2203, and an action 2204 for each play information 206.
　再生ボタン２２０１は、当該再生ボタン２２０１が対応付けられているプレイ情報２０６に係るプレイ映像２１００を再生するためのボタンである。 The play button 2201 is a button for playing the play video 2100 related to the play information 206 to which the play button 2201 is associated.
　チーム２２０２は、アクション２２０４を行った選手が、自分のチームと相手のチームとの何れであるかを示す。例えば、「ａ」は自分のチーム、「ｅ」は相手のチームを表す。また、チーム２２０２はボタンになっており、プレイ情報制御部１１０は、当該チーム２２０２が押下された場合、修正用に「ａ」と「ｅ」のボタンを表示する。 Team 2202 indicates whether the player who performed action 2204 is his own team or the opponent's team. For example, "a" represents your team and "e" represents your opponent's team. Further, the team 2202 is a button, and when the team 2202 is pressed, the play information control unit 110 displays "a" and "e" buttons for correction.
　選手番号２２０３は、アクション２２０４を行った選手の番号（背番号）を示す。また、選手番号２２０３はボタンになっており、プレイ情報制御部１１０は、当該選手番号２２０３が押下された場合、選手番号の修正用のボタンを表示する。選手番号の修正用のボタンは、数字を入力するためのボタンであってよい。或いは、選手番号の修正用のボタンは、チームに登録されている選手の番号のリストであってもよい。 The player number 2203 indicates the number (uniform number) of the player who performed the action 2204. Further, the player number 2203 is a button, and the play information control unit 110 displays a button for correcting the player number when the player number 2203 is pressed. The button for correcting the player number may be a button for inputting a number. Alternatively, the button for correcting the player number may be a list of player numbers registered in the team.
　アクション２２０４は、アクションの種別を示す。例えば、「Ｓ」はサーブ、「Ｒ」はレセプション、「Ｅ」はトス、「Ａ」はアタック、「Ｄ」はディグ、「Ｂ」はブロックを示す。また、アクション２２０４はボタンになっており、プレイ情報制御部１１０は、当該アクション２２０４が押下されると、修正用に、「Ｓ」、「Ｒ」、「Ｅ」、「Ａ」、「Ｄ」及び「Ｂ」のボタンを表示する。 Action 2204 indicates the type of action. For example, "S" is a serve, "R" is a reception, "E" is a toss, "A" is an attack, "D" is a dig, and "B" is a block. Further, the action 2204 is a button, and when the action 2204 is pressed, the play information control unit 110 "S", "R", "E", "A", "D" for correction. And the "B" button is displayed.
　例えば、図４に示すプレイ情報リスト２２００の１行目は、自分のチーム（「ａ」）の背番号「６」の選手が、レセプション「Ｒ」を行ったことを示す。 For example, the first line of the play information list 2200 shown in FIG. 4 indicates that the player with the uniform number "6" of his team ("a") has performed the reception "R".
＜スコア情報リスト＞
　次に、スコア情報リスト２３００について説明する。 <Score information list>
Next, the score information list 2300 will be described.
　スコア情報リスト２３００は、上述のとおり、１セットにおける各ラリーに対応するスコア情報の内容を時系列に並べて表示したものである。スコア情報リスト２３００は、図４に示すように、スコア情報毎に、再生ボタン２３０１と、スコア２３０２と、チェックボックス２３０３とを有する。 As described above, the score information list 2300 displays the contents of the score information corresponding to each rally in one set in chronological order. As shown in FIG. 4, the score information list 2300 has a play button 2301, a score 2302, and a check box 2303 for each score information.
　再生ボタン２３０１は、当該再生ボタン２３０１が対応付けられているスコア情報に係るプレイ映像２１００を再生するためのボタンである。 The play button 2301 is a button for playing the play video 2100 related to the score information associated with the play button 2301.
　スコア２３０２は、ラリー後の自分のチームと相手のチームのスコアを示す。また、スコア２３０２は選択（クリック）可能になっており、当該スコア２３０２が選択された場合、スコア情報制御部１１１は、当該スコア２３０２のラリー（スコア情報）に対応付けられているプレイ情報リスト２２００を表示する。 Score 2302 indicates the score of one's own team and the opponent's team after the rally. Further, the score 2302 can be selected (clicked), and when the score 2302 is selected, the score information control unit 111 uses the play information list 2200 associated with the rally (score information) of the score 2302. Is displayed.
　チェックボックス２３０３は、ユーザがスコア単位でラリーに対するチェックをオン及びオフするためのものである。例えば、試合中などのリアルタイム状況では、プレイ情報リスト２２００の内容（チーム２２０２、選手番号２２０３、アクション２２０４）に誤りを発見しても、次のラリーがすぐに始まってしまうため、或るラリーを後で再検証（修正）したい場合に備えて、ユーザは、当該ラリーに対応するチェックボックス２３０３をオン（識別情報の入力）にして、プレイ情報リスト２２００の内容が暫定状態にあることを識別できるようにする。これにより、ユーザは、後で、チェックボックス２３０３をオンにしたラリーをまとめて再検証できる。例えば、スコア情報制御部１１１が、ユーザからの指示に応じて、チェックボックス２３０３がオンになっているラリーのみを限定して表示することにより、ユーザは、効率的に修正作業を行える。また、チェックボックス２３０３の数は、１つに限られない。 Check box 2303 is for the user to turn on and off the check for the rally on a score-by-score basis. For example, in a real-time situation such as during a match, even if an error is found in the contents of the play information list 2200 (team 2202, player number 2203, action 2204), the next rally will start immediately, so a certain rally The user can identify that the contents of the play information list 2200 are in the provisional state by selecting the check box 2303 corresponding to the rally (input of identification information) in case of wanting to re-verify (correct) later. To do so. This allows the user to later re-verify the rally with the check box 2303 selected. For example, the score information control unit 111 limits and displays only the rallies in which the check box 2303 is selected in response to the instruction from the user, so that the user can efficiently perform the correction work. Further, the number of check boxes 2303 is not limited to one.
＜映像再生部の詳細＞
　次に、映像制御部１０９の詳細について説明する。 <Details of the video playback section>
Next, the details of the video control unit 109 will be described.
　映像制御部１０９は、ユーザがプレイ情報リスト２２００の再生ボタン２２０１の押下すると、当該再生ボタン２２０１に対応するプレイ情報２０６のフレーム時刻の近傍のプレイ映像２１００を再生する。これにより、ユーザは、プレイ情報２０６に対応する選手のプレイを簡単に確認することができる。 When the user presses the play button 2201 of the play information list 2200, the video control unit 109 plays the play video 2100 in the vicinity of the frame time of the play information 206 corresponding to the play button 2201. As a result, the user can easily confirm the play of the player corresponding to the play information 206.
　また、映像制御部１０９は、ユーザがスコア情報リスト２３００の再生ボタン２３０１の押下すると、当該再生ボタン２３０１に対応するラリーのプレイ映像２１００を再生する。このとき、まず、映像制御部１０９は、押下された再生ボタン２３０１のスコアに対応付けられたラリーを特定する。次に、映像制御部１０９は、その特定したラリー区間の開始を示すラリー区間情報２０５を含むプレイ情報２０６を特定し、その特定したプレイ情報２０６のフレーム時刻をラリー区間の開始時刻として認識する。また、映像制御部１０９は、ラリー区間の終了を示すラリー区間情報２０５を含むプレイ情報２０６を特定し、その特定したプレイ情報２０６のフレーム時刻をラリーの終了時刻として認識する。そして、映像制御部１０９は、ラリー区間の開始時刻から終了時刻までの間の動画フレーム２０１を取得して、ラリー区間のプレイ映像２１００を再生する。つまり、映像制御部１０９は、ラリー区間判定部１０６の判定結果にしたがって、プレイ映像２１００からラリー区間のプレイ映像２１００を抽出する。 Further, when the user presses the play button 2301 of the score information list 2300, the video control unit 109 plays the play video 2100 of the rally corresponding to the play button 2301. At this time, first, the video control unit 109 identifies the rally associated with the score of the pressed play button 2301. Next, the video control unit 109 identifies the play information 206 including the rally section information 205 indicating the start of the specified rally section, and recognizes the frame time of the specified play information 206 as the start time of the rally section. Further, the video control unit 109 identifies the play information 206 including the rally section information 205 indicating the end of the rally section, and recognizes the frame time of the specified play information 206 as the end time of the rally. Then, the video control unit 109 acquires the moving image frame 201 between the start time and the end time of the rally section, and reproduces the play video 2100 of the rally section. That is, the video control unit 109 extracts the play video 2100 of the rally section from the play video 2100 according to the determination result of the rally section determination unit 106.
＜プレイ情報生成処理のフロー＞
　次に、図７～１０に示すフローチャートを参照しながら、プレイ分析装置１００におけるプレイ情報生成処理のフローについて説明する。なお、本実施の形態では、撮影中にプレイ情報２０６を生成する場合について説明する。 <Flow of play information generation processing>
Next, the flow of the play information generation process in the play analyzer 100 will be described with reference to the flowcharts shown in FIGS. 7 to 10. In this embodiment, a case where the play information 206 is generated during shooting will be described.
　図７に示すように、映像受信部１０１は、カメラ３Ａ～３Ｄから送信される動画フレーム２０１を受信し、記憶部１１２に格納する（Ｓ１）。移動体軌跡検出部１０２は、記憶部１１２に格納された複数（所定数）の動画フレーム２０１を取得する（Ｓ２）。移動体軌跡検出部１０２は、動画フレーム２０１からボールを検出できたか否かを判定する（Ｓ３）。移動体軌跡検出部１０２がボールを検出できなかったと判定した場合（Ｓ３：ＮＯ）、映像受信部１０１は、ボール検出に用いた動画フレーム２０１を記憶部１１２から破棄し（Ｓ４）、Ｓ１に戻る。 As shown in FIG. 7, the video receiving unit 101 receives the moving image frame 201 transmitted from the cameras 3A to 3D and stores it in the storage unit 112 (S1). The mobile locus detection unit 102 acquires a plurality (predetermined number) of moving image frames 201 stored in the storage unit 112 (S2). The moving body locus detection unit 102 determines whether or not the ball can be detected from the moving image frame 201 (S3). When the moving body locus detection unit 102 determines that the ball could not be detected (S3: NO), the video receiving unit 101 discards the moving image frame 201 used for ball detection from the storage unit 112 (S4) and returns to S1. ..
　一方、移動体軌跡検出部１０２は、ボールを検出できた場合（Ｓ３：ＹＥＳ）、各フレーム時刻におけるボールの３次元位置及び移動速度を算出し、ボール軌跡情報２０２を生成する（Ｓ５）。移動体軌跡検出部１０２でボール軌跡情報２０２が生成されると、ラリー区間判定部１０６は、ボール軌跡情報２０２に基づきラリー区間が開始されたか否かを判定する（Ｓ６）。ラリー区間判定部１０６は、ラリー区間が開始されていないと判定した場合（Ｓ６：ＮＯ）、ボール軌跡情報２０２を破棄する（Ｓ７）。その後、Ｓ４の処理が行われて、Ｓ１に戻る。 On the other hand, when the moving body locus detection unit 102 can detect the ball (S3: YES), the moving body locus detection unit 102 calculates the three-dimensional position and the moving speed of the ball at each frame time, and generates the ball locus information 202 (S5). When the ball locus information 202 is generated by the moving body locus detection unit 102, the rally section determination unit 106 determines whether or not the rally section has been started based on the ball locus information 202 (S6). When the rally section determination unit 106 determines that the rally section has not started (S6: NO), the ball trajectory information 202 is discarded (S7). After that, the processing of S4 is performed, and the process returns to S1.
　一方、ラリー区間判定部１０６は、ラリー区間が開始されたと判定した場合（Ｓ６：ＹＥＳ）、ラリー区間の開始を示すラリー区間情報２０５を生成する（Ｓ８）。次に、図８に示すように、結果出力部１０７は、ラリー区間開始を示すプレイ情報２０６を生成済みか否かを判定する（Ｓ９）。結果出力部１０７は、ラリー区間開始を示すプレイ情報２０６を生成済みでないと判定した場合（Ｓ９：ＮＯ）、図９に示すように、プレイ情報２０６を生成する（Ｓ１０）。このようにＳ９でＮＯと判定された後にＳ１０の処理を行う場合、結果出力部１０７は、ボール軌跡情報２０２と、ラリー区間情報２０５とを含むプレイ情報２０６を生成する。 On the other hand, when the rally section determination unit 106 determines that the rally section has started (S6: YES), the rally section determination unit 106 generates rally section information 205 indicating the start of the rally section (S8). Next, as shown in FIG. 8, the result output unit 107 determines whether or not the play information 206 indicating the start of the rally section has been generated (S9). When the result output unit 107 determines that the play information 206 indicating the start of the rally section has not been generated (S9: NO), the result output unit 107 generates the play information 206 as shown in FIG. 9 (S10). When the processing of S10 is performed after the determination of NO in S9 in this way, the result output unit 107 generates play information 206 including ball trajectory information 202 and rally section information 205.
　一方、図８に示すように、結果出力部１０７がラリー区間開始を示すプレイ情報２０６を生成済みであると判定した場合（Ｓ９：ＹＥＳ）、映像受信部１０１は、Ｓ１の処理と同様に、カメラ３Ａ～３Ｄからの動画フレーム２０１を記憶部１１２に格納する（Ｓ１１）。次に、移動体軌跡検出部１０２は、Ｓ２及びＳ５と同様の処理を行う。すなわち、移動体軌跡検出部１０２は、記憶部１１２から所定数の動画フレーム２０１を取得し（Ｓ１２）、ボール軌跡情報２０２を生成する（Ｓ１３）。 On the other hand, as shown in FIG. 8, when the result output unit 107 determines that the play information 206 indicating the start of the rally section has been generated (S9: YES), the video receiving unit 101 performs the same process as in S1. The moving image frames 201 from the cameras 3A to 3D are stored in the storage unit 112 (S11). Next, the moving body locus detection unit 102 performs the same processing as in S2 and S5. That is, the moving body locus detection unit 102 acquires a predetermined number of moving image frames 201 from the storage unit 112 (S12) and generates the ball locus information 202 (S13).
　アクション判定部１０３は、ボール軌跡情報２０２から、ボールに対するアクションが発生したか否かを判定する（Ｓ１４）。アクション判定部１０３がアクションが未発生と判定した場合（Ｓ１４：ＮＯ）、図９に示すように、結果出力部１０７は、Ｓ１０の処理（プレイ情報２０６の生成処理）を行う。このようにＳ１４でＮＯと判定された後にＳ１０の処理を行う場合、結果出力部１０７は、ボール軌跡情報２０２のみを含むプレイ情報２０６を生成する。一方、図８に示すように、アクション判定部１０３は、アクションが発生したと判定した場合（Ｓ１４：ＹＥＳ）、アクション情報２０３を生成する（Ｓ１５）。次に、番号認識部１０４は、アクター情報２０４を生成する（Ｓ１６）。そして、審判動作検出部１０５は、ボール軌跡情報２０２の生成に用いられた動画フレーム２０１に基づき線審１４の旗１５の動きを検出し、その動きがラリー区間の終了を示すか否かに関する旗動作フラグを生成する（Ｓ１７）。 The action determination unit 103 determines whether or not an action has occurred on the ball from the ball trajectory information 202 (S14). When the action determination unit 103 determines that the action has not occurred (S14: NO), the result output unit 107 performs the process of S10 (the process of generating the play information 206) as shown in FIG. When the process of S10 is performed after the determination of NO in S14 is performed in this way, the result output unit 107 generates play information 206 including only the ball trajectory information 202. On the other hand, as shown in FIG. 8, when the action determination unit 103 determines that an action has occurred (S14: YES), the action determination unit 103 generates action information 203 (S15). Next, the number recognition unit 104 generates the actor information 204 (S16). Then, the referee motion detection unit 105 detects the movement of the flag 15 of the line umpire 14 based on the moving image frame 201 used to generate the ball trajectory information 202, and the flag motion relating to whether or not the movement indicates the end of the rally section. Generate a flag (S17).
　その後、図９に示すように、ラリー区間判定部１０６は、ラリー区間の終了判定処理を行う（Ｓ１８）。なお、ラリー区間の終了判定処理の詳細については後述する。次に、ラリー区間判定部１０６は、ラリー区間の終了判定処理において、ラリーの終了を判定できたか否かを判定する（Ｓ１９）。ラリー区間判定部１０６は、ラリー区間の終了を判定できた場合（Ｓ１９：ＹＥＳ）、ラリー区間の終了を示すラリー区間情報２０５を生成する（Ｓ２０）。そして、結果出力部１０７は、Ｓ１０の処理（プレイ情報２０６の生成処理）を行う。このようにＳ１９でＹＥＳと判定された後にＳ２０の処理を行う場合、結果出力部１０７は、ボール軌跡情報２０２と、ラリー区間情報２０５とを含むプレイ情報２０６を生成する。 After that, as shown in FIG. 9, the rally section determination unit 106 performs the end determination process of the rally section (S18). The details of the end determination process of the rally section will be described later. Next, the rally section determination unit 106 determines whether or not the end of the rally could be determined in the end determination process of the rally section (S19). When the rally section determination unit 106 can determine the end of the rally section (S19: YES), the rally section determination unit 106 generates rally section information 205 indicating the end of the rally section (S20). Then, the result output unit 107 performs the processing of S10 (the processing of generating the play information 206). When the processing of S20 is performed after the determination of YES in S19 in this way, the result output unit 107 generates play information 206 including ball trajectory information 202 and rally section information 205.
　ラリー区間判定部１０６は、ラリー区間の終了を判定できなかった場合（Ｓ１９：ＮＯ）、ユーザからのラリー区間の終了登録操作があったか否かを判定する（Ｓ２１）。ラリー区間判定部１０６は、終了登録操作があったと判定した場合（Ｓ２１：ＹＥＳ）、Ｓ２０の処理（ラリー区間の終了を示すラリー区間情報２０５の生成処理）を行う。そして、結果出力部１０７は、Ｓ１０の処理（プレイ情報２０６の生成処理）を行う。このようにＳ２１でＹＥＳと判定された後にＳ１０の処理を行う場合、結果出力部１０７は、Ｓ１９でＹＥＳと判定された場合と同様に、各情報２０２，２０５を含むプレイ情報２０６を生成する。 When the rally section determination unit 106 cannot determine the end of the rally section (S19: NO), the rally section determination unit 106 determines whether or not there has been a rally section end registration operation from the user (S21). When it is determined that the end registration operation has been performed (S21: YES), the rally section determination unit 106 performs the process of S20 (the process of generating the rally section information 205 indicating the end of the rally section). Then, the result output unit 107 performs the processing of S10 (the processing of generating the play information 206). When the processing of S10 is performed after the determination of YES in S21 in this way, the result output unit 107 generates play information 206 including the respective information 202 and 205, as in the case of the determination of YES in S19.
　一方、ラリー区間判定部１０６は、ラリー区間の終了登録操作がなかったと判定した場合（Ｓ２１：ＮＯ）、ラリー中を示すラリー区間情報２０５を生成する（Ｓ２２）。そして、結果出力部１０７は、Ｓ１０の処理（プレイ情報２０６の生成処理）を行う。このようにＳ２２の処理後にＳ１０の処理を行う場合、結果出力部１０７は、ボール軌跡情報２０２と、アクション情報２０３と、アクター情報２０４と、ラリー区間情報２０５とを含むプレイ情報２０６を生成する。 On the other hand, when it is determined that the rally section end registration operation has not been performed (S21: NO), the rally section determination unit 106 generates rally section information 205 indicating that the rally is in progress (S22). Then, the result output unit 107 performs the processing of S10 (the processing of generating the play information 206). When the processing of S10 is performed after the processing of S22 in this way, the result output unit 107 generates play information 206 including ball trajectory information 202, action information 203, actor information 204, and rally section information 205.
　結果出力部１０７は、Ｓ１０の処理を行うと、図１０に示すように、ラリー区間が終了したか否かを判定する（Ｓ２３）。結果出力部１０７は、ラリー区間が終了していないと判定した場合（Ｓ２３：ＮＯ）、つまり、Ｓ１０で生成したプレイ情報２０６がラリー区間開始又はラリー中を表す場合、Ｓ９の処理を行う。一方、結果出力部１０７は、ラリー区間が終了したと判定した場合（Ｓ２３：ＹＥＳ）、ユーザからの試合終了の登録操作があったか否かを判定する（Ｓ２４）。結果出力部１０７は、試合終了登録操作があったと判定した場合（Ｓ２４：ＹＥＳ）、プレイ情報２０６に試合終了を登録する（Ｓ２５）。そして、プレイ情報生成処理が終了する。 When the result output unit 107 performs the processing of S10, as shown in FIG. 10, it determines whether or not the rally section has ended (S23). The result output unit 107 performs the process of S9 when it is determined that the rally section has not ended (S23: NO), that is, when the play information 206 generated in S10 indicates the start of the rally section or the middle of the rally. On the other hand, when the result output unit 107 determines that the rally section has ended (S23: YES), the result output unit 107 determines whether or not the user has registered the end of the match (S24). When the result output unit 107 determines that the match end registration operation has been performed (S24: YES), the result output unit 107 registers the match end in the play information 206 (S25). Then, the play information generation process ends.
　一方、結果出力部１０７は、試合終了登録操作がなかったと判定した場合（Ｓ２４：ＮＯ）、ユーザからのセット終了の登録操作があったか否かを判定する（Ｓ２６）。結果出力部１０７は、セット終了登録操作があったと判定した場合（Ｓ２６：ＹＥＳ）、プレイ情報２０６にセット終了を登録する（Ｓ２７）。そして、Ｓ１に戻る。一方、結果出力部１０７は、セット終了登録操作がなかったと判定した場合（Ｓ２６：ＮＯ）、Ｓ２７の処理を行わずに、Ｓ１に戻る。 On the other hand, when it is determined that the match end registration operation has not been performed (S24: NO), the result output unit 107 determines whether or not there has been a set end registration operation from the user (S26). When the result output unit 107 determines that the set end registration operation has been performed (S26: YES), the result output unit 107 registers the set end in the play information 206 (S27). Then, it returns to S1. On the other hand, when the result output unit 107 determines that the set end registration operation has not been performed (S26: NO), the result output unit 107 returns to S1 without performing the process of S27.
＜ラリー区間終了判定処理のフロー＞
　次に、図１１に示すフローチャートを参照しながら、プレイ分析装置１００におけるラリー区間終了判定処理のフローについて説明する。 <Flow of rally section end judgment processing>
Next, the flow of the rally section end determination process in the play analyzer 100 will be described with reference to the flowchart shown in FIG.
　ラリー区間判定部１０６は、旗動作フラグの生成処理（Ｓ１７）を行った後、図１１に示すように、旗動作フラグがラリー終了を示しているか否かを判定する（Ｓ３１）。ラリー区間判定部１０６は、旗動作フラグがラリー終了を示していると判定した場合（Ｓ３１：ＹＥＳ）、ラリー区間が終了したと判定する（Ｓ３２）。そして、ラリー区間終了判定処理が終了し、フローは、図９に示すＳ１９に進む。 After performing the flag operation flag generation process (S17), the rally section determination unit 106 determines whether or not the flag operation flag indicates the end of the rally (S31), as shown in FIG. When the rally section determination unit 106 determines that the flag operation flag indicates the end of the rally (S31: YES), the rally section determination unit 106 determines that the rally section has ended (S32). Then, the rally section end determination process is completed, and the flow proceeds to S19 shown in FIG.
　ラリー区間判定部１０６は、旗動作フラグがラリー終了を示していないと判定した場合（Ｓ３１：ＮＯ）、ボール軌跡情報２０２に基づきボールの移動速度が閾速度未満か否かを判定する（Ｓ３３）。ラリー区間判定部１０６は、ボールの移動速度が閾速度未満であると判定した場合（Ｓ３３：ＹＥＳ）、Ｓ３２に進む。 When the rally section determination unit 106 determines that the flag operation flag does not indicate the end of the rally (S31: NO), the rally section determination unit 106 determines whether or not the moving speed of the ball is less than the threshold speed based on the ball trajectory information 202 (S33). .. When the rally section determination unit 106 determines that the moving speed of the ball is less than the threshold speed (S33: YES), the rally section determination unit 106 proceeds to S32.
　ラリー区間判定部１０６は、ボールの移動速度が閾速度未満でないと判定した場合（Ｓ３３：ＮＯ）、ボール軌跡情報２０２に基づきボールの位置が試合の有効範囲外であるか否かを判定する（Ｓ３４）。ラリー区間判定部１０６は、ボールの位置が試合の有効範囲外であると判定した場合（Ｓ３４：ＹＥＳ）、Ｓ３２に進む。 When the rally section determination unit 106 determines that the moving speed of the ball is not less than the threshold speed (S33: NO), the rally section determination unit 106 determines whether or not the position of the ball is out of the effective range of the game based on the ball trajectory information 202 (S33: NO). S34). When the rally section determination unit 106 determines that the position of the ball is out of the effective range of the match (S34: YES), the rally section determination unit 106 proceeds to S32.
　ラリー区間判定部１０６は、ボールの位置が試合の有効範囲外でないと判定した場合（Ｓ３４：ＮＯ）、ボール軌跡情報２０２に基づきボールを検出できない時間が閾時間以上継続したか否かを判定する（Ｓ３５）。ラリー区間判定部１０６は、ボールを検出できない時間が閾時間以上継続したと判定した場合（Ｓ３５：ＹＥＳ）、Ｓ３２に進む。 When the rally section determination unit 106 determines that the position of the ball is not outside the effective range of the game (S34: NO), the rally section determination unit 106 determines whether or not the time during which the ball cannot be detected continues for the threshold time or longer based on the ball trajectory information 202. (S35). When the rally section determination unit 106 determines that the time during which the ball cannot be detected continues for the threshold time or longer (S35: YES), the process proceeds to S32.
　ラリー区間判定部１０６は、ボールを検出できない時間が閾時間以上継続していないと判定した場合（Ｓ３５：ＮＯ）、ラリー区間終了判定処理を終了する。そして、フローは、図９に示すＳ１９に進む。 When the rally section determination unit 106 determines that the time during which the ball cannot be detected has not continued for the threshold time or longer (S35: NO), the rally section end determination process ends. Then, the flow proceeds to S19 shown in FIG.
＜実施形態のまとめ＞
　本実施の形態では、プレイ分析装置１００は、プレイ映像２１００を用いて、線審１４の動作パターンを検出する。次に、プレイ分析装置１００は、プレイ映像２１００に基づきラリー区間の開始を判定すると共に、線審１４の動作パターンに基づきラリー区間の終了を判定する。 <Summary of Embodiment>
In the present embodiment, the play analyzer 100 uses the play video 2100 to detect the operation pattern of the linesmen 14. Next, the play analyzer 100 determines the start of the rally section based on the play video 2100, and determines the end of the rally section based on the operation pattern of the linesmen 14.
　これにより、プレイ分析装置１００は、ラリー区間の開始及び終了の判定結果に基づきプレイ映像２１００から得点ごとのラリー区間のプレイ映像２１００を容易に取得できる。また、バレーボールにおいて、プレイ映像２１００からボールが床に落ちたことを判定することは困難であり、床に落ちた後もプレイが続けられると、この続けられたプレイもラリー中と判定されるおそれがある。しかし、プレイ分析装置１００は、線審１４の動作パターンに基づきラリー区間の終了を判定するため、ボールが床に落ちたか否かの判定が困難な状況においても、適切にラリーの終了を判定できる。 As a result, the play analyzer 100 can easily acquire the play video 2100 of the rally section for each score from the play video 2100 based on the determination result of the start and end of the rally section. Further, in volleyball, it is difficult to determine from the play video 2100 that the ball has fallen to the floor, and if the play is continued even after the ball has fallen to the floor, this continued play may be determined to be in the rally. There is. However, since the play analyzer 100 determines the end of the rally section based on the operation pattern of the linesmen 14, it can appropriately determine the end of the rally even in a situation where it is difficult to determine whether or not the ball has fallen to the floor.
　また、本実施の形態では、プレイ分析装置１００は、線審１４の旗１５の動作パターンに基づきラリー区間の終了を判定する。 Further, in the present embodiment, the play analyzer 100 determines the end of the rally section based on the operation pattern of the flag 15 of the linesmen 14.
　これにより、プレイ分析装置１００は、外観が目立つ旗１５に基づきラリー区間の終了を容易に判定できる。また、プレイ分析装置１００は、４人の線審１４の旗１５の動きパターンを検出しているため、それぞれ１人である主審１２や副審１３の動きパターンを検出する場合と比べて、判定精度を向上できる。 As a result, the play analyzer 100 can easily determine the end of the rally section based on the flag 15 whose appearance is conspicuous. Further, since the play analyzer 100 detects the movement patterns of the flags 15 of the four line referees 14, the determination accuracy is improved as compared with the case of detecting the movement patterns of the referee 12 and the assistant referee 13 who are one person each. Can be improved.
（実施の形態の変形例）
　以上、本開示の具体例を詳細に説明したが、これらは例示にすぎず、以上に例示した具体例を様々に変形、変更してもよい。 (Modified example of the embodiment)
Although the specific examples of the present disclosure have been described in detail above, these are merely examples, and the specific examples illustrated above may be variously modified or changed.
　例えば、分析対象のスポーツとして、卓球、テニス、ビーチバレー、又はバドミントン等のラリーが行われる球技全般を適用してもよい。また、プレイ分析装置１００は、撮影終了後にプレイ情報２０６の生成処理を行ってもよい。 For example, as a sport to be analyzed, general ball games in which rallies such as table tennis, tennis, beach volleyball, or badminton are performed may be applied. Further, the play analysis device 100 may perform a process of generating play information 206 after the shooting is completed.
　また、プレイ分析装置１００は、線審１４に代えて、又は、線審１４と共に、主審１２又は副審１３のうち少なくとも１人の動作パターンに基づきラリー区間終了を判定してもよい。プレイ分析装置１００は、旗１５の移動速度に代えて、又は、移動速度と共に、旗１５の移動量に基づきラリー区間終了を判定してもよい。プレイ分析装置１００は、ボールの移動速度及び位置のうち少なくとも一方に代えて、又は、当該少なくとも一方と共に、ボールの移動量に基づきラリー区間終了を判定してもよい。 Further, the play analyzer 100 may determine the end of the rally section instead of the linesmen 14 or together with the linesmen 14 based on the operation pattern of at least one of the referee 12 or the assistant referee 13. The play analyzer 100 may determine the end of the rally section based on the movement amount of the flag 15 instead of the movement speed of the flag 15 or together with the movement speed. The play analyzer 100 may determine the end of the rally section based on the amount of movement of the ball instead of or together with at least one of the moving speed and position of the ball.
　また、プレイ分析装置１００は、ボールの軌跡パターンに代えて、又は、ボールの軌跡パターンと共に、主審１２又は副審１３のうち少なくとも１人の動作パターンに基づきラリー区間開始を判定してもよい。 Further, the play analyzer 100 may determine the start of the rally section instead of the trajectory pattern of the ball or based on the motion pattern of at least one of the referee 12 or the assistant referee 13 together with the trajectory pattern of the ball.
　また、プレイ情報２０６は、終了情報を含まなくてもよいし、終了情報に加えて又はこれに代えてセット及び試合のうち少なくとも一方の開始を示す開始情報を含んでもよい。終了情報は、セット終了のみ又は試合終了のみを示してもよい。 Further, the play information 206 may not include the end information, or may include start information indicating the start of at least one of the set and the match in addition to or instead of the end information. The end information may indicate only the end of the set or only the end of the match.
　また、プレイ分析装置１００は、ラリー区間の終了登録操作があったか否かの判定処理（Ｓ２１）と、ボールの移動速度が閾速度未満か否かの判定処理（Ｓ３３）と、ボールの位置が試合の有効範囲外であるか否かの判定処理（Ｓ３４）と、ボールを検出できない時間が閾時間以上継続したか否かかの判定処理（Ｓ３５）とのうち少なくとも１つの処理を行わなくてもよい。 Further, the play analysis device 100 determines whether or not there is an end registration operation of the rally section (S21), determines whether or not the moving speed of the ball is less than the threshold speed (S33), and the position of the ball is a match. Even if at least one of the process of determining whether or not the ball is out of the effective range (S34) and the process of determining whether or not the time during which the ball cannot be detected continues for the threshold time or longer (S35) is not performed. Good.
　また、プレイ分析装置１００は、上記実施の形態では、ボール検出を基準にしてラリー区間開始を判定したが、図１２に示すように、サーブのアクション検出を基準にしてラリー区間開始を判定してもよい。この場合、まず、プレイ分析装置１００は、Ｓ１及びＳ２の処理後、Ｓ５の処理（ボール軌跡情報２０２の生成処理）を行う。次に、アクション判定部１０３は、サーブのアクションが発生したか否かを判定する（Ｓ４１）。このサーブのアクションが発生したか否かの判定は、ボール軌跡情報２０２のボールの軌跡に基づき行ってもよいし、動画フレームの選手の動きに基づき行ってもよい。アクション判定部１０３がサーブのアクションが発生していないと判定した場合（Ｓ４１：ＮＯ）、プレイ分析装置１００は、Ｓ７及びＳ４の処理を行う。 Further, in the above embodiment, the play analyzer 100 determines the start of the rally section based on the ball detection, but as shown in FIG. 12, the play analyzer 100 determines the start of the rally section based on the action detection of the serve. May be good. In this case, first, the play analyzer 100 performs the processing of S5 (the processing of generating the ball locus information 202) after the processing of S1 and S2. Next, the action determination unit 103 determines whether or not the serve action has occurred (S41). Whether or not the action of this serve has occurred may be determined based on the trajectory of the ball in the ball trajectory information 202, or may be determined based on the movement of the player in the moving image frame. When the action determination unit 103 determines that the serve action has not occurred (S41: NO), the play analyzer 100 performs the processes of S7 and S4.
　一方、アクション判定部１０３がサーブのアクションが発生したと判定した場合（Ｓ４１：ＹＥＳ）、ラリー区間判定部１０６は、Ｓ８の処理（ラリー区間の開始を示すラリー区間情報２０５の生成処理）を行う。そして、プレイ分析装置１００は、Ｓ１５及びＳ１６と同様の処理（アクション情報２０３の生成処理（Ｓ４２）及びアクター情報２０４の生成処理（Ｓ４３））を行った後、図８に示すＳ９の処理を行う。その後、生成されるラリー区間開始を示すプレイ情報２０６は、ボール軌跡情報２０２と、サーブであることを示すアクション情報２０３と、アクター情報２０４と、ラリー区間情報２０５とを含む。 On the other hand, when the action determination unit 103 determines that the serve action has occurred (S41: YES), the rally section determination unit 106 performs the process of S8 (the process of generating the rally section information 205 indicating the start of the rally section). .. Then, the play analyzer 100 performs the same processing as S15 and S16 (action information 203 generation processing (S42) and actor information 204 generation processing (S43)), and then performs the processing of S9 shown in FIG. .. After that, the generated play information 206 indicating the start of the rally section includes the ball trajectory information 202, the action information 203 indicating that the rally section is serving, the actor information 204, and the rally section information 205.
　以上、本開示に係る実施形態について図面を参照して詳述してきたが、上述したプレイ分析装置１００の機能は、コンピュータプログラムにより実現され得る。 Although the embodiments according to the present disclosure have been described in detail with reference to the drawings, the functions of the play analyzer 100 described above can be realized by a computer program.
　図１３は、各装置の機能をプログラムにより実現するコンピュータのハードウェア構成を示す図である。このコンピュータ３１００は、キーボード又はマウス、タッチパッドなどの入力装置３１０１、ディスプレイ又はスピーカーなどの出力装置３１０２、ＣＰＵ（Central Processing Unit）３１０３、ＧＰＵ（Graphics Processing Unit）３１０４、ＲＯＭ（Read Only Memory）３１０５、ＲＡＭ（Random Access Memory）３１０６、ハードディスク装置又はＳＳＤ（Solid State Drive）などの記憶装置３１０７、ＤＶＤ－ＲＯＭ（Digital Versatile Disk Read Only Memory）又はＵＳＢ（Universal Serial Bus）メモリなどの記録媒体から情報を読み取る読取装置３１０８、ネットワークを介して通信を行う送受信装置３１０９を備え、各部はバス３１１０により接続される。 FIG. 13 is a diagram showing a hardware configuration of a computer that realizes the functions of each device by a program. The computer 3100 includes an input device 3101 such as a keyboard or mouse and a touch pad, an output device 3102 such as a display or a speaker, a CPU (Central Processing Unit) 3103, a GPU (Graphics Processing Unit) 3104, and a ROM (Read Only Memory) 3105. Read information from a recording medium such as RAM (RandomAccessMemory) 3106, hard disk device or storage device 3107 such as SSD (SolidStateDrive), DVD-ROM (DigitalVersatileDiskReadOnlyMemory) or USB (UniversalSerialBus) memory. It includes a reading device 3108 and a transmitting / receiving device 3109 that communicates via a network, and each unit is connected by a bus 3110.
　そして、読取装置３１０８は、上記各装置の機能を実現するためのプログラムを記録した記録媒体からそのプログラムを読み取り、記憶装置３１０７に記憶させる。あるいは、送受信装置３１０９が、ネットワークに接続されたサーバ装置と通信を行い、サーバ装置からダウンロードした上記各装置の機能を実現するためのプログラムを記憶装置３１０７に記憶させる。 Then, the reading device 3108 reads the program from the recording medium on which the program for realizing the function of each of the above devices is recorded, and stores the program in the storage device 3107. Alternatively, the transmission / reception device 3109 communicates with the server device connected to the network, and stores the program downloaded from the server device for realizing the function of each device in the storage device 3107.
　そして、ＣＰＵ３１０３が、記憶装置３１０７に記憶されたプログラムをＲＡＭ３１０６にコピーし、そのプログラムに含まれる命令をＲＡＭ３１０６から順次読み出して実行することにより、上記各装置の機能が実現される。 Then, the CPU 3103 copies the program stored in the storage device 3107 to the RAM 3106, and sequentially reads and executes the instructions included in the program from the RAM 3106, whereby the functions of the above devices are realized.
　上記の実施の形態の説明に用いた各機能ブロックは、典型的には集積回路であるＬＳＩとして実現される。これらは個別に１チップ化されてもよいし、一部又は全てを含むように１チップ化されてもよい。ここでは、ＬＳＩとしたが、集積度の違いにより、ＩＣ、システムＬＳＩ、スーパーＬＳＩ、ウルトラＬＳＩと呼称されることもある。 Each functional block used in the description of the above embodiment is typically realized as an LSI which is an integrated circuit. These may be individually integrated into one chip, or may be integrated into one chip so as to include a part or all of them. Although it is referred to as LSI here, it may be referred to as IC, system LSI, super LSI, or ultra LSI depending on the degree of integration.
　また、集積回路化の手法はＬＳＩに限るものではなく、専用回路又は汎用プロセッサで実現してもよい。ＬＳＩ製造後に、プログラムすることが可能なＦＰＧＡ（Field Programmable Gate Array）、又は、ＬＳＩ内部の回路セルの接続や設定を再構成可能なリコンフィギュラブル・プロセッサを利用してもよい。 Further, the method of making an integrated circuit is not limited to LSI, and may be realized by a dedicated circuit or a general-purpose processor. An FPGA (Field Programmable Gate Array) that can be programmed after the LSI is manufactured, or a reconfigurable processor that can reconfigure the connection and settings of the circuit cells inside the LSI may be used.
　さらには、半導体技術の進歩又は派生する別技術によりＬＳＩに置き換わる集積回路化の技術が登場すれば、当然、その技術を用いて機能ブロックの集積化を行ってもよい。バイオ技術の適用等が可能性としてありえる。 Furthermore, if an integrated circuit technology that replaces an LSI appears due to advances in semiconductor technology or another technology derived from it, it is naturally possible to integrate functional blocks using that technology. There is a possibility of applying biotechnology.
　２０１９年５月２３日出願の特願２０１９－０９６７０６の日本出願に含まれる明細書、図面および要約書の開示内容は、すべて本願に援用される。 The disclosures of the specifications, drawings and abstracts contained in the Japanese application of Japanese Patent Application No. 2019-09676 filed on May 23, 2019 are all incorporated herein by reference.
　本開示の一態様は、スポーツを分析するためのシステムに有用である。 One aspect of the disclosure is useful for systems for analyzing sports.
　１　プレイ分析システム
　３，３Ａ，３Ｂ，３Ｃ，３Ｄ　カメラ
　４　表示装置
　５　入力装置
　１２　主審
　１３　副審
　１４（１４Ａ，１４Ｂ，１４Ｃ，１４Ｄ）　線審
　１５　旗
　１００　プレイ分析装置
　１０１　映像受信部
　１０２　移動体軌跡検出部
　１０３　アクション判定部
　１０４　番号認識部
　１０５　審判動作検出部
　１０６　ラリー区間判定部
　１０７　結果出力部
　１０８　ＵＩ制御部
　１０９　映像制御部
　１１０　プレイ情報制御部
　１１１　スコア情報制御部
　１１２　記憶部
　２０１　動画フレーム
　２０２　ボール軌跡情報
　２０３　アクション情報
　２０４　アクター情報
　２０５　ラリー区間情報
　２０６　プレイ情報
　２１００　プレイ映像
　２２００　プレイ情報リスト
　２２０１　再生ボタン
　２２０２　チーム
　２２０３　選手番号
　２２０４　アクション
　２３００　スコア情報リスト
　２３０１　再生ボタン
　２３０２　スコア
　２３０３　チェックボックス
　３１００　コンピュータ
　３１０１　入力装置
　３１０２　出力装置
　３１０３　ＣＰＵ
　３１０４　ＧＰＵ
　３１０５　ＲＯＭ
　３１０６　ＲＡＭ
　３１０７　記憶装置
　３１０８　読取装置
　３１０９　送受信装置
　３１１０　バス
　Ｂ　ボール
　Ｐ１，Ｐ２，Ｐ３，Ｐ４，Ｐ５　画像 1 Play analysis system 3,3A, 3B, 3C, 3D camera 4 Display device 5 Input device 12 Referee 13 Deputy referee 14 (14A, 14B, 14C, 14D) Linesmen 15 Flag 100 Play analyzer 101 Video receiver 102 Moving object trajectory detection Unit 103 Action judgment unit 104 Number recognition unit 105 Referee motion detection unit 106 Rally section judgment unit 107 Result output unit 108 UI control unit 109 Video control unit 110 Play information control unit 111 Score information control unit 112 Storage unit 201 Video frame 202 Ball trajectory Information 203 Action information 204 Actor information 205 Rally section information 206 Play information 2100 Play video 2200 Play information list 2201 Play button 2202 Team 2203 Player number 2204 Action 2300 Score information list 2301 Play button 2302 Score 2303 Check box 3100 Computer 3101 Input device 3102 Output Device 3103 CPU
3104 GPU
3105 ROM
3106 RAM
3107 Storage device 3108 Reader 3109 Transmission / reception device 3110 Bus B ball P1, P2, P3, P4, P5 Image

Claims (10)

  1. 　球技のプレイを分析するプレイ分析装置であって、
    　プレイ映像を用いて、前記球技の審判員の動作パターンを検出する審判動作検出部と、
    　前記プレイ映像に基づきラリー区間の開始を判定すると共に、前記審判員の動作パターンに基づき前記ラリー区間の終了を判定するラリー区間判定部と、
    　を備える、プレイ分析装置。     A play analyzer that analyzes the play of ball games.
    A referee motion detection unit that detects the motion pattern of the referee of the ball game using the play video,
    A rally section determination unit that determines the start of the rally section based on the play video and determines the end of the rally section based on the operation pattern of the referee.
    A play analyzer equipped with.
  2. 　前記審判員の動作パターンは、線審が所持する旗の動作パターンである、
    　請求項１に記載のプレイ分析装置。     The movement pattern of the referee is the movement pattern of the flag possessed by the linesmen.
    The play analyzer according to claim 1.
  3. 　前記旗の動作パターンは、前記旗が静止状態から動作状態に変化したタイミング、前記旗の移動速度、及び、前記旗の移動量のうち少なくとも１つの情報を含む、
    　請求項２に記載のプレイ分析装置。     The operation pattern of the flag includes at least one information of the timing when the flag changes from the stationary state to the operating state, the moving speed of the flag, and the moving amount of the flag.
    The play analyzer according to claim 2.
  4. 　前記プレイ映像を用いて、前記球技の移動体の軌跡パターンを検出する移動体軌跡検出部を更に備え、
    　前記ラリー区間判定部は、前記審判員の動作パターンに基づき前記ラリー区間の終了を判定できない場合、前記移動体の軌跡パターンに基づき前記ラリー区間の終了を判定する、
    　請求項１から３のいずれか１項に記載のプレイ分析装置。     Further, a moving body locus detection unit for detecting the locus pattern of the moving body of the ball game by using the play video is provided.
    When the rally section determination unit cannot determine the end of the rally section based on the movement pattern of the referee, the rally section determination unit determines the end of the rally section based on the locus pattern of the moving body.
    The play analyzer according to any one of claims 1 to 3.
  5. 　前記ラリー区間判定部が前記ラリー区間の終了の判定に用いる前記移動体の軌跡パターンは、前記移動体の移動速度、移動量及び位置のうち少なくとも１つの情報を含む、
    　請求項４に記載のプレイ分析装置。     The locus pattern of the moving body used by the rally section determining unit to determine the end of the rally section includes at least one information of the moving speed, moving amount, and position of the moving body.
    The play analyzer according to claim 4.
  6. 　前記ラリー区間判定部は、所定時間の前記プレイ映像において前記移動体を確認できない場合、前記ラリー区間が終了したと判定する、
    　請求項４又は５に記載のプレイ分析装置。     When the moving body cannot be confirmed in the play video for a predetermined time, the rally section determination unit determines that the rally section has ended.
    The play analyzer according to claim 4 or 5.
  7. 　前記ラリー区間判定部は、前記移動体の軌跡パターンに基づき前記ラリー区間の開始を判定する、
    　請求項４から６のいずれか１項に記載のプレイ分析装置。     The rally section determination unit determines the start of the rally section based on the locus pattern of the moving body.
    The play analyzer according to any one of claims 4 to 6.
  8. 　前記ラリー区間判定部が前記ラリー区間の開始の判定に用いる前記移動体の軌跡パターンは、前記移動体を投げ上げた際の軌跡パターンである、
    　請求項７に記載のプレイ分析装置。     The locus pattern of the moving body used by the rally section determining unit to determine the start of the rally section is a locus pattern when the moving body is thrown up.
    The play analyzer according to claim 7.
  9. 　前記ラリー区間判定部における前記ラリー区間の判定結果にしたがって、前記プレイ映像から前記ラリー区間の映像を抽出する映像制御部を更に備える、
    　請求項１から８のいずれか１項に記載のプレイ分析装置。     A video control unit that extracts the video of the rally section from the play video according to the determination result of the rally section in the rally section determination unit is further provided.
    The play analyzer according to any one of claims 1 to 8.
  10. 　球技のプレイを分析するプレイ分析方法であって、
    　プレイ映像を用いて前記球技の審判員の動作パターンを検出し、
    　前記プレイ映像に基づきラリー区間の開始を判定すると共に、前記審判員の動作パターンに基づき前記ラリー区間の終了を判定する、
    　プレイ分析方法。     It is a play analysis method that analyzes the play of ball games.
    The movement pattern of the referee of the ball game is detected using the play video,
    The start of the rally section is determined based on the play video, and the end of the rally section is determined based on the operation pattern of the referee.
    Play analysis method.
PCT/JP2020/018462 2019-05-23 2020-05-01 Play analyzing device, and play analyzing method WO2020235339A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2019096706A JP2020188979A (en) 2019-05-23 2019-05-23 Play analyzer and play analysis method
JP2019-096706 2019-05-23

Publications (1)

Publication Number Publication Date
WO2020235339A1 true WO2020235339A1 (en) 2020-11-26

Family

ID=73453072

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2020/018462 WO2020235339A1 (en) 2019-05-23 2020-05-01 Play analyzing device, and play analyzing method

Country Status (2)

Country Link
JP (1) JP2020188979A (en)
WO (1) WO2020235339A1 (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR102626100B1 (en) * 2021-11-30 2024-01-17 마이스포 주식회사 Foul judgment system in badminton matches and method of operation of system
WO2023106201A1 (en) * 2021-12-09 2023-06-15 Ｎｅｃソリューションイノベータ株式会社 Play analysis device, play analysis method, and computer-readable storage medium
JP7144778B1 (en) 2022-08-02 2022-09-30 シンメディクト株式会社 Competition Management Program, Competition Management Device

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2001251581A (en) * 2000-03-03 2001-09-14 Jisedai Joho Hoso System Kenkyusho:Kk Sports video digest production method, and computer- readable recording medium with program for performing production method via computer recorded therein
WO2006077806A1 (en) * 2005-01-24 2006-07-27 Matsushita Electric Industrial Co., Ltd. Scene segment dividing device
JP2015070503A (en) * 2013-09-30 2015-04-13 キヤノン株式会社 Information processing apparatus, information processing method, and program
WO2015098260A1 (en) * 2013-12-27 2015-07-02 ソニー株式会社 Analysis device, recording medium and analysis method
US20190005331A1 (en) * 2017-06-29 2019-01-03 Electronics And Telecommunications Research Institute Apparatus and method for detecting event based on deterministic finite automata in soccer video

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2001251581A (en) * 2000-03-03 2001-09-14 Jisedai Joho Hoso System Kenkyusho:Kk Sports video digest production method, and computer- readable recording medium with program for performing production method via computer recorded therein
WO2006077806A1 (en) * 2005-01-24 2006-07-27 Matsushita Electric Industrial Co., Ltd. Scene segment dividing device
JP2015070503A (en) * 2013-09-30 2015-04-13 キヤノン株式会社 Information processing apparatus, information processing method, and program
WO2015098260A1 (en) * 2013-12-27 2015-07-02 ソニー株式会社 Analysis device, recording medium and analysis method
US20190005331A1 (en) * 2017-06-29 2019-01-03 Electronics And Telecommunications Research Institute Apparatus and method for detecting event based on deterministic finite automata in soccer video

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
KATO, KENTA ET AL.: "Data analysis and team strengthening in soccer /Non-official translation/", IEICE COMMUNICATIONS SOCIETY MAGAZINE, vol. 10, no. 1, 2016, pages 29 - 34, XP055762910, Retrieved from the Internet <URL:https://www.jstage.jst.go.jp/article/bplus/10/1/10_29/_article/-char/ja> [retrieved on 20200527] *

Also Published As

Publication number Publication date
JP2020188979A (en) 2020-11-26

Similar Documents

Publication Publication Date Title
US11049258B2 (en) Stereoscopic image capture with performance outcome prediction in sporting environments
WO2020235339A1 (en) Play analyzing device, and play analyzing method
JPWO2019225415A1 (en) Ball game video analysis device and ball game video analysis method
US20140301600A1 (en) True space tracking of axisymmetric object flight using diameter measurement
US20180137363A1 (en) System for the automated analisys of a sporting match
JP6249706B2 (en) Information processing apparatus, information processing method, and program
US11745077B1 (en) System and method for a user adaptive training and gaming platform
CN111444890A (en) Sports data analysis system and method based on machine learning
US20190201794A1 (en) Experience-oriented virtual baseball game apparatus and virtual baseball game control method using the same
US9333412B2 (en) Virtual golf simulation apparatus and method and sensing device and method used for the same
WO2020255948A1 (en) Play analysis device, play analysis method, and computer program
JP7113335B2 (en) Play analysis device and play analysis method
JP7113336B2 (en) Play analysis device and play analysis method
WO2020255716A1 (en) Play analysis device, play analysis method, and computer program
WO2020071092A1 (en) Play analysis device and play analysis method
JP7437652B2 (en) Ball game video analysis device, ball game video analysis system, ball game video analysis method, and computer program
WO2020230677A1 (en) Play analysis apparatus and play analysis method
JP7296546B2 (en) Play analysis device and play analysis method
US11707663B1 (en) System for tracking, locating and predicting the position of a ball in a game of baseball or similar
WO2023106201A1 (en) Play analysis device, play analysis method, and computer-readable storage medium
US11900678B2 (en) System for tracking, locating and calculating the position of an object in a game involving moving objects
JP7024862B2 (en) Performance assessors, data acquisition devices, their methods, and programs
TWI782649B (en) Badminton smash measurement system and method
WO2022056315A1 (en) System and method for capture and analysis of sporting performance data and broadcast of the same
GB2597108A (en) Extended reality emulation of a real-life sporting event

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20810583

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20810583

Country of ref document: EP

Kind code of ref document: A1