WO2021187193A1

WO2021187193A1 - Ball game footage analysis device, ball game footage analysis system, ball game footage analysis method, and computer program

Info

Publication number: WO2021187193A1
Application number: PCT/JP2021/008950
Authority: WO
Inventors: 優麻齋藤; 純子上田; 井村　康治
Original assignee: パナソニックＩｐマネジメント株式会社
Priority date: 2020-03-16
Filing date: 2021-03-08
Publication date: 2021-09-23
Also published as: JP2021145702A; JP7437652B2

Abstract

This ball game footage analysis device: generates trajectory information of a moving body on the basis of a first video and a second video that capture, from differing positions, an action of a person on a moving body for a ball game; manages a plurality of pieces of record information indicating the correspondence relationship between the trajectory information and the first video and the second video; displays a UI that allows selection of the record information on the basis of the trajectory information from among the plurality of pieces of record information; and displays the first video or the second video that is in a correspondence relationship with the record information selected through the UI.

Description

Ball game video analysis device, ball game video analysis system, ball game video analysis method, and computer program

This disclosure relates to a ball game video analysis device, a ball game video analysis system, a ball game video analysis method, and a computer program.

Conventionally, "Data Volley" has been marketed as volleyball analysis software, and a technique for digitizing the situation of team players based on the subjective judgment of an analyst who is familiar with this software is known.

In recent years, without relying on the operation of analysts, cameras have been installed at each of the four corners overlooking the court, and these four cameras have been synchronized to capture images of the play, and the movement of the ball has been tracked to track the ball. There is known a technique for obtaining three-dimensional position information of the camera and at the same time outputting analysis data regarding the player's uniform number and the type of action (serve, reception, dig, toss, attack, block) included in the play (patent document). 1).

International Publication No. 2019/225415

By the way, the practice of the action of a person who plays a ball game (for example, serving volleyball) is shot with a video using a camera that is close to us, and the video is played back later to be useful for the practice. However, in the past, we searched for videos by relying on the shooting date and time of the video and thumbnails, and identified the target video and comparative video from multiple videos in which the same action was shot, and the proficiency level and form of the action. It is difficult to grasp the points to be improved.

The purpose of the present disclosure is to identify the target video and the comparative video from a plurality of videos of the action of a person performing a ball game, and to easily grasp the proficiency level of the action and the improvement points of the form. To provide technology.

The ball game video analysis device according to the present disclosure is locus information indicating the movement locus of the moving body based on the first moving image and the second moving image of the actions of a person with respect to the moving body used in the ball game taken from different positions. A locus analysis unit that generates A display for displaying a UI (User Interface) for selecting the recorded information based on the locus information, and displaying a first moving image or a second moving image having a corresponding relationship with the recorded information selected through the UI. It includes a control unit.

The ball game video analysis system according to the present disclosure includes a first mobile terminal with a camera, a second mobile terminal with a camera, a ball game video analysis device, and a display terminal. The first mobile terminal captures a person's action on a moving body used in a ball game from a first position to generate a first moving image. The second mobile terminal captures the action of the person from a second position to generate a second moving image. The ball game video analysis device generates locus information indicating the locus of the moving body based on the first moving image and the second moving image, and generates the locus information indicating the locus of the moving body, and the first moving image, the second moving image, and the locus information. A UI for managing a plurality of recorded information which is information indicating a correspondence relationship with the above and selecting a recorded information based on the locus information from the plurality of recorded information is displayed on the display terminal and selected through the UI. The first moving image or the second moving image having a corresponding relationship in the recorded information is displayed on the display terminal.

In the ball game video analysis method according to the present disclosure, the apparatus obtains the movement locus of the moving body based on the first moving image and the second moving image of the action of the person with respect to the moving body used in the ball game taken from different positions. The locus information to be shown is generated, a plurality of recorded information which is information indicating the correspondence between the first moving image and the second moving image and the locus information is managed, and the locus information is selected from the plurality of recorded information. A UI for selecting recording information is displayed based on the above, and a first moving image or a second moving image having a corresponding relationship in the recording information selected through the UI is displayed.

The computer program according to the present disclosure generates trajectory information indicating the movement trajectory of the moving body based on the first moving image and the second moving image of the actions of the person with respect to the moving body used in the ball game taken from different positions. Then, a plurality of recorded information, which is information indicating the correspondence between the first moving image and the second moving image and the locus information, is managed, and the recorded information is recorded from the plurality of recorded information based on the locus information. Is displayed, and a computer is made to execute a process of displaying a first moving image or a second moving image having a corresponding relationship in the recorded information selected through the UI.

It should be noted that these comprehensive or specific aspects may be realized by a system, a device, a method, an integrated circuit, a computer program or a recording medium, and any of the system, a device, a method, an integrated circuit, a computer program and a recording medium. It may be realized by various combinations.

According to the present disclosure, it is possible to identify the target video and the comparative video from a plurality of videos of the action of a person performing a ball game, and easily grasp the proficiency level of the action and the improvement points of the form. ..

The figure which shows the configuration example of the ball game image analysis system which concerns on Embodiment 1. Diagram for explaining how to identify the position of the ball A block diagram showing a configuration example of the ball game video analysis device according to the first embodiment. Diagram for explaining how to identify the reference frame image Flowchart showing an example of pre-work Sequence chart showing an example of processing related to pre-work A sequence chart showing an example of processing related to pre-work when an operation to change the reference frame image is performed in FIG. A diagram showing a detailed example of the reference frame confirmation UI (User Interface) The figure which shows the detailed example of the UI for changing a reference frame Diagram showing a display example of analysis results The figure which shows the configuration example of the ball game image analysis system which concerns on Embodiment 2. A block diagram showing a configuration example of the ball game video analysis device according to the second embodiment. Block diagram showing a configuration example of a display terminal according to the second embodiment Diagram for explaining an example of a posture model Diagram to illustrate the sub-actions that make up a jumping serve Diagram showing a display example of the video comparison UI Diagram showing a display example of the UI for sub-action video comparison The figure which shows an example of the posture comparison UI when the posture models are displayed side by side. The figure which shows an example of the posture comparison UI when the posture model is superposed and displayed. Diagram showing an example of the UI for sub-action posture comparison Diagram for explaining an example when selecting a posture model in the posture comparison UI The figure which shows the hardware structure of the computer which realizes the function of the ball game image analysis apparatus which concerns on this disclosure by a program. The figure which shows the hardware composition of the computer which realizes the function of the 1st mobile terminal, the 2nd mobile terminal and the display terminal by a program which concerns on this disclosure.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings as appropriate. However, more detailed explanation than necessary may be omitted. For example, detailed explanations of already well-known matters and duplicate explanations for substantially the same configuration may be omitted. This is to avoid unnecessary redundancy of the following description and to facilitate the understanding of those skilled in the art. It should be noted that the accompanying drawings and the following description are provided for those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter of the claims.

(Embodiment 1)
<Structure of ball game video analysis system>
FIG. 1 is a diagram showing a configuration example of a ball game video analysis system according to the present embodiment.

The ball game video analysis system 10 is a system that analyzes a video of a ball game and analyzes the trajectory of a moving body used in the ball game. In the present embodiment, volleyball, which is one of the ball games, will be described as an example. However, the ball game video analysis system 10 is not limited to volleyball, and can be applied to various ball games such as soccer, baseball, table tennis, basketball, tennis, rugby, American football, lacrosse, badminton, and ice hockey. The moving body targeted by the ball game video analysis system 10 is typically a ball. However, the moving body targeted by the ball game video analysis system 10 is not limited to a ball, and may have a shape that does not fit the concept of a "ball" such as a badminton shuttle or an ice hockey puck.

The ball game video analysis system 10 includes a first mobile terminal 20A, a second mobile terminal 20B, a display terminal 30, and a ball game video analysis device 100. The first mobile terminal 20A, the second mobile terminal 20B, the display terminal 30, and the ball game video analysis device 100 can transmit and receive information to and from each other through the communication network N. The communication network N is composed of, for example, a wireless communication network (wireless LAN (Local Area Network), 4G, 5G, etc.), an Internet network, or a combination thereof.

The first mobile terminal 20A and the second mobile terminal 20B are wireless communication terminals provided with a camera. Examples of the first mobile terminal 20A and the second mobile terminal 20B are mobile phones, smartphones, tablets, mobile PCs, and the like.

The first mobile terminal 20A and the second mobile terminal 20B photograph the action of a person performing a ball game (hereinafter referred to as "player"). In the present embodiment, a volleyball serve, which is one of the actions of the player, will be described as an example. However, the action of the player is not limited to the serve of volleyball, and may be a dig, a reception, a toss, an attack, a block, or the like. Alternatively, the action of the person performing the ball game may be a specific action in the above sports other than volleyball.

The first mobile terminal 20A and the second mobile terminal 20B, for example, shoot a player's serve practice as a moving image from different directions, and generate a first moving image 201A and a second moving image 201B, respectively. For example, as shown in FIG. 1, the first mobile terminal 20A photographs the player from the rear right, and the second mobile terminal 20B photographs the player from the rear left.

The first mobile terminal 20A and the second mobile terminal 20B transmit the generated first moving image 201A and second moving image 201B to the ball game video analysis device 100 through the communication network N, respectively.

The display terminal 30 displays the analysis result by the ball game video analysis device 100. A display example of the analysis result will be described later (see FIG. 10). Examples of the display terminal 30 are mobile phones, smartphones, tablets, mobile PCs, and the like. The display terminal 30 may be read as a third mobile terminal. Although the configuration in which the display terminal 30 for displaying the analysis result is separately provided is illustrated, either the first mobile terminal 20A or the second mobile terminal 20B may be used as a terminal for displaying the analysis result.

The ball game video analysis device 100 analyzes the movement trajectory of the ball in the three-dimensional space based on the first moving image 201A and the second moving image 201B received from the first mobile terminal 20A and the second mobile terminal 20B, respectively. .. The movement locus of the ball in the three-dimensional space can be generated by arranging the three-dimensional coordinates of the ball in chronological order.

As shown in FIG. 2, the three-dimensional coordinates of the ball at time t can be calculated from the frame image F1 at time t in the first moving image 201A and the frame image F2 at time t in the second moving image 201B. .. That is, as shown in FIG. 2, the vector V1 extending from the position U1 of the first mobile terminal 20A (camera) to the position B1 of the ball in the frame image F1 and the position of the second mobile terminal 20B (camera). The intersection with the vector V2 extending from U2 to the position B2 of the ball in the frame image F2 is the three-dimensional coordinate Pa of the ball.

Here, the time code (time information) given to each frame image of the first moving image 201A by the first mobile terminal 20A and the time given to each frame image of the second moving image 201B by the second mobile terminal 20B. If the code is a completely accurate time, the three-dimensional coordinates of the ball can be accurately specified by the above method using the frame images F1 and F2 to which the time code of the time t is assigned.

However, when a general-purpose camera such as a camera mounted on a mobile terminal is used instead of a dedicated camera specialized in ball game video analysis as shown in Patent Document 1, the first mobile terminal 20A and the second mobile phone are used. The terminal 20B cannot always give an accurate time code. The reason will be described below.

Since the ball of sports related to ball games moves at high speed, in order to specify the three-dimensional position of the ball, for example, a frame image taken by a first mobile terminal 20A and a second frame image taken at a high frame rate of 30 fps or more are taken. The synchronization accuracy in milliseconds with the frame image captured by the mobile terminal 20B of the above is required.

Here, it is conceivable to synchronize the internal clocks of the first mobile terminal 20A and the second mobile terminal 20B by using NTP (Network Time Protocol). However, since the synchronization accuracy of NTP is in seconds, there is a possibility that a synchronization shift in milliseconds may occur between the internal clock of the first mobile terminal 20A and the internal clock of the second mobile terminal 20B. Therefore, with this method, it may not be possible to specify the three-dimensional coordinates of a ball moving at high speed with sufficient accuracy.

Alternatively, it is conceivable to synchronize the internal clocks of the first mobile terminal 20A and the second mobile terminal 20B by using a satellite positioning system such as GPS (Global Positioning System). However, with this method, the camera of the mobile terminal that does not have the GPS function cannot be used. Further, when the GPS signal cannot be received in an indoor facility or the like, there is a possibility that a synchronization shift may occur between the internal clock of the first mobile terminal 20A and the internal clock of the second mobile terminal 20B. Therefore, with this method, it may not be possible to specify the three-dimensional coordinates of a ball moving at high speed with sufficient accuracy.

Alternatively, when the first mobile terminal 20A and / or the second mobile terminal 20B assigns a time code, a synchronization shift may occur. For example, when the processing performance of the second mobile terminal 20B is lower than that of the first mobile terminal 20A, the processing load of time code assignment of the mobile terminal 20B is large, and it is slightly slower than the first mobile terminal 20A as a whole. There is a possibility of giving a time code. In this case, even if the time codes of the frame image F1 and the frame image F2 are the same, the frame image F2 may actually be taken a little earlier than the frame image F1. In the frame image F2 taken shortly before this, the position of the ball is B3, and as shown in FIG. 2, the three-dimensional coordinates of the ball are also shifted to Pb. Therefore, the calculated movement locus of the ball also deviates.

Therefore, the ball game video analysis device 100 according to the present embodiment has a function of specifying the synchronization deviation of the frame image between the first mobile terminal 20A and the second mobile terminal 20B. Then, the ball game image analysis device 100 calculates the movement locus of the ball in consideration of the specified synchronization deviation. As a result, the movement locus of the ball can be calculated by using a general-purpose camera such as a camera mounted on a mobile terminal instead of a dedicated camera specialized in ball game video analysis as shown in Patent Document 1. Hereinafter, a detailed description will be given.

<Structure of ball game video analysis device>
FIG. 3 is a block diagram showing a configuration example of the ball game video analysis device 100 according to the present embodiment.

The ball game video analysis device 100 includes a moving image receiving unit 101, an image conversion unit 102, a reference frame specifying unit 103, a reference frame changing unit 104, a trajectory analysis unit 105, and a result display unit 106.

The video receiving unit 101 receives the first video 201A from the first mobile terminal 20A, and receives the second video 201B from the second mobile terminal 20B. Examples of video formats include MPEG4, TS files, H.D. 264, H. It is 265 mag. A time code is assigned to each frame image constituting the first moving image 201A by the first mobile terminal 20A. A time code is assigned to each frame image constituting the second moving image 201B by the second mobile terminal 20B. As described with reference to FIG. 1, in the first moving image 201A and the second moving image 201B, for example, a scene in which the player hits a serve is shot.

The image conversion unit 102 converts each frame image of the first moving image 201A into a still image, and generates the first frame image group 202A. The image conversion unit 102 converts each frame image of the second moving image 201B into a still image, and generates a second frame image group 202B. Examples of still image formats are JPEG, GIF, TIFF and the like.

The reference frame specifying unit 103 identifies a reference frame image (hereinafter referred to as “reference frame image”) for frame synchronization from the first frame image group 202A and the second frame image group 202B, respectively. .. Hereinafter, the reference frame image specified from the first frame image group 202A is referred to as a first reference frame image, and the reference frame image specified from the second frame image group 202B is referred to as a second reference frame. It is called an image.

The reference frame specifying unit 103 specifies a frame image in which a specific sub-action constituting the action is captured as a reference frame image. For example, the reference frame specifying unit 103 captures a frame image in which the moment (sub-action) when the hand hitting the serve hits the ball is captured in the frame image group in which the series of actions of the serve (action) is captured. Specify as a reference frame image. Next, an example of a method of specifying the reference frame image will be described with reference to FIG.

As shown in FIG. 4, the reference frame specifying unit 103 sets the ball detection region 211A for the first frame image group 202A. The reference frame specifying unit 103 sets the ball detection area 211B for the second frame image group 202B. As a result, the ball detection area becomes smaller, so that the time required for the ball detection process can be shortened. It is not necessary to set the detection area of the ball.

The reference frame specifying unit 103 detects a ball from each frame image of the first frame image group 202A by a known method. For example, the reference frame specifying unit 103 detects a ball from a frame image by pattern matching. Alternatively, the reference frame specifying unit 103 detects the ball from the frame image by using deep learning learned in advance for ball detection. Similarly, the reference frame specifying unit 103 detects the ball from each frame image of the second frame image group 202B.

The reference frame specifying unit 103 calculates the two-dimensional movement locus of the ball based on the position of the ball detected from each frame image of the first frame image group 202A. Then, the reference frame specifying unit 103 identifies the first reference frame image based on the change in the two-dimensional movement locus of the ball. For example, the reference frame specifying unit 103 identifies the first reference frame image by the processing of the following steps S11 to S13.

(S11) The reference frame specifying unit 103 obtains a first frame image from the first frame image group 202A in which the amount of movement of the ball in the X direction is equal to or greater than a predetermined threshold value (that is, the ball has moved significantly in the lateral direction). Extract. As a result, it is possible to extract the first frame image in which the ball is flying toward the net immediately after the hand hitting the serve hits the ball.

(S12) In the reference frame specifying unit 103, the movement amount of the ball in the Y-axis direction is a predetermined threshold value among the plurality of first frame images included in a certain period before the first frame image extracted in step S11. The first frame image that changes as described above and is closest to the first frame image extracted in step S11 is specified. Thereby, among the plurality of first frame images in which the ball thrown for the serve is rising or falling, the first frame image at the moment when the hand hitting the serve hits the ball can be identified. ..

(S13) The reference frame specifying unit 103 uses the first frame image specified in step S12 as the first reference frame image.

The reference frame specifying unit 103 specifies the second reference frame image by the same processing as described above. Then, the reference frame specifying unit 103 has a time code corresponding to the first reference frame image (hereinafter referred to as “first reference time code”) and a time code corresponding to the second reference frame image (hereinafter referred to as “second reference time code”). "Reference time code") is written in the synchronization deviation information 203.

Note that the reference frame specifying unit 103 may specify the reference frame image by an action and a sub-action different from the serve. For example, the reference frame specifying unit 103 may use a frame image in which the moment when the hand that made the attack (or dig, reception, toss, block, etc.) of the player hits the ball is captured as the reference frame image.

Further, the reference frame specifying unit 103 may write a value obtained by subtracting the second reference time code from the first reference time code (hereinafter referred to as “synchronization time difference”) in the synchronization deviation information 203. The synchronization time difference written in the synchronization deviation information 203 corresponds to the magnitude of the synchronization deviation of the frame image between the first mobile terminal 20A and the second mobile terminal 20B. For example, when the synchronization time difference is almost 0, it is shown that the time code assigned by the first mobile terminal 20A and the time code assigned by the second mobile terminal 20B tend to be almost the same. When the synchronization time difference is a positive value, it indicates that the time code assigned by the second mobile terminal 20B tends to be later than the time code assigned by the first mobile terminal 20A. When the synchronization time difference is a negative value, it indicates that the time code assigned by the first mobile terminal 20A tends to be later than the time code assigned by the second mobile terminal 20B.

Return to the explanation in Fig. 3.

The reference frame image automatically identified by the reference frame specifying unit 103 by the method described above is not necessarily the correct reference frame image. Therefore, the reference frame specifying unit 103 provides a UI (User Interface) (hereinafter referred to as “reference frame confirmation UI”) 310 for the user to confirm the reference frame image. The details of the reference frame confirmation UI 310 will be described later (see FIG. 8).

The reference frame changing unit 104 provides a UI (hereinafter referred to as "reference frame changing UI") 320 for the user to change the reference frame image. The details of the reference frame changing UI 320 will be described later (see FIG. 9). When the reference frame image is changed through the reference frame change UI 320, the reference frame change unit 104 changes the synchronization shift information 203 based on the changed reference frame image.

The locus analysis unit 105 identifies the movement locus of the ball in the three-dimensional space related to the action by using the synchronization deviation information 203 from, for example, the first video 201A and the second video 201B in which the action of the serve by the player is photographed. , Generates analysis result information 204 including information indicating the movement trajectory of the ball. The first moving image 201A and the second moving image 201B to be analyzed by the locus analysis unit 105 are the first moving image 201A and the second moving image 201B taken to generate the above-mentioned synchronization deviation information 203. It can be different.

For example, the trajectory analysis unit 105 uses a perspective projection conversion matrix for converting the camera parameters (world coordinates into a camera image) of the first mobile terminal 20A and the second mobile terminal 20B obtained in advance by calibration processing. ), The position of the ball in the first frame image, and the position of the ball in the second frame image synchronized with the first frame image corrected by the synchronization deviation information 203. The three-dimensional coordinates of the ball at the timing when the frame image and the second frame image are taken are specified. The locus analysis unit 105 specifies the movement locus of the ball in the three-dimensional space by arranging the three-dimensional coordinates of the ball thus specified in chronological order. The locus analysis unit 105 generates analysis result information 204 including information indicating the movement locus of the specified ball. As a result, the movement trajectory of the ball in the three-dimensional space can be analyzed with sufficient accuracy even if a camera mounted on the mobile terminal is used instead of a dedicated camera specialized in ball game video analysis.

The result display unit 106 displays the analysis result information 204 on, for example, the display terminal 30. The result display unit 106 may display the analysis result information 204 on the first mobile terminal 20A and / or the second mobile terminal 20B.

<Preliminary work>
FIG. 5 is a flowchart showing an example of the preliminary work. The pre-work is a work performed before the trajectory analysis unit 105 analyzes the actual action, and is, for example, a work performed to generate the synchronization deviation information 203.

The user installs the first mobile terminal 20A and the second mobile terminal 20B at positions where the player and the ball can be photographed from different directions (S101).

The user pairs the first mobile terminal 20A and the second mobile terminal 20B (S102). The pairing information may be transmitted to the ball game video analyzer 100.

The user instructs the first mobile terminal 20A and the second mobile terminal 20B to start shooting related to the preliminary work (S103). As a result, the first mobile terminal 20A and the second mobile terminal 20B start shooting.

The player hits the serve (S104). As a result, the first mobile terminal 20A and the second mobile terminal 20B can shoot the first moving image 201A and the second moving image 201B in which the player hits the serve and the ball flies, respectively. The user and the player may be the same person.

The user instructs the first mobile terminal 20A and the second mobile terminal 20B to stop shooting related to the preliminary work (S105). The first mobile terminal 20A and the second mobile terminal 20B receive an instruction to stop shooting, respectively, and transmit the first moving image 201A and the second moving image 201B to the ball game video analysis device 100, respectively. The ball game video analysis device 100 identifies the first reference frame image from the first moving image 201A and the second reference frame image from the second moving image 201B, and displays the reference frame confirmation UI 310 on the display terminal 30.

The user confirms the first reference frame image and the second reference frame image from the reference frame confirmation UI 310 displayed on the display terminal 30 (S106). For example, when changing the reference frame image, the user instructs the ball game video analysis device 100 to change the reference frame image through the reference frame confirmation UI 310. In this case, the ball game video analysis device 100 causes the display terminal 30 to display the UI 320 for changing the reference frame.

Note that the instructions for starting and stopping shooting in steps S103 and S105 may be given from either mobile terminal. For example, when an instruction to start and stop shooting is given from the first mobile terminal 20A, the start and instruction of shooting is automatically transmitted to the second mobile terminal 20B via the ball game video analyzer 100. You can.

The user operates the reference frame change UI 320 displayed on the display terminal 30 to change the reference frame image (S107). The details of steps S106 and S107 will be described later (see FIGS. 7 and 8).

By the above processing, the ball game image analysis device 100 can generate the synchronization deviation information 203 based on the first reference frame image and the second reference frame image confirmed and corrected by the user.

<Processing related to pre-work>
FIG. 6 is a sequence chart showing an example of processing related to pre-work. FIG. 6 corresponds to a detailed description of steps S106 to S107 in FIG.

The video receiving unit 101 receives the first video 201A from the first mobile terminal 20A (S201), and receives the second video 201B from the second mobile terminal 20B (S202).

The image conversion unit 102 generates the first frame image group 202A from the first moving image 201A, and generates the second frame image group 202B from the second moving image 201B (S203).

The reference frame specifying unit 103 searches the first frame image group 202A for a first reference frame image in which a specific sub-action constituting the action is captured (S204). For example, as described above, the reference frame specifying unit 103 captures the moment when the hand hitting the player's serve hits the ball based on the change in the two-dimensional (X direction, Y direction) movement locus of the ball. The first reference frame image is searched.

Similar to the above, the reference frame specifying unit 103 searches for the second reference frame image from the second frame image group 202B (S205). At this time, the reference frame specifying unit 103 may use the second frame image group 202B after the time code that goes back a predetermined time from the time code of the first reference frame image as the search range of the second reference frame image. As a result, the time required for the search can be shortened as compared with the case where the entire second frame image group 202B is set as the search range.

The reference frame specifying unit 103 uses the time code of the first reference frame image (first reference time code) and the time code of the second reference frame image (second reference time code) as synchronization deviation information 203. Write to (S206).

The reference frame changing unit 104 transmits information for displaying the first reference frame image, the second reference frame image, and the reference frame confirmation UI 310 to the display terminal 30 (S207).

The display terminal 30 displays the first reference frame image, the second reference frame image, and the reference frame confirmation UI 310 (S208). The details of the reference frame confirmation UI 310 will be described later (see FIG. 8).

When the user performs an operation to the effect that the reference frame image is not changed with respect to the reference frame confirmation UI 310 displayed in step S208, the display terminal 30 provides instruction information indicating no change to the ball game video analysis device 100. (S209).

When the reference frame changing unit 104 receives the instruction information indicating that there is no change, the reference frame changing unit 104 determines the content of the synchronization deviation information 203 written in step S206 (S210). Then, the ball game video analysis device 100 ends this process.

FIG. 7 is a sequence chart showing an example of processing related to pre-work when an operation to change the reference frame image is performed in FIG.

When the user performs an operation to change the reference frame image with respect to the reference frame confirmation UI 310 displayed in step S208, the display terminal 30 provides the instruction information indicating the change to the ball game video analysis device 100. (S221).

When the reference frame changing unit 104 receives the instruction information indicating that there is a change, the reference frame changing unit 104 performs the following processing. That is, the reference frame changing unit 104 extracts a plurality of first frame images taken before and after the first reference frame image searched in step S204 from the first frame image group 202A, and extracts a plurality of the extracted first frame images. The first frame image of is used as a candidate for the first reference frame image.

Similarly, the reference frame changing unit 104 extracts a plurality of second frame images taken before and after the second reference frame image searched in step S205 from the second frame image group 202B, and extracts them. A plurality of second frame images are candidates for the second reference frame image. Then, the reference frame changing unit 104 displays a plurality of first candidate frame candidates and their time codes, a plurality of second reference frame image candidates and these time codes, and a reference frame changing UI 320. Information for this is transmitted to the display terminal 30 (S222).

The display terminal 30 displays the reference frame change UI 320 (S223). The details of the reference frame changing UI 320 will be described later (see FIG. 9).

The user operates the reference frame change UI 320 displayed in step S223 to select the changed first reference frame image from the plurality of first reference frame image candidates, and a plurality of first reference frame images. From the two reference frame image candidates, the changed second reference frame image is selected (S224).

The display terminal 30 uses the time code of the first reference frame image after the change (hereinafter referred to as “the first reference time code after the change”) selected in step S224, and / or the second after the change. The time code of the reference frame image (hereinafter referred to as “the second reference time code after the change”) is transmitted to the ball game image analysis device 100 (S225).

The reference frame changing unit 104 overwrites the changed first reference time code and / or the changed second reference time code received from the display terminal 30 on the synchronization deviation information 203, and the synchronization deviation. The content of the information 203 is confirmed (S226). As a result, the content of the synchronization deviation information 203 automatically generated by the reference frame specifying unit 103 is overwritten with the time code of the reference frame image changed by the user through the reference frame changing unit 104. Then, the ball game video analysis device 100 ends this process.

<Details of the reference frame confirmation UI>
FIG. 8 is a diagram showing a detailed example of the reference frame confirmation UI.

As shown in FIG. 8, the reference frame confirmation UI 310 includes a first image display area 210A, a second image display area 210B, an OK button 312, a change button 313, and a redo button 314. The reference frame confirmation UI 310 may not include either the change button 313 or the redo button 314.

The display terminal 30 displays the first reference frame image transmitted from the ball game image analysis device 100 in step S207 of FIG. 6 in the first image display area 210A. The display terminal 30 displays the second reference frame image transmitted from the ball game image analysis device 100 in step S207 of FIG. 6 in the second image display area 210B. As shown in FIG. 8, the first image display area 210A and the second image display area 210B may be arranged side by side. However, the arrangement mode of the first image display area 210A and the second image display area 210B is not limited to the arrangement arranged side by side. For example, the first image display area 210A and the second image display area 210B may be displayed as separate windows.

When the OK button 312 is pressed, the display terminal 30 transmits instruction information indicating no change in step S209 of FIG. 6 to the ball game video analysis device 100. The user correctly captures both the first reference frame image displayed in the first image display area 210A and the second reference frame image displayed in the second image display area 210B. In the case of the image, the OK button 312 may be pressed. As a result, as shown in step S210 of FIG. 6, the content of the synchronization shift information 203 is determined.

When the change button 313 is pressed, the display terminal 30 transmits instruction information indicating that there is a change in step S221 of FIG. 7 to the ball game video analysis device 100. The user correctly sub-acts at least one of the first reference frame image displayed in the first image display area 210A and the second reference frame image displayed in the second image display area 210B. If is not the captured image, the change button 313 may be pressed. As a result, as shown in step S223 of FIG. 7, the reference frame change UI 320 is displayed on the display terminal 30.

When the redo button 314 is pressed, the display terminal 30 transmits instruction information indicating redoing the pre-work to the ball game video analysis device 100. When this instruction is received, the ball game video analysis device 100, the first mobile terminal 20A, and the second mobile terminal 20B redo the preliminary work from step S103 of FIG. For example, if the serve in step S104 fails, the first reference frame image and / or the second reference frame image is clearly different from the ball served by the player (eg, a ball lying on the floor or another). If the player recognizes the ball being practiced, the user may press the redo button 314.

The first reference frame image includes three first frame images immediately before the first reference frame image searched in step S204 and three third images immediately after the first reference frame. A frame image of 1 may be included. In this case, the number of the first reference frame images is seven in total, including the first reference frame image searched in step S204. Similarly, the second reference frame image includes three second frame images immediately before the second reference frame image searched in step S205 and three images immediately after the second reference frame image. A second frame image of the above may be included. In this case, the number of the second reference frame images is seven in total, including the second reference frame image searched in step S205.

In this case, the display terminal 30 displays seven first reference frame images and seven second reference frame images in the first image display area 210A and the second image display area 210B, respectively. It may be displayed continuously in synchronization.

Note that the above 7 images are an example, and the number of a plurality of first and second reference frame images may be any number. Further, the plurality of first and second reference frame images are not limited to the case where each frame is continuous, and may be skipped at predetermined frame intervals.

As a result, the user not only confirms the synchronization between the one first reference frame image and the second reference frame image, but also a plurality of first reference frame images and the second reference frame image. You can easily check the synchronization with.

Although an example of displaying the reference frame confirmation UI 310 shown in FIG. 8 on the display terminal 30, the reference frame confirmation UI 310 is displayed on either the first mobile terminal 20A or the second mobile terminal 20B. It may be displayed in. Alternatively, the reference frame confirmation UI 310 for the first reference frame image may be displayed on the first mobile terminal 20A, and the reference frame confirmation UI 310 for the second reference frame image may be displayed on the second mobile terminal 20B. ..

<Details of UI for changing reference frame>
FIG. 9 is a diagram showing a detailed example of the UI for changing the reference frame.

As shown in FIG. 9, the reference frame changing UI 320 includes a first image display area 210A, a second image display area 210B, an OK button 322, and a return button 323. Further, the reference frame change UI 320 includes a 1-frame return button 324A, an N-frame return button 325A, a 1-frame advance button 326A, and an N-frame advance button 327A corresponding to the first image display area 210A. Here, N is a predetermined integer of 2 or more. Further, the reference frame change UI 320 includes a 1-frame return button 324B, an N-frame return button 325B, a 1-frame advance button 326B, and an N-frame advance button 327B corresponding to the second image display area 210B.

When the return button 323 is pressed, the display terminal 30 returns to the display of the reference frame confirmation UI 310 shown in FIG.

When the 1-frame return button 324A is pressed, the display terminal 30 selects a candidate for the first reference frame image one frame before the candidate for the first reference frame image displayed in the first image display area 210A. , Is displayed in the first image display area 210A. The same applies to the 1-frame return button 324B corresponding to the second image display area 210B.

When the N-frame return button 325A is pressed, the display terminal 30 selects a candidate for the first reference frame image N frames or earlier than the candidate for the first reference frame image displayed in the first image display area 210A. , Is displayed in the first image display area 210A. The same applies to the N-frame return button 325B corresponding to the second image display area 210B.

When the one-frame advance button 326A is pressed, the display terminal 30 selects a candidate for the first reference frame image one frame or later from the candidate for the first reference frame image displayed in the first image display area 210A. , Is displayed in the first image display area 210A. The same applies to the one-frame advance button 326B corresponding to the second image display area 210B.

When the N-frame advance button 327A is pressed, the display terminal 30 selects a candidate for the first reference frame image after the N frame rather than a candidate for the first reference frame image displayed in the first image display area 210A. , Is displayed in the first image display area 210A. The same applies to the N-frame advance button 327B corresponding to the second image display area 210B.

The user presses the buttons 324A to 327A and 324B to 327B described above so that the image in which the sub-action is correctly captured is displayed in both the first image display area 210A and the second image display area 210B. You may operate it.

When the OK button 322 is pressed, the display terminal 30 has a time code of a candidate for the first reference frame image being displayed in the first image display area 210A (first reference time code after the change), and The time code of the candidate for the second reference frame image displayed in the second image display area 210B (the changed second reference time code) is transmitted to the ball game image analysis device 100. This process corresponds to step S225 in FIG. The user may press the OK button 322 after operating so that the image in which the sub-action is correctly captured is displayed in both the first image display area 210A and the second image display area 210B.

<Analysis result screen>
FIG. 10 is a diagram showing a display example of the analysis result.

The first mobile terminal 20A and the second mobile terminal 20B photograph the player's serve practice and generate and transmit the first video 201A and the second video 201B, respectively. The first moving image 201A and the second moving image 201B may be different from the first moving image 201A and the second moving image 201B taken for the preliminary work.

The trajectory analysis unit 105 corrects the synchronization deviation of the frame image between the first moving image 201A and the second moving image 201B in which the practice of serving is taken based on the synchronization deviation information 203, and then the movement trajectory of the ball. Is analyzed, and analysis result information 204 is generated. For example, in the synchronization deviation information 203, when the second reference time code is later than the first reference time code, the locus analysis unit 105 between the second reference time code and the first reference time code. After correcting to delay each frame image of the first moving image 201A by the difference time of the first moving image 201, each frame image of the first moving image 201 and each frame image of the second moving image 201B are synchronized to form a ball. Analyze the movement trajectory of. As a result, the delay in assigning the time code by the second mobile terminal 20B is corrected, and the movement locus of the ball with sufficient accuracy can be calculated.

The result display unit 106 causes the display terminal 30 to display the analysis result screen 330 based on the analysis result information 204. For example, on the analysis result screen 330, as shown in FIG. 10, the movement locus 331 of the ball indicated by the analysis result information 204 may be superimposed and displayed on the reproduction of the first moving image 201A or the second moving image 201B. .. In addition, the analysis result screen 330 may display the height and velocity of the ball at the moment when the ball hit by the hand leaves the hand. In addition, the analysis result screen 330 may display the height and velocity of the ball at the moment when the ball passes over the net.

As a result, the player can confirm in detail the movement trajectory of the ball by the serve in addition to the movement of the body at the time of the serve from the analysis result screen 330.

In the above, as a preliminary work, a video for synchronization adjustment is shot in advance, synchronization adjustment and confirmation are performed, then a video for analysis processing is shot, and the video on which the trajectory of the ball is superimposed is analyzed. An example of displaying as is described. However, in the present embodiment, without distinguishing between the moving image for synchronization adjustment and the moving image for analysis processing, for example, in the first moving image taken, the synchronization adjustment and the analysis processing are continuously performed. May be good. Further, the synchronization adjustment may be executed at an arbitrary timing while the detection of the synchronization deviation is automated and the analysis processing for the sequentially shot moving images is performed.

<Modification example>
In the above description, as information indicating the frame synchronization deviation, the first time code assigned to the first frame image by the first mobile terminal 20A and the second time code assigned to the second frame image by the second mobile terminal 20B. An example of using the time code of 2 has been described. However, the information indicating the frame synchronization deviation is not limited to the time code.

For example, as information indicating the deviation of frame synchronization, the frame number of the first frame image in the first moving image 201A generated by the first mobile terminal 20A (hereinafter referred to as "first frame number") and the second A frame image (hereinafter referred to as "second frame number") in the second moving image 201B generated by the mobile terminal 20B may be used. In this case, the reference frame specifying unit 103 has a first frame number of the first reference frame image (hereinafter referred to as "first reference frame number") and a second frame number of the second reference frame image (hereinafter referred to as "first reference frame number"). “Second reference frame number”) may be written in the synchronization shift information 203. Then, the reference frame changing unit 104 uses the frame number of the changed first reference frame image (hereinafter referred to as “changed first reference frame number”) selected through the reference frame changing UI 320, and / or , The frame number of the changed second reference frame image (hereinafter referred to as “the changed second reference frame number”) may be overwritten on the synchronization shift information 203.

(Embodiment 2)
In the second embodiment, the target video and the comparative video are specified from a plurality of videos of the action of the person (player) performing the ball game, and the proficiency level of the action and the improvement points of the form can be grasped. The analysis system will be described. In the second embodiment, the same reference numbers may be assigned to the components common to the first embodiment, and the description may be omitted.

Further, in the second embodiment, a case where the action practice by the player is a volleyball serve practice will be described as an example. However, the second embodiment is not limited to this example, and various exercises such as tennis serve or volley practice, badminton serve or smash practice, soccer heading practice, baseball batting practice, and golf swing practice. It can be applied to practice actions.

<Structure of ball game video analysis system>
FIG. 11 is a diagram showing a configuration example of the ball game video analysis system 10 according to the second embodiment.

The ball game video analysis system 10 according to the second embodiment is the same as the ball game video analysis system 10 according to the first embodiment shown in FIG. However, as shown in FIG. 11, the first mobile terminal 20A (camera) may be installed near the net on one side of the court toward the serving player. The second mobile terminal 20B (camera) may be installed near the net on the other side of the court towards the serving player.

Further, the ball game video analysis system 10 described in the second embodiment may be configured so that the synchronization deviation or the like is corrected by the method described in the first embodiment.

<Structure of ball game video analysis device>
FIG. 12 is a block diagram showing a configuration example of the ball game video analysis device 100 according to the second embodiment.

As shown in FIG. 12, the ball game video analysis device 100 includes a moving image receiving unit 101, a trajectory analysis unit 105, a posture analysis unit 131, a sub-action analysis unit 132, a record management unit 133, and a display control unit 134.

The video receiving unit 101 receives the first video 201A from the first mobile terminal 20A, and receives the second video 201B from the second mobile terminal 20B. In the first moving image 201A and the second moving image 201B, for example, a scene in which the player serves is shot.

The trajectory analysis unit 105 identifies the movement trajectory of the ball from the first moving image 201A and the second moving image 201B by using, for example, the technique disclosed in Patent Document 1, and generates the trajectory information 220. The locus information 220 includes a plurality of information indicating the correspondence relationship between the time code at a certain time point of the moving image and the three-dimensional position of the ball at the time code.

The posture analysis unit 131 generates posture model information 221 indicating the posture of the player during the period in which the action is performed, based on the first video 201A and the second video 201B. For example, the posture analysis unit 131 uses a known joint estimation technique to determine the three-dimensional position of each predetermined joint of the player (hereinafter referred to as “joint position”) from the first moving image 201A and the second moving image 201B. The movement of each joint position is specified. The posture analysis unit 131 generates information indicating each of the specified joint positions and the movement of each joint position as posture model information 221. The details of the posture model information 221 will be described later (see FIG. 14).

The sub-action analysis unit 132 analyzes each sub-action constituting the action captured in the first moving image 201A and the second moving image 201B based on the locus information 220 and / or the posture model information 221. Then, the sub-action analysis unit 132 generates sub-action information 222 indicating the occurrence timing of each analyzed sub-action. For example, a jumping serve, which is an example of a volleyball action, is a series of sub-actions such as "run-up", "toss", "jump", and "hit" which means that the hand that hit the serve hits the ball. It is composed of. The details of the sub-action will be described later (see FIG. 15).

The record management unit 133 manages a plurality of record information 200. The recording information 200 is generated from the shooting date and time of the first moving image 201A and the second moving image 201B, the first moving image 201A and the second moving image 201B, and the first moving image 201A and the second moving image 201B. It is information showing the correspondence relationship with the locus information 220, the posture model information 221 and the sub-action information 222. That is, the recording information 200 is information indicating the correspondence relationship between the moving image of the action performed by the player at a certain date and time and the locus information 220, the posture model information 221 and the sub action information 222 generated from the moving image. It can be said that.

The display control unit 134 causes the display terminal 30 to display a UI for selecting the record information 200 based on the locus information 220 from the plurality of record information 200 managed by the record management unit 133. Further, the display control unit 134 causes the display terminal 30 to display the first moving image 201A or the second moving image 201B having a corresponding relationship in the recorded information 200 selected through the UI.

Further, the display control unit 134 has, for example, the first record information 200A (see FIG. 13) and the first record information 200A from among the plurality of record information 200 managed by the record management unit 133 through the UI displayed on the display terminal 30. When the recording information 200B (see FIG. 13) of No. 2 is selected, the following processing may be performed. That is, the display control unit 134 has a correspondence relationship between the first moving image 201A or the second moving image 201B in the first recording information 200A and the first moving image 201A or the first moving image 201A or the second moving image 201B having a correspondence relationship in the second recording information 200B. A UI (hereinafter referred to as "video comparison UI") 400A for displaying the moving image 201B of 2 side by side may be displayed on the display terminal 30. The details of the video comparison UI 400A will be described later (see FIGS. 16 and 17).

Further, the display control unit 134 uses, for example, the first record information 200A and the second record information 200A from among the plurality of record information 200 managed by the record management unit 133 through the video comparison UI 400A displayed on the display terminal 30. When the record information 200B is selected, the following processing may be performed (see FIG. 14). That is, the display control unit 134 has the posture model information 221 that has a correspondence relationship with the first posture model 231A generated from the posture model information 221 that has a correspondence relationship with the first recording information 200A and the second recording information 200B. The display terminal 30 may display a UI (hereinafter referred to as “posture comparison UI”) 400C for displaying the second posture model 231B generated from the above side by side or in an overlapping manner. The details of the posture comparison UI 400C will be described later (see FIGS. 18 to 21).

Further, the display control unit 134 has a locus information corresponding to the movement locus 241A of the moving body (ball) generated from the locus information 220 having a correspondence relationship in the first recording information 200A and the locus information having a correspondence relationship in the second recording information 200B. The movement locus 241B of the moving body (ball) generated from the 220 may be displayed on the display terminal 30 together with the

posture models

231A and 231B in the posture comparison UI400C (see FIGS. 18 to 21).

Further, the display control unit 134 may display the sub-action selection UI (447) capable of selecting one of the plurality of sub-actions on the display terminal 30. Then, the display control unit 134 causes the display terminal 30 to display the

posture models

231A and 231B at the occurrence timing of one sub-action selected through the sub-action selection UI (447) on the posture comparison UI (400D). Good (see Figure 20).

<Display terminal configuration>
FIG. 13 is a block diagram showing a configuration example of the display terminal 30 according to the second embodiment.

The display terminal 30 has an information transmission / reception unit 151 and a UI control unit 152.

The information transmission / reception unit 151 transmits / receives information through the ball game video analysis device 100 and the communication network N. For example, the information transmission / reception unit 151 receives a plurality of recording information 200 (for example, the first recording information 200A and the second recording information 200B) from the ball game video analysis device 100.

The UI control unit 152, in cooperation with the display control unit 134 of the ball game image analysis device 100, displays the image comparison UI 400A, the posture comparison UI 400C, and the like on a display which is an example of the output device 1002 (see FIG. 23). Further, the UI control unit 152 detects an input from the user to the UI through a touch panel which is an example of the input device 1001 (see FIG. 23). When the UI control unit 152 detects an input to the UI, the UI control unit 152 transmits the input content to the ball game video analysis device 100 through the information transmission / reception unit 151.

<Details of posture model>
FIG. 14 is a diagram for explaining an example of the posture model.

The posture model information 221 includes information indicating each joint position of the player and the movement of each joint position.

For example, the display control unit 134 or the UI control unit 152 refers to the posture model information 221 that has a corresponding relationship in the first recording information 200A, and refers to each joint position at the timing of the sub-action “jump” during the action period (FIG. 14). (White circle and black circle) are specified. Similarly, the display control unit 134 or the UI control unit 152 identifies each joint position at the timing of the sub-action "jump" in the action period with reference to the posture model information 221 that is in a corresponding relationship in the second recording information 200B. do. Then, the display control unit 134 generates the

player posture models

231A and 231B as shown in FIG. 14 by connecting the specified joint positions with a line based on the structure of the human body. Since the

posture models

231A and 231B are three-dimensional models, the display control unit 134 or the UI control unit 152 can display the posture model by arbitrarily changing the viewpoint in the 3D space by an operation such as 3D rotation. can.

Further, as shown in FIG. 14, the display control unit 134 or the UI control unit 152 sets the first posture model 231A and the second posture model 231B at the timing of the sub-action “jump” and the neck joint position 232A. It can be displayed in an overlapping manner based on 232B.

The first posture model 231A and the second posture model 231B may be generated based on moving images of serves taken by the same player at different dates and times. In this case, the user can visually recognize the difference in posture at the time of the sub-action "jump" performed by the same player at different dates and times from the superimposed display of the first posture model 231A and the second posture model 231B. .. Therefore, the user can easily confirm the progress of the serve of the same player, for example.

The first posture model 231A and the second posture model 231B may be generated based on moving images of serves performed by different players. In this case, the user
From the superimposed display of the first posture model 231A and the second posture model 231B, it is possible to visually recognize the difference in posture at the time of the sub-action "jump" performed by different players. Therefore, the user can easily confirm, for example, the difference in posture at the time of serving between the model player and the player who is practicing.

Further, by referring to the posture model information 221, the timing at which the player jumps and the maximum reaching point when the player jumps can be specified. For example, the sub-action analysis unit 132 may detect the timing when the joint positions of both ankles of the player are separated from the ground by a certain distance or more as the timing when the player jumps. For example, the sub-action analysis unit 132 may detect the timing when the

joint positions

232A and 232B of the player's neck are at the highest point as the timing when the player's jump reaches the highest point.

Note that the

posture models

231A and 231B are not limited to the model in which each joint position is connected by a line as shown in FIG. For example, the

attitude models

231A and 231B may be texture-mapped three-dimensional CG (Computer Graphics) models.

<Analysis of sub-actions>
FIG. 15 is a diagram for explaining sub-actions constituting the jumping serve.

As shown in FIG. 15, jumping serve, which is an example of volleyball action, includes "running", "toss", "jump", and "hit" which means that the hand hitting the serve hits the ball. It consists of a series of sub-actions.

The sub-action analysis unit 132 specifies a frame image at the timing when each sub-action occurs based on the ball trajectory information 220 and / or the player's posture model information 221.

For example, the sub-action analysis unit 132 refers to the ball locus information 220 to specify a frame image at the timing when the ball's movement locus changes from a stationary state to an upward movement. Then, the sub-action analysis unit 132 writes the timing (for example, time information) at which the specified frame image is taken in the sub-action information 222 as the occurrence timing of the sub-action "toss".

For example, the sub-action analysis unit 132 refers to the posture model information 221 of the player and specifies a frame image at the timing when the joint positions of both ankles of the player are separated from the ground by a certain distance or more. Then, the sub-action analysis unit 132 writes the timing at which the specified frame image is taken in the sub-action information 222 as the occurrence timing of the sub-action “jump”.

For example, the sub-action analysis unit 132 refers to the trajectory information 220 of the ball and specifies a frame image at the timing when the amount of movement of the ball in the direction approaching the net changes to a predetermined threshold value or more. Then, the sub-action analysis unit 132 writes the timing at which the specified frame image is captured in the sub-action information 222 as the occurrence timing of the sub-action “hit”.

As a result, information (for example, time information) indicating the timing at which each sub-action constituting the action occurs is written in the sub-action information 222. Therefore, by referring to the sub-action information 222, the frame image at the time of the desired sub-action can be specified from the moving image. Further, by referring to the sub-action information 222, the posture model at the time of the desired sub-action can be specified from the posture model information 221.

<Details of UI for video comparison>
FIG. 16 is a diagram showing a display example of the video comparison UI 400A. The video comparison UI 400A is an example of a comparison UI, and is a UI for comparing moving images related to a plurality of recorded information 200 that are different from each other.

As shown in FIG. 16, the video comparison UI 400A includes selection lists 401A, 401B,

display areas

402A, 402B, add button 403, seek bar 410, play button 411, pause button 412, slow play button 413, repeat button 414, and so on. It includes a side-by-side button 415, an enlargement button 416, a camera switching button 417, a video inversion button 418, and a 3D switching button 419.

In the selection lists 401A and 401B, the information specified from the locus information 220 related to the record information 200 is displayed in a list format. The information specified from the locus information 220 includes, for example, information indicating the height, speed, and angle of the ball when passing through the net in the serve. One line of the selection lists 401 and 401B may correspond to one recording information 200.

Further, the recorded information 200 may include information indicating the evaluation of the action (for example, serve). In this case, the selection lists 401A and 401B may also display the evaluation of the action, as shown in FIG. The evaluation of the action may be manually entered by the user. Alternatively, the evaluation of the action may be automatically determined based on the height, speed and angle of the ball when passing through the net. In this case, the evaluation of the action may be determined based on whether or not the angle of the ball when passing through the net is equal to or greater than a predetermined threshold value. Here, the threshold value referred to in the evaluation of the action may be arbitrarily set by the user.

In the display area 402A, the first moving image 201A or the second moving image 201B related to the first recording information 200A selected from the selection list 401A (for example, the line filled with the dots in FIG. 16) is displayed. In the display area 402B, the first moving image 201A or the second moving image 201B related to the second recording information 200B (for example, the line filled with the dots in FIG. 16) selected from the selection list 401B is displayed.

As shown in FIG. 16, in the video comparison UI 400A, a set of the selection list 401A and the display area 402A (hereinafter referred to as the “first set”) and a set of the selection list 401B and the display area 402B (hereinafter referred to as the “second set”). ") May be displayed side by side. As a result, the user selects a line (recording information 200) to be compared from the selection list 401A and the selection list 401B, respectively, and compares the moving images related to the selected line displayed in the

display areas

402A and 402B, respectively. You can watch while watching.

The selection list 401A included in the first set includes records generated from the first moving image 201A and the second moving image 201B taken on the same day using the first mobile terminal 20A and the second mobile terminal 20B. Information 200 may be displayed at any time.

When the add button 403 is pressed, the display control unit 134 adds a new set including a new selection list and a display area to the video comparison UI 400A. In this case, the UI control unit 152 may inquire of the user about the date of the recorded information 200 to be displayed in the new selection list. Then, the UI control unit 152, in cooperation with the display control unit 134, acquires one or a plurality of record information 200 including the date input by the user from the record management unit 133, and based on the acquired record information 200. , You may want to display the selection list of the new set added.

The seek bar 410 is a bar indicating the playing time of the moving image displayed in the

display areas

402A and 402B. When the user selects a certain position of the seek bar 410, the UI control unit 152 may play the moving image from the time corresponding to the selected position.

When the play button 411 is pressed, the UI control unit 152 starts playing the moving image displayed in the

display areas

402A and 402B.

When the pause button 412 is pressed, the UI control unit 152 pauses the playback of the moving image displayed in the

display areas

402A and 402B.

When the slow playback button 413 is pressed, the UI control unit 152 slow-plays (for example, plays back at 1/2 speed) the moving images displayed in the display areas 402A and 402.

When the repeat button 414 is pressed, the UI control unit 152 repeatedly plays back the moving image displayed in the

display areas

402A and 402B. The slow play button 413 and the repeat button 414 may be used together.

When a plurality of sets in the video comparison UI 400A are selected and the arranging button 415 is pressed, the UI control unit 152 is a UI for comparing the sub-actions of the selected plurality of sets for sub-action video comparison. Switch to the display of UI400B (see FIG. 17). The details of the sub-action video comparison UI 400B will be described later (see FIG. 17).

When at least one set in the video comparison UI 400A is selected and the enlargement button 416 is pressed, the UI control unit 152 expands the range including the player of the moving image displayed in the display area of the selected set. indicate. The range to be enlarged may be arbitrarily specified by the user. Further, the enlargement ratio of the display may be such that the user can arbitrarily specify the magnification.

When at least one set in the video comparison UI 400A is selected and the camera switching button 417 is pressed, the UI control unit 152 uses the moving image displayed in the display area of the selected set (for example, the first moving image). 201A) is switched to a moving image (for example, a second moving image 201B) taken by another mobile terminal. Thereby, for example, the moving images displayed in the

display areas

402A and 402B can be aligned with the moving images shot from the same direction.

When at least one set in the video comparison UI 400A is selected and the video inversion button 418 is pressed, the UI control unit 152 inverts the moving image displayed in the display area of the selected set. This makes it easy to compare, for example, the actions of a right-handed player with the actions of a left-handed player.

When at least one set in the video comparison UI 400A is selected and the 3D switching button 419 is pressed, the UI control unit 152 performs the posture comparison UI 400C for comparing the posture models of the selected set (see FIG. 18). ) Is displayed. The details of the posture comparison UI 400C will be described later (see FIG. 18).

FIG. 17 is a diagram showing a display example of the sub-action video comparison UI 400B.

For example, in the video comparison UI 400A shown in FIG. 16, when the first set and the second set are selected and the arranging button 415 is pressed, the UI control unit 152 is used for sub-action video comparison shown in FIG. Display UI400B.

The sub-action video comparison UI 400B includes

display areas

421A and 421B, and a plurality of sub-action selection buttons 420 extended from the side-by-side buttons 415. The sub-action selection button 420 is an example of the sub-action selection UI.

The

display areas

421A and 421B are arranged side by side as shown in FIG. The UI control unit 152 expands the range including the player of the moving image displayed in the first set and displays it in the display area 421A. Similarly, the UI control unit 152 expands the range including the player of the moving image displayed in the second set and displays it in the display area 421B. At this time, the UI control unit 152 may adjust the display of the moving image in the

display areas

421A and 421B so that the joint position of the player's neck is located at the center of the

display areas

421A and 421B.

The plurality of sub-action selection buttons 420 are buttons for selecting sub-actions, and correspond to, for example, "toss", "jump", and "hit" sub-actions.

For example, when the user selects the sub-action selection button 420 "hit", the UI control unit 152 performs the following processing. That is, the UI control unit 152 refers to the sub-action information 222 related to the first recording information 200A, and refers to the frame at the timing when the sub-action "hit" is shot from the moving images displayed in the first set. Identify the image. Then, the UI control unit 152 displays the specified frame image in the display area 421A. Similarly, the UI control unit 152 refers to the sub-action information 222 related to the second recording information 200B, and refers to the timing at which the sub-action "hit" is shot from the moving images displayed in the second set. Identify the frame image. Then, the UI control unit 152 displays the specified frame image in the display area 421B. When the play button 411 is pressed in this state, the UI control unit 152 may start playing the moving image from the frame image at the timing of the specified sub-action “hit”. This allows the user to easily compare, for example, the player's footage at the time of the sub-action "hit" of the serve performed at different dates and times. If the timing of the frame image of the sub-action "hit" is aligned, the moving image may be played including "run-up" and "toss".

<Details of posture comparison UI>
FIG. 18 is a diagram showing an example of a posture comparison UI when the posture models are displayed side by side. FIG. 19 is a diagram showing an example of a posture comparison UI when the posture models are superimposed and displayed. The posture comparison UI 400C is an example of a comparison UI, and is a UI for comparing posture model information 221 related to a plurality of different recorded information 200.

For example, in the video comparison UI 400A shown in FIG. 16, when the first set and the second set are selected and the 3D switching button 419 is pressed, the UI control unit 152 takes the posture shown in FIG. 18 or FIG. The comparison UI 400C is displayed.

The posture comparison UI 400C includes a display area 434, a seek bar 410, a play button 411, a pause button 412, a slow play button 413, a repeat button 414, a video switching button 441, a trajectory display switching button 442, a ball display switching button 443, and a player display. It includes a changeover button 444, a superposition separation changeover button 445, and a side-by-side button 446.

As shown in FIGS. 18 and 19, the UI control unit 152 includes a posture model 231A, a ball 240A, and a ball movement locus 241A corresponding to the first set, and a posture model 231B, a ball 240B corresponding to the second set. And the movement locus 241B of the ball are displayed in the display area 434. Hereinafter, the posture model 231A, the ball 240A, and the ball movement locus 241A corresponding to the first set are collectively referred to as the first posture model and the like. Similarly, the posture model 231B, the ball 240B, and the ball movement locus 241B corresponding to the second set are collectively referred to as a second posture model and the like.

By performing a predetermined operation on the display area 434, the user can arbitrarily rotate the first and second posture models and the like displayed in the display area 434 in 3D space. The viewpoint can be changed and displayed. As a result, the user can confirm and compare the postures of the players at the time of serving at different dates and times from various viewpoints.

The seek bar 410 is a bar indicating the playing time of the first and second posture models and the like displayed in the display area 434. That is, the posture model or the like displayed in the display area 434 represents the posture or the like of the player at the timing of the reproduction time being displayed on the seek bar 410.

When the play button 411 is pressed, the UI control unit 152 starts playing the actions of the first and second posture models and the like displayed in the display area 434. During playback, in the display area 434, the first and second posture models and the like are displayed like an animation movie. During the playback of the animation videos of the first and second posture models, the first video 201A and the second video 201B, which are the actual videos, are synchronizedly played in a separate window (not shown). May be good.

When the pause button 412 is pressed, the UI control unit 152 suspends the operations of the first and second posture models displayed in the display area 434.

When the slow playback button 413 is pressed, the UI control unit 152 slow-plays (for example, plays at 1/2 speed) the operations of the first and second posture models displayed in the display area 434.

When the repeat button 414 is pressed, the UI control unit 152 repeatedly reproduces the operations of the first and second posture models and the like displayed in the display area 434. The slow play button 413 and the repeat button 414 may be used together.

When the video switching button 441 is pressed, the UI control unit 152 switches to the display of the video comparison UI 400A shown in FIG.

When the locus display switching button 442 is pressed, the UI control unit 152 switches between displaying and hiding the

movement loci

241A and 241B in the display area 434.

When the ball display switching button 443 is pressed, the UI control unit 152 switches between displaying and hiding the

balls

240A and 240B in the display area 434.

When the player display switching button 444 is pressed, the UI control unit 152 switches between displaying and hiding the

posture models

231A and 231B in the display area 434.

When the superimposition separation switching button 445 is pressed, the UI control unit 152 displays in the display area 434 by separating the first posture model and the like as shown in FIG. 18 and the second posture model and the like, and in FIG. The display is switched between the first posture model and the like as shown and the display in which the second posture model and the like are superimposed. When the first posture model 231A and the second posture model 231B are displayed on top of each other, the UI control unit 152 may display the first posture model 231A and the second posture model 231B on top of each other with reference to the neck

joint positions

232A and 232B. Further, when the UI control unit 152 displays the first posture model 231A and the second posture model 231B in an overlapping manner, as shown in FIG. 19, the UI control unit 152 matches the movement of the positions of the

posture models

231A and 231B with the ball 240A. , 240B and the positions of the

ball movement loci

241A and 241B are also moved and displayed.

When the line-up button 446 is pressed, the UI control unit 152 displays the sub-action posture comparison UI 400D (see FIG. 20). Details of the UI400D for sub-action posture comparison will be described later (see FIG. 20).

Note that the posture comparison UI400C may include a model inversion button (not shown). When at least one posture model in the posture comparison UI 400C is selected and the model reversal button is pressed, the UI control unit 152 displays the selected posture model in a left-right reversal. Thereby, for example, the posture of a right-handed player and the posture of a left-handed player can be easily compared.

FIG. 20 is a diagram showing an example of a UI for sub-action posture comparison.

For example, when the line-up button 446 is pressed in the posture comparison UI 400C shown in FIGS. 18 and 19, the UI control unit 152 displays the sub-action posture comparison UI 400D shown in FIG. 20.

The sub-action posture comparison UI 400D includes

display areas

451A and 451B, and a plurality of sub-action selection buttons 447 extended from the side-by-side buttons 446. The sub-action selection button 447 is an example of the sub-action selection UI.

The

display areas

451A and 451B are arranged side by side as shown in FIG. The UI control unit 152 enlarges the first posture model and the like (231A, 240A, 241A) and displays them in the display area 451A. Similarly, the UI control unit 152 enlarges the second posture model and the like (231B, 240B, 241B) and displays them in the display area 451B. At this time, the UI control unit 152 may adjust the display of the

posture models

231A and 231B so that the

joint positions

232A and 232B of the neck of the posture model are located at the center of the

display areas

451A and 451B.

The plurality of sub-action selection buttons 447 are buttons for selecting sub-actions, and correspond to, for example, "toss", "jump", and "hit" sub-actions.

For example, when the user selects the sub-action selection button 447 "jump", the UI control unit 152 performs the following processing. That is, the UI control unit 152 specifies the first posture model and the like of the timing of the sub-action "jump" with reference to the sub-action information 222 related to the first recorded information 200A. Then, the UI control unit 152 displays the specified first posture model and the like in the display area 451A. Similarly, the UI control unit 152 specifies the second posture model and the like of the timing of the sub action "jump" with reference to the sub action information 222 related to the second recorded information 200B. Then, the UI control unit 152 displays the specified second posture model and the like in the display area 451B. When the play button 411 is pressed in this state, the UI control unit 152 may start playing the motion of the posture model or the like from the posture model or the like at the timing of the specified sub-action “jump”. This allows the user to easily compare, for example, the posture of the player at the time of the sub-action "jump" of the serve performed at different dates and times and the movement thereof. If the timing of the frame image of the sub-action "jump" is aligned, the moving image may be played including "run-up" and "toss".

FIG. 21 is a diagram for explaining an example in the case of selecting a posture model in the posture comparison UI 400C.

When the user presses the right sidebar 430 (see FIGS. 18 and 19) included in the posture comparison UI 400C, the UI control unit 152 displays the selection list 431 as shown in FIG. 21. In the selection list 431, the information specified from the locus information 220 related to the recorded information 200 is displayed in a list format as in the case of FIG.

In addition, a check box 432 is displayed in each line of the selection list 431. The user sets the check box 432 of the row to be compared from the selection list 431 to ON.

The UI control unit 152 identifies the recording information 200 in which the check box 432 is set to ON, and displays the posture model or the like generated from the posture model information 221 related to the specified recording information 200 in the display area 434. As a result, the user can display a desired posture model or the like in the display area 434 and compare them.

Further, the ball filtering button 433 may be displayed on the sidebar 430. When the ball filtering button 433 is pressed, the UI control unit 152 displays a condition box (not shown) for filtering the recording information 200 to be displayed in the selection list 431. In the condition box, at least one of the ball height, speed, angle and evaluation, which is the threshold for filtering, may be entered. Then, the UI control unit 152 displays the recording information 200 that matches the conditions input in the condition box as the selection list 431. As a result, the user can efficiently extract the recorded information 200 as a comparison candidate from the large amount of recorded information 200.

<Hardware configuration>
The embodiments according to the present disclosure have been described in detail with reference to the drawings. The functions of the first mobile terminal 20A, the second mobile terminal 20B, the display terminal 30, and the ball game video analysis device 100 described above can be realized by a computer program.

FIG. 22 is a diagram showing a hardware configuration of a computer that realizes the functions of the ball game video analysis device 100 by a program.

The computer 1000A includes an input device 1001 such as a keyboard or mouse and a touch pad, an output device 1002 such as a display or a speaker, a CPU (Central Processing Unit) 1003, a GPU (Graphics Processing Unit) 1004, and a ROM (Read Only Memory) 1005. RAM (RandomAccessMemory) 1006, HDD (HardDiskDrive), SSD (SolidStateDrive) or storage device 1007 such as flash memory, DVD-ROM (DigitalVersatileDiskReadOnlyMemory) or USB (UniversalSerialBus) memory, etc. A reading device 1008 that reads information from the recording medium of the above, and a transmitting / receiving device 1009 that communicates by wire or wirelessly via the communication network N are provided, and each part is connected by a bus 1010.

Then, the reading device 1008 reads the program from the recording medium on which the program for realizing the functions of the above devices is recorded, and stores the program in the storage device 1007. or,
The transmission / reception device 1009 communicates with the server device connected to the communication network N, and stores the program downloaded from the server device for realizing the function of each device in the storage device 1007.

Then, the CPU 1003 copies the program stored in the storage device 1007 to the RAM 1006, and sequentially reads and executes the instructions included in the program from the RAM 1006, whereby the functions of the above devices are realized.

FIG. 23 is a diagram showing a hardware configuration of a computer that programmatically realizes the functions of the first mobile terminal 20A, the second mobile terminal 20B, and the display terminal 30. The components described with reference to FIG. 11 may be designated by the same reference numerals and the description thereof may be omitted.

This computer 1000B includes an input device 1001, an output device 1002, a CPU 1003, a GPU 1004, a ROM 1005, a RAM 1006, a storage device 1007, a reading device 1008, a transmission / reception device 1009, and a camera device 1011.

The camera device 1011 includes an image sensor and generates captured images (moving images and still images). The generated captured image is stored in the storage device 1007. The captured image stored in the storage device 1007 may be transmitted to an external server or the like through the transmission / reception device 1009 and the communication network N.

Each functional block used in the description of the above embodiment is typically realized as an LSI which is an integrated circuit. These may be individually integrated into one chip, or may be integrated into one chip so as to include a part or all of them. Although it is referred to as LSI here, it may be referred to as IC, system LSI, super LSI, or ultra LSI depending on the degree of integration.

Furthermore, if an integrated circuit technology that replaces an LSI appears due to advances in semiconductor technology or another technology derived from it, it is naturally possible to integrate functional blocks using that technology.

(Summary of this disclosure)
The ball game video analysis device (100) according to the present disclosure is a first moving image (201A) and a second moving image (201A) in which the actions of a person (for example, a player) with respect to a moving body (for example, a ball) used in the ball game are photographed from different positions. Based on (201B), the locus analysis unit (105) that generates the locus information (220) indicating the locus of the moving body and the information indicating the correspondence between the first moving image and the second moving image and the locus information. A record management unit (133) that manages a plurality of recorded information (200) and a UI (User Interface) that selects recorded information based on the trajectory information from the plurality of recorded information are displayed and selected through the UI. A display control unit (134) for displaying a first moving image or a second moving image having a corresponding relationship in the recorded information is provided.

According to this configuration, the recorded information is selected based on the locus information from the plurality of recorded information managed by the record management unit, and the corresponding moving images are displayed in the selected recorded information. In the video, the action of a person performing a ball game is recorded. Therefore, the user can easily find and watch a video useful for practicing ball games, such as a video in which the action succeeds or fails, based on the trajectory information, from a plurality of videos in which the action of the person is shot.

Further, when the first recorded information and the second recorded information are selected from the plurality of recorded information, the display control unit has a first moving image or a second moving image having a corresponding relationship in the first recorded information. The moving image and the first moving image or the second moving image having a corresponding relationship in the second recorded information may be displayed side by side.

According to this configuration, the moving image related to the selected first recording information and the moving image related to the selected second recording information are displayed side by side. Therefore, the user can watch a moving image of the actions displayed side by side and use it for practicing ball games.

Further, the ball game video analysis device may further include a posture analysis unit (131) that generates posture model information (221) indicating the posture of the person at the time of action based on the first moving image and the second moving image. .. The recorded information may be information indicating the correspondence between the first moving image and the second moving image, the locus information, and the posture model information. When the first recorded information and the second recorded information are selected from the plurality of recorded information, the display control unit generates a posture model (posture model) generated from the posture model information having a corresponding relationship in the first recorded information. 231A) and the posture model (231B) generated from the posture model information corresponding to each other in the second recorded information may be displayed side by side or overlapped.

According to this configuration, the posture model related to the selected first recorded information and the posture model related to the selected second recorded information are displayed side by side or overlapped. Therefore, the user can see the posture model at the time of action displayed side by side or overlapped, and can use it for practicing ball games.

Further, the display control unit has a movement locus (241A) of the moving body generated from the locus information having a correspondence relationship in the first recording information, and a movement generated from the locus information having a correspondence relationship in the second recording information. The movement locus (241B) of the body may be displayed together with the posture model.

According to this configuration, the movement locus of the moving body is displayed along with the posture model. Therefore, the user can see the movement of the moving body at the time of the action displayed together with the posture model and use it for practicing the ball game.

In addition, the ball game video analysis device includes a sub-action analysis unit (132) that generates sub-action information (222) indicating the occurrence timing of each of the plurality of sub-actions constituting the action based on the trajectory information and the posture model information. You may prepare further. The recorded information may be information indicating the correspondence between the first moving image and the second moving image, the trajectory information, the posture model information, and the sub-action information. The display control unit displays a sub-action selection UI (420) capable of selecting one of a plurality of sub-actions, and displays a posture model at the occurrence timing of one sub-action selected through the sub-action selection UI. May be displayed.

According to this configuration, the posture model at the occurrence timing of one sub-action selected through the sub-action selection UI is displayed. Therefore, the user can display the posture model at the time of the desired sub-action through the sub-action selection UI, which can be useful for practicing ball games.

Although the embodiment has been described above with reference to the attached drawings, the present disclosure is not limited to such an example. It is clear that a person skilled in the art can come up with various modifications, modifications, substitutions, additions, deletions, and equality within the scope of the claims. It is understood that it belongs to the technical scope of the present disclosure. Further, each component in the above-described embodiment may be arbitrarily combined as long as the gist of the invention is not deviated.

Note that this application is based on a Japanese patent application (Japanese Patent Application No. 2020-045203) filed on March 16, 2020, the contents of which are incorporated herein by reference.

The technology of the present disclosure can be used for analysis of sports using mobile objects.

10 Ball game video analysis system 20A 1st mobile terminal 20B 2nd mobile terminal 30 Display terminal 100 Ball game video analysis device 101 Video receiver 102 Image conversion unit 103 Reference frame identification unit 104 Reference frame change unit 105 Trajectory analysis unit 106 Result display Unit 131 Posture analysis unit 132 Sub-action analysis unit 133 Recording management unit 134 Display control unit 151 Information transmission / reception unit 152 UI control unit 200 Recording information 200A First recording information 200B Second recording information

201A First video

201B Second Movie 202A First frame image group 202B Second frame image group 203 Synchronous deviation information 204 Analysis result information 210A First image display area 210B Second

image display area

211A, 211B Ball detection area 220 Trajectory information 221 Attitude model Information 222 Sub-action information 231A First posture model 231B

Second posture model

232A, 232B Neck

joint position

240A,

240B Ball

241A, 241B Ball movement trajectory 310 Reference frame confirmation UI
312 OK button 313 Change button 314 Redo button 320 Reference frame change UI
322 OK button 323

Return button

324A, 324B 1

frame return button

325A, 325B N

frame return button

326A, 326B 1

frame advance button

327A, 327B N frame advance button 330 Analysis result screen 331 Ball movement trajectory 400A Video comparison UI
400B Sub-action video comparison UI
400C Posture comparison UI
400D Sub-action posture comparison UI
401A,

401B Selection list

402A, 402B Display area 403 Add button 410 Seek bar 411 Play button 412 Pause button 413 Slow play button 414 Repeat button 415 Arrange button 416 Enlarge button 417 Camera switch button 418 Video inversion button 419 3D switch button 420 Sub

action Selection button

421A, 421B Display area 430 Sidebar 431 Selection list 432 Check box 433 Ball filtering button 434 Display area 441 Video switching button 442 Trajectory display switching button 443 Ball display switching button 444 Player display switching button 445 Superimposition separation switching button 446 Arrange button 447

Sub-action selection buttons

451A, 451B Display area 1000 Computer 1001 Input device 1002 Output device 1003 CPU
1004 GPU
1005 ROM
1006 RAM
1007 Storage device 1008 Reading device 1009 Transmission / reception device 1010 Bus 1011 Camera device

Claims

A trajectory analysis unit that generates trajectory information indicating the movement trajectory of the moving body based on the first moving image and the second moving image of the action of the person with respect to the moving body used in the ball game from different positions.
A record management unit that manages a plurality of record information that is information indicating a correspondence relationship between the first moving image and the second moving image and the locus information.
A UI (User Interface) for selecting the recorded information based on the locus information from the plurality of recorded information is displayed, and a first moving image or a second moving image or a second video having a corresponding relationship in the recorded information selected through the UI is displayed. Equipped with a display control unit that displays the video of
Ball game video analysis device.
When the first recorded information and the second recorded information are selected from the plurality of recorded information, the display control unit has a first moving image or a second moving image having a corresponding relationship in the first recorded information. And the first moving image or the second moving image having a corresponding relationship in the second recorded information are displayed side by side.
The ball game video analysis device according to claim 1.
A posture analysis unit that generates posture model information indicating the posture of the person at the time of the action based on the first moving image and the second moving image is further provided.
The recorded information is information indicating a correspondence relationship between the first moving image and the second moving image, the locus information, and the posture model information.
When the first recorded information and the second recorded information are selected from the plurality of recorded information, the display control unit is generated from the posture model information having a corresponding relationship in the first recorded information. The posture model and the posture model generated from the posture model information having a corresponding relationship in the second recorded information are displayed side by side or overlapped.
The ball game video analysis device according to claim 1.
The display control unit has the movement locus of the moving body generated from the locus information having a correspondence relationship in the first recording information, and the movement generated from the locus information having a correspondence relationship in the second recording information. The movement locus of the body is displayed together with the posture model.
The ball game video analysis device according to claim 3.
A sub-action analysis unit that generates sub-action information indicating the occurrence timing of each of the plurality of sub-actions constituting the action based on the locus information and the posture model information is further provided.
The recorded information is information indicating the correspondence between the first moving image and the second moving image, the locus information, the posture model information, and the sub action information.
The display control unit
Display the sub-action selection UI that allows you to select one of the plurality of sub-actions.
The posture model at the occurrence timing of one sub-action selected through the sub-action selection UI is displayed.
The ball game video analysis device according to claim 3.
A ball game video analysis system including a first mobile terminal with a camera, a second mobile terminal with a camera, a ball game video analysis device, and a display terminal.
The first mobile terminal captures a person's action on a moving body used in a ball game from a first position and generates a first moving image.
The second mobile terminal captures the action of the person from a second position to generate a second moving image.
The ball game video analysis device is
Based on the first moving image and the second moving image, locus information indicating the locus of the moving body is generated.
It manages a plurality of recorded information which is information indicating a correspondence relationship between the first moving image and the second moving image and the locus information.
A UI for selecting the recorded information based on the locus information from the plurality of recorded information is displayed on the display terminal.
A first moving image or a second moving image having a corresponding relationship in the recorded information selected through the UI is displayed on the display terminal.
Ball game video analysis system.
Depending on the information processing device
Based on the first moving image and the second moving image in which the actions of the person with respect to the moving body used in the ball game are taken from different positions, the locus information indicating the moving locus of the moving body is generated.
It manages a plurality of recorded information which is information indicating a correspondence relationship between the first moving image and the second moving image and the locus information.
A UI for selecting recorded information based on the locus information from the plurality of recorded information is displayed, and a first moving image or a second moving image having a corresponding relationship in the recorded information selected through the UI is displayed. ,
Ball game video analysis method.
Based on the first moving image and the second moving image in which the actions of the person with respect to the moving body used in the ball game are taken from different positions, the locus information indicating the moving locus of the moving body is generated.
It manages a plurality of recorded information which is information indicating a correspondence relationship between the first moving image and the second moving image and the locus information.
A UI for selecting recorded information based on the locus information from the plurality of recorded information is displayed, and a first moving image or a second moving image having a corresponding relationship in the recorded information selected through the UI is displayed. ,
A computer program that lets a computer perform processing.