WO2023219049A1

WO2023219049A1 - Motion analysis system and motion analysis method

Info

Publication number: WO2023219049A1
Application number: PCT/JP2023/017219
Authority: WO
Inventors: 貴紀生田; 健汰西村; 義之川口
Original assignee: 京セラ株式会社
Priority date: 2022-05-13
Filing date: 2023-05-08
Publication date: 2023-11-16

Abstract

In order to improve the skill of a user, the present invention effectively presents the movement of the body of the user. This motion analysis system comprises: an analysis request reception unit which acquires a first video which captures motions of a user and detection information pertaining to motions detected by a movement sensor; a target event detection unit which detects, on the basis of the detection information, a target event that has occurred during the motions; and a first extraction unit which extracts, from the first video, at least one first static image corresponding to the detected target event.

Description

Motion analysis system, motion analysis method

The present disclosure relates to a motion analysis system and a motion analysis method that analyze body movements.

There is a known technique for improving a player's skill by attaching a sensor to a racket and analyzing the movement of the racket (for example, Patent Document 1 and Patent Document 2).

Patent Document 1 discloses a technique in which a table tennis racket is provided with an acceleration sensor and a contact sensor. Patent Document 2 discloses a technique for displaying an image in which a subject is photographed and an index indicating the subject's movement that is generated based on a measurement result obtained by measuring the subject's movement using a sensor.

Japanese Patent Application Publication No. 2009-183455 Japanese Patent Application Publication No. 2020-205608

A motion analysis system according to an aspect of the present disclosure includes: a first acquisition unit that acquires a first moving image of a user's motion; and a second acquisition unit that acquires detection information regarding the motion detected by a motion sensor. , a target event detection unit that detects a target event occurring during the operation based on the detection information, and extracting at least one first still image corresponding to the detected target event from the first video. A first extractor.

A motion analysis method according to an aspect of the present disclosure includes an acquisition step in which a computer acquires a first moving image of a user's motion and detection information regarding the motion detected by a motion sensor; a target event detection step of detecting a target event occurring during the operation based on detection information; and a computer extracting at least one first still image corresponding to the detected target event from the first video. a first extraction step.

FIG. 1 is a diagram illustrating an example of a schematic configuration of a motion analysis system according to an aspect of the present disclosure. FIG. 2 is a perspective view showing a configuration example of a racket to which a motion sensor device is attached. FIG. 1 is a block diagram showing an example of a schematic configuration of a motion analysis system. FIG. 2 is a block diagram showing an example of a schematic configuration of a user terminal. FIG. 2 is a block diagram showing an example of a schematic configuration of a server device. FIG. 2 is a sequence diagram showing an example of the flow of main processing in the motion analysis system. It is a figure showing an example of a display screen in a user terminal. It is a figure showing an example of a display screen in a user terminal. It is a figure showing an example of a display screen in a user terminal. It is a figure showing an example of a display screen in a user terminal. It is a figure showing an example of a display screen in a user terminal. It is a figure showing an example of a display screen in a user terminal. It is a figure showing an example of a display screen in a user terminal. It is a figure showing an example of a display screen in a user terminal.

Hereinafter, one embodiment of the present disclosure will be described in detail with reference to schematically shown FIG. 1 and the like.

(Scope of application of motion analysis system 100 according to the present disclosure)
First, the scope of application of the motion analysis system 100 according to the present disclosure will be described. The motion analysis system 100 is a system that can analyze a predetermined target event included in a user's motion. Here, the "user's motion" may include any general motion that the user performs using his or her own body. Further, the "target event" is an event that can occur due to an action by a user, and may be any event that can be an analysis target.

For example, if the user's action is to move a predetermined moving object by striking it with sports equipment, the target event is that the equipment comes into contact with the predetermined moving object. , or the tool may be in a predetermined position.

"Equipment" is intended to be a tool that a user moves when playing sports, etc., and "moving object" is intended to be a moving object used in sports, etc.

Examples of equipment include rackets for tennis, badminton, table tennis, squash, lacrosse, etc., baseball and cricket bats, golf clubs, hockey and ice hockey sticks, gymnastics equipment, fishing rods, clothing worn by the user, and the like. . On the other hand, examples of moving objects include balls used in various ball games, badminton shuttles, and ice hockey pucks.

Furthermore, a cue in billiards is also a tool for hitting the ball, and can be included in the equipment. For example, the action of kicking a ball in rugby and soccer can be considered as the action of hitting the ball with a shoe. Therefore, shoes in rugby and soccer may also be included in the equipment.

Additionally, the equipment may include clothes, gloves, socks, etc. worn by the user. The equipment may also include tights-like clothing worn over all or part of the user's body. In this case, by analyzing the movement of the tool, it becomes possible to analyze the movement of the user's body.

The user's actions are not limited to sports. For example, the user's action may be work, dancing, etc. that requires a predetermined posture (or maintaining the predetermined posture). In this case, the target event may be that the user assumes a predetermined posture. Alternatively, the user's action may be an action of cooking predetermined ingredients using cooking utensils. The cooking utensils may be, for example, knives and frying pans. In this case, the target event may be that the tool comes into contact with a predetermined food material, or that the tool assumes a predetermined posture.

In the following embodiment, a motion analysis system 100 that analyzes a user's motion using a racket 6 will be described as an example in order to improve the user's table tennis skills. In this case, the equipment is a table tennis racket 6, and the moving object is, for example, a celluloid table tennis ball 7 (also called a ping pong ball). In the embodiments below, the "target event" is intended to mean, for example, that the user hits the ball 7 using the racket 6 (that is, the racket 6 comes into contact with the ball 7).

[Embodiment 1]
(Configuration of motion analysis system 100)
The configuration of the motion analysis system 100 will be explained with reference to FIG. 1 and the like as appropriate. FIG. 1 is a diagram showing an example of a schematic configuration of a motion analysis system 100.

The motion analysis system 100 includes a user terminal 2 capable of acquiring detection information regarding the user's motion from the motion sensor device 5 and capturing a first moving image of the motion, and a user terminal 2 that can detect a target event and analyze the user's motion. A server 1 (motion analysis device) that analyzes motion. The server 1 and the user terminal 2 are communicably connected to each other via a network 4, as shown in FIG. The network 4 may be the Internet or a local network provided within the facility.

The user terminal 2 is capable of transmitting an analysis instruction regarding the user's actions to the server device 1. The transmitted analysis instruction includes a first moving image of the user's motion and detected information regarding the motion. The user terminal 2 may include a touch panel 29 having the functions of an input section 23 and a display section 24, and an imaging section 26 capable of capturing a first moving image. FIG. 1 shows how a user terminal 2 is used to capture an image of a user's motion of gripping a racket 6 to which a motion sensor device 5 is attached.

The server device 1 analyzes the detection information and the first video acquired from the user terminal 2 in accordance with the analysis instruction received from the user terminal 2. The server device 1 transmits the analysis results to the user terminal 2. In this way, the motion analysis system 100 can provide the user with an analysis result based on the first moving image and the detection information detected by the sensor device 5. For example, when the detection information indicates the movement of the tool and a change in posture, the analysis result includes suggestions regarding not only how the user moves his or her body but also how to handle the tool used. Therefore, the motion analysis system 100 can help the user recognize corrections in how to move the user's body and how to handle his or her own equipment, and perform effective exercises. Therefore, the user can perform effective exercises to improve his or her skills.

Although FIG. 1 shows a behavior analysis system 100 that includes one server 1 and one user terminal 2, the configuration is not limited to this. That is, the motion analysis system 100 may include multiple servers 1 and multiple user terminals 2.

<Configuration of the racket 6 to which the motion sensor device 5 is attached>
A user using the motion analysis system 100 uses a racket 6 to which a motion sensor device 5 (motion sensor) is attached. The motion sensor device 5 measures the motion of the racket 6 held by the user in real time.

FIG. 2 is a perspective view showing an example of the configuration of the racket 6 to which the motion sensor device 5 is attached. Although a shake-hand type racket 6 is illustrated here as an example, the present invention is not limited to this. For example, the racket 6 may be a racket of other shapes, such as a pen holder type.

The racket 6 has a hitting part 62 for hitting the ball 7 and a grip 61 for the user to hold. The hitting portion 62 has a hitting surface 9a for hitting the ball 7. The racket 6 shown in FIG. 2 is of a shake-hand type and has a pair of front and back hitting surfaces 9a. The hitting surface 9a of the racket 6 is often filled with rubber having various properties (for example, a resilient sheet made of a sponge sheet and a rubber sheet).

The grip 61 is a part that the user holds to manipulate the racket 6. The striking portion 62 and the grip 61 may be formed integrally with each other.

The motion sensor device 5 is fixed to the racket 6 and functions as an inertial sensor that detects at least angular velocity. The attachment position, shape, size, etc. of the motion sensor device 5 to the racket 6 may be set as appropriate. In the illustrated example, the motion sensor device 5 is attached to the end of the grip 61 of the racket 6 on the opposite side from the hitting portion 62, although the movement sensor device 5 is not limited thereto. The motion sensor device 5 may be removably attached to the racket 6, or may be fixed to the racket 6 in a non-removable manner. The motion sensor device 5 can also detect an impact when the ball 7 contacts the racket 6. Furthermore, the motion sensor device 5 can also detect that the racket 6 has assumed a predetermined posture.

Next, the configuration of the motion analysis system 100 will be explained using FIG. 3. FIG. 3 is a block diagram showing an example of a schematic configuration of the motion analysis system 100.

<Schematic configuration of motion sensor device 5>
The motion sensor device 5 includes a storage section 52, an angular velocity sensor 53 that detects the angular velocity of the racket 6, an acceleration sensor 54 that detects the acceleration of the racket 6, a communication section 55 that communicates with the user terminal 2, and a CPU (Central Processing Unit). ) 50. The angular velocity sensor 53 is, for example, a gyro sensor that detects the angular velocity of the racket 6. The CPU 50 processes signals input to or output from the angular velocity sensor 53, the acceleration sensor 54, and the communication unit 55. Further, the motion sensor device 5 may include a power source that supplies power to the various configurations described above. In FIG. 4, illustration of the power supply is omitted.

The storage unit 52 may include a ROM (Read Only Memory), a RAM (Random Access Memory), an external storage device, and the like. The storage unit 52 stores predetermined application programs and the like executed by the CPU 50.

The angular velocity sensor 53 is constituted by a three-axis angular velocity sensor capable of detecting angular velocity in each of three axes: x-axis, y-axis, and z-axis. Examples of such an angular velocity sensor include a combination of an angular velocity sensor that detects angular velocity about the x-axis, an angular velocity sensor that detects angular velocity about the y-axis, and an angular velocity sensor that detects angular velocity about the z-axis.

The acceleration sensor 54 is composed of a three-axis acceleration sensor that can detect acceleration in directions along each of the three axes: x-axis, y-axis, and z-axis. Any known configuration can be used as the configuration of the three-axis acceleration sensor.

The motion sensor device 5 can detect the moving direction and rotational movement of the racket 6 using the angular velocity sensor 53, and can measure the moving distance and speed of the racket 6 using the acceleration sensor 54. In this way, the motion of the racket 6 can be analyzed by the motion sensor device 5.

Here, the three axes, the x-axis, y-axis, and z-axis, can be set arbitrarily. In FIG. 2, the axis perpendicular to the hitting surface 9a of the racket 6 is the x-axis, the axis parallel to the hitting surface 9a and perpendicular to the axis of the grip 61 is the y-axis, and the axis parallel to the hitting surface 9a and possessed by the grip 61 is the y-axis. Although an example is shown in which the parallel axes are the z-axis, the invention is not limited to this. For example, the configuration may be such that the three axes, the x-axis, y-axis, and z-axis, are an axis parallel to the direction of gravity and two axes that are perpendicular to the direction of gravity and perpendicular to each other. In this specification, when "parallel" is used, it is not strictly required to be "parallel". Furthermore, in this specification, when "vertical" is used, it is not strictly required to be "vertical".

The communication unit 17 has a configuration that allows communication with at least the user terminal 2. This communication may be wireless communication or wired communication. In the case of wireless communication, those using radio waves, those using infrared rays, etc. can be applied. Bluetooth (registered trademark), WiFi (registered trademark), and the like can be applied as devices that use radio waves.

<Schematic configuration of user terminal 2>
Next, the configuration of the user terminal 2 will be explained. The user terminal 2 may include a computer. For example, the user terminal 2 may be a smartphone, a tablet, or the like. Any known computer hardware and OS (Operating System) may be used. The user terminal 2 can be obtained by installing a predetermined application program on a general computer.

As shown in FIG. 3, the user terminal 2 may include a CPU 20, a storage section 22, an input section 23 that receives various user operations, a display section 24, a communication section 25, and an imaging section 26. The user terminal 2 may be configured to acquire a first moving image captured by a video camera or the like. Therefore, in the user terminal 2, the imaging unit 26 is not an essential component. Further, the user terminal 2 may be configured to output the analysis result to a display device (not shown) that is separate from the user terminal 2 and is used by the user. Therefore, in the user terminal 2, the display section 24 is not an essential component.

The CPU 20 processes signals input to or output from the input unit 23, display unit 24, communication unit 25, and imaging unit 26. The storage unit 22 may include a ROM (Read Only Memory), a RAM (Random Access Memory), an external storage device, and the like. The storage unit 22 stores various control programs executed by the CPU 20, data used for various processes executed by the CPU 20, and the like.

The user terminal 2 is capable of capturing an image of the user's motion while simultaneously acquiring detection information regarding the user's motion detected by the motion sensor device 5. For example, the user terminal 2 can start acquiring detection information regarding the user's motion and capturing an image of the user's motion at the same time (or almost simultaneously).

<Schematic configuration of server device 1>
Next, the configuration of the server device 1 will be explained. The server device 1 may include a computer. Any known computer hardware and OS (Operating System) may be used. The server device 1 can be obtained by installing a predetermined application program on a general computer.

As shown in FIG. 3, the server device 1 includes a CPU 10, a storage section 12, and a communication section 15.

The CPU 10 processes signals input to or output from the communication unit 15. The storage unit 12 may include a ROM (Read Only Memory), a RAM (Random Access Memory), an external storage device, and the like. The storage unit 12 stores analysis programs executed by the CPU 10 and the like.

The server device 1 detects the target event and executes the motion analysis in response to acquiring the detection information regarding the user's motion, the first video, and the analysis instruction from the user terminal 2. The server device 1 provides the analysis results to the user terminal 2 that is the source of the analysis instruction. The server device 1 may generate a second video showing the change in posture of the racket 6 used by the user during the operation based on the acquired detection information. The second video may be an animation video showing changes in the posture of the racket 6.

The analysis results provided from the server device 1 to the user terminal 2 may include the following.
・Second video ・First still image and second still image described later ・First related image group and second related image group described later ・Estimated trajectory and estimated rotation speed of ball 7 launched by racket 6 ・User's body A skeletal model showing the movement of.

(Configuration of user terminal 2)
The configuration of the user terminal 2 will be explained in more detail using FIG. 4. FIG. 4 is a block diagram showing an example of a schematic configuration of the user terminal 2. As shown in FIG. The control unit 21 shown in FIG. 4 corresponds to the CPU 20 in FIG. 3.

The user terminal 2 may include a control section 21 , a storage section 22 , an imaging section 26 , an input section 23 , a display section 24 , and a communication section 25 . The control unit 21 controls the user terminal 2 to execute processing of each function.

The storage unit 22 is a storage device that stores various control programs read by the control unit 21 and analysis target data used in various processes executed by the control unit 21.

The control unit 21 includes an imaging control unit 211 and a measurement value acquisition unit 212. FIG. 4 shows a configuration further including a display control section 214 and a data management section 213.

The imaging control unit 211 controls the imaging unit 26 to acquire the first moving image. The first video is an image of a user playing table tennis using a racket 6 to which a motion sensor device 5 is attached. The first video may show the trajectory of the ball 7 actually hit by the racket 6 held by the user. The imaging control unit 211 may store the first moving image captured by the imaging unit 26 in the analysis target data 222.

The control unit 21 may include a moving image acquisition unit (not shown) as an alternative to the imaging control unit 211. The user terminal 2 equipped with this moving image acquisition section can acquire, via the communication section 25, a moving image captured by an external device (not shown) such as a digital camera and a digital video. However, in this case, it is necessary to synchronize the detection information (for example, measured value) received from the motion sensor device 5 and the moving image captured by the external device. Therefore, for example, the acquired moving image may be stored in the analysis target data 222 of the storage unit 22 together with time information at which imaging was started in order to synchronize with the measured values.

The measurement value acquisition unit 212 acquires the detection information detected by the motion sensor device 5 via the communication unit 25. The detection information is information in any format output from the motion sensor device 5, and may be, for example, a graph representing changes in the position of the racket 6 over time, or a graph representing changes in the position and posture of the racket 6 over time. It may be a measured value indicating a change. The detection information may include time information indicating the measured time. The measurement value acquisition section 212 may be configured integrally with the imaging control section 211. The measured value acquisition unit 212 may store the acquired detection information in the analysis target data 222.

For example, the control unit 21 simultaneously starts capturing a moving image by the imaging unit and acquiring detection information from the motion sensor device 5. Below, the measurement value acquisition unit 212 measures the detection information from the motion sensor device 5. An example of how to obtain a value will be explained.

The data management unit 213 associates the captured first video and the acquired measurement values and stores them in the analysis target data 222. Further, the data management unit 213 stores the first video and the measured values in the analysis target data 222 in association with the date on which the first video was captured, identification information corresponding to the person who performed the captured action, and the like. You can. If this configuration is adopted, the user terminal 2 can receive the selection of the date on which the analysis request was sent, and display the analysis result corresponding to the analysis request on the display unit 24.

The data management unit 213 may be able to store analysis results related to multiple users in the storage unit 22. Here, the plurality of users may be, for example, a plurality of players who receive guidance from an instructor in order to improve their table tennis skills. By employing the motion analysis system 100, it is possible to provide an instructor who instructs a plurality of players with the analysis results obtained by analyzing the motions of each player. This allows the coach to understand the characteristics of each player's movements and provide guidance suitable for each player.

The data management unit 213 generates an analysis request that includes the first video and detection information specified by the user. The data management unit 213 may store the analysis request in the analysis request log 221 of the storage unit 22 in association with the date on which the analysis request was generated.

The data management unit 213 may store the analysis results received from the server device 1 in the analysis results 223. In this case, the data management unit 213 may associate the transmitted analysis request with the analysis result corresponding to the analysis request and store it in the analysis result 223. For example, the data management unit 213 may manage the first still image acquired from the server device 1 by associating it with the date on which the first video transmitted to the server device 1 was captured.

The display control unit 214 causes the display unit 24 to display the captured first video, a screen for allowing the user to input an analysis instruction to be sent to the server device 1, the received analysis results, and the like. The display control unit 214 may display the analysis result and the second moving image at the same time. The display screen that the display control unit 214 causes the display unit 24 to display will be described later using a specific example.

The communication unit 25 communicates with the motion sensor device 5, the user terminal 2, and the server device 1. The user terminal 2 receives detection information from the motion sensor device 5, transmits an analysis request to the server device 1, receives analysis results from the server device 1, etc. via the communication unit 25.

(Configuration of server device 1)
The configuration of the server device 1 will be explained in more detail using FIG. 5. FIG. 5 is a block diagram showing an example of a schematic configuration of the server device 1. As shown in FIG. The control unit 11 shown in FIG. 5 corresponds to the CPU 10 in FIG. 3. That is, each part of the control unit 11 is a functional block realized by the CPU 10 in FIG. 3 executing the analysis program 121.

The server device 1 includes a control section 11, a storage section 12, and a communication section 15. The control unit 11 controls the server device 1 to execute processing of each function.

The storage unit 12 is a storage device that stores various computer programs read by the control unit 11 and data used in various processes, such as the analysis program 121 executed by the control unit 11. As shown in FIG. 5, the storage unit 12 may store analysis results 122, user information 123, and the like. The analysis result 122 stores the analysis result sent by the server device 1 in response to the acquired analysis request in association with the analysis request. The user information 123 may store information regarding users who are registered as users of the motion analysis system 100. The information regarding the user may include, for example, address and information indicating contact information such as an email address. The user information 123 may store information about the user terminal 2 that is the sender of the analysis request, a history of sending analysis requests from each user, and the like.

The control unit 11 includes an analysis request reception unit 111 (first acquisition unit, second acquisition unit), a target event detection unit 112, and a first extraction unit 113.

The analysis request reception unit 111 acquires an analysis request from the user terminal 2. The analysis request includes a first moving image of the user's motion and detection information regarding the motion detected by the motion sensor. In addition, the analysis request includes information specific to the user terminal 2 that sent the analysis request, identification information of the user performing the action shown in the video to be analyzed, and the name of the action. May be included. Here, the information unique to the user terminal 2 may include information indicating the destination of the analysis result. The user's identification information may be, for example, the user's name or identification information given to the user within a team to which the user belongs as a member. The identification information may be a membership number assigned to each member of a team to which the user belongs as a member. The name of the action may be the name of a target event to be detected in the analysis process (for example, "hit the ball"). Alternatively, it may be the name of an action that includes the target event. The name of the operation may be, for example, "serve," "drive," "push," and "serve return."

For example, the server device 1 may have a configuration including a first acquisition unit that acquires the first video and a second acquisition unit that acquires the detection information, and is not limited to the configuration shown in FIG. 5. The analysis request reception unit 111 shown in FIG. 5 has a function as a first acquisition unit and a function as a second acquisition unit. Hereinafter, the server device 1 including the analysis request reception unit 111 will be described as an example.

The target event detection unit 112 detects a target event during the user's action based on the detection information. The target event detection unit 112 may detect the target event based on at least one of the speed of posture change and the movement speed of the racket 6 during the motion, which are calculated based on the detection information. For example, when the racket 6 and the ball 7 come into contact, vibrations generated in the racket 6 can be detected. This vibration affects at least one of the speed of posture change and the movement speed of the racket 6 during operation. Therefore, the target event detection unit 112 can detect the target event based on the fact that at least one of the speed of attitude change and the movement speed of the racket 6 is influenced by the vibration.

The first extraction unit 113 extracts at least one first still image corresponding to the detected target event from the first moving image. Here, the first still image is a still image corresponding to a time point at which the target event detected by the target event detection unit 112 occurs or a time point in the vicinity thereof.

The first extraction unit 113 may be able to extract a first related image group, which is a plurality of still images before and after the first still image, from the first moving image. When a plurality of target events are detected from the detection information, the first extraction unit 113 may extract at least one first still image for each detected target event. The first related image group will be explained later using a specific example of a display screen.

The first extraction unit 113 analyzes the movement of the racket 6 based on the acquired measurement values. For example, the first extraction unit 113 may synchronize the first video and the measured value and detect a target event included in the first video. The first extraction unit 113 stores the analysis result in the storage unit 22 (analysis result 223 in FIG. 4) in association with the measured values and the first video used for the analysis.

In response to the analysis request, the server device 1 may analyze the movement of the racket 6 and create the second video using any known method. In this case, the server device 1 may further include a second moving image generation section 114 and a second extraction section 115, as shown in FIG.

The second video generation unit 114 generates a second video showing a change in the posture of the racket 6 during the user's movement based on the detection information. The second extraction unit 115 may be able to extract a second related image group, which is a plurality of still images before and after the second still image, from the second moving image. The first related image group will be explained later using a specific example of a display screen.

The second extraction unit 115 extracts at least one second still image corresponding to the target event detected by the target event detection unit from the second moving image. Here, the second still image is a still image corresponding to a time point at which the target event detected by the first extraction unit 113 based on the detection information occurs or a time point in the vicinity thereof. When a plurality of target events are detected from the detection information, the second extraction unit 115 may extract a plurality of second still images corresponding to each of the detected target events.

The second video generation unit 114 may be able to generate a plurality of second videos corresponding to each of the detected target events. In this case, the second extraction unit 115 may be able to extract at least one second still image from each of the generated second moving images.

The communication unit 15 communicates with the user terminal 2. The server device 1 receives analysis requests to the server device 1 and transmits analysis results from the server device 1 via the communication unit 15 .

(Main processing flow in motion analysis system 100)
Next, the main processing flow in the motion analysis system 100 will be explained using FIG. 6. FIG. 6 is a sequence diagram showing an example of the main processing flow in the motion analysis system 100.

First, the motion sensor device 5 is activated by the user (step S10). Further, the imaging unit of the user terminal 2 is activated by the user (step S11).

Next, in response to the user's operation, the user terminal 2 outputs a detection start instruction to the motion sensor device 5 (step S12), starts imaging (step S13), and starts capturing images after a predetermined period of time (for example, 30 seconds). 1 Capture a video. The motion sensor device 5 that has received the detection start instruction starts transmitting a measured value indicating the movement of the racket 6 to the user terminal 2. The user terminal 2 acquires detection information indicating the user's motion while capturing the first video from the motion sensor device 5 (step S15). The motion sensor device 5 that has received the detection start instruction may collect measured values indicating the motion of the racket 6 measured over a predetermined period of time (for example, 30 seconds) and transmit them to the user terminal 2.

The user terminal 2 receives a predetermined operation by the user and transmits an analysis request to the server device 1 (step S16). The analysis instruction includes a first moving image of the user's motion and detection information regarding the motion.

The server device 1 acquires an analysis instruction from the user terminal 2 (step S17: acquisition step), and analyzes the detection information and the first video (step S18: target event detection step, first extraction step). In step S18, the server device 1 detects a target event that occurs during operation based on the detection information, and extracts at least one first still image corresponding to the detected target event from the first video. The server device 1 transmits the analysis result to the user terminal 2 (step S19).

The user terminal 2 receives the analysis result received from the server device 1 (step S20), and displays the analysis result on the display unit 24 (step S21). The user terminal 2 that has received the analysis result may display the analysis result on the display unit 24 in any manner. For example, the user terminal 2 may read out the first moving image to be analyzed from the analysis target data 222 and display it on the display unit 24 together with the analysis result.

(Display screen example)
Next, examples of display screens displayed on the display unit 24 by the display control unit 214 will be described using FIGS. 7 to 14. 7 to 14 are diagrams showing examples of display screens on the user terminal 2. 7 to 11 show examples of display screens from the user terminal 2 until the analysis request is sent to the server device 1. 12 to 14 show examples of display screens from when the user terminal 2 receives the analysis results from the server device 1 to when the received analysis results are displayed on the display unit 24.

Here, the explanation will be given using an example of an instructor coaching a plurality of players and a user terminal 2 shared by the plurality of players. In this case, each player and coach may be a user. In this way, by using the motion analysis system 100, the coach can manage the practice diaries, analysis requests, and analysis results of each of a plurality of players in association with each other.

<Send analysis request>
FIG. 7 is an example of a display screen (login screen) that is displayed when the user starts using the motion analysis system 100. Display R1 of the login screen shown in FIG. 7 displays a column for allowing the user to input user information. The user information may be, for example, an email address and a password. The user terminal 2 may transmit the input user information to the server device 1. In this case, the server device 1 may approve the user's use of the motion analysis system 100 by comparing the received user information with the user information 123 registered in advance.

Next, FIGS. 8 and 9 are examples of display screens that display the practice diaries of each of a plurality of players. Display R2 shown in FIG. 8 displays a list of players being coached by the instructor. This screen is a display screen that allows the coach or user to select a certain player from among the players displayed in the player list. Hereinafter, FIGS. 9 to 14 show the case where "Player A" is selected in FIG. 8.

FIG. 9 is an example of a display screen that displays player A's practice diary. Display R3 displays the selected date and calendar. FIG. 9 shows an example of player A's practice diary displayed on January 9, 2022. In display R3, the selected date "January 9th (Sunday)" is displayed, and a circular mark is attached to January 9th on the calendar. The date column and calendar display displayed in display R3 have a function that allows the instructor or user to select the date of the practice diary to be displayed.

A square mark is attached to January 8th on the calendar of display R3. This square mark indicates that January 8th was the day on which the request for analysis of Athlete A's movements was sent and the analysis results were received. A square mark will not be placed on days when no analysis request has been sent. When the instructor or user selects "January 8th" in display R3, the user terminal 2 displays information indicating the analysis request sent on January 8th and information indicating the received analysis results on the display. 24 (for example, see FIG. 14). Here, the analysis request includes the first moving image and detection information, and the analysis result includes the first still image extracted by the server device 1. Therefore, in display R3, days to which the first still image acquired from the server device 1 is associated and days to which it is not are displayed are distinguished. The mark indicating the date on which the analysis result was received is not limited to a square mark, and may be changed as appropriate.

When “Create a practice diary” in display R4 in FIG. 24.

FIG. 10 is an example of a display screen for inputting player A's practice diary for January 9th. The display screen for entering the practice diary includes ``Practice goals'', ``Practice content'', ``Important points and precautions (what the coach taught me)'', ``What I noticed (image, future Headings such as ``Goal for 2019'' and ``Free Entry Field'' are displayed. Athlete A or the coach can input information into the practice diary, update the practice diary, etc. on this display screen.

When "Register data to be analyzed" in display R5 in FIG. 10 is selected, the display control unit 214 causes the display unit 24 to display a display screen for accepting an operation for transmitting an analysis request.

FIG. 11 is an example of a display screen for accepting an operation for transmitting an analysis request. Display R6 in FIG. 11 includes columns for allowing the user or instructor to input or prepare items and data necessary for transmitting the analysis request to be transmitted.

"Title (required)" in display R6 is a field that accepts input of the name of the analysis request. FIG. 11 shows how "Serve 1" is input. "Sensor data CSV (required)" in display R6 is a column for allowing the user or instructor to specify detection information to be analyzed by the server device 1 (uninputted columns are shown). "Swing video (required)" in display R6 is a column for allowing the user or instructor to specify the first video to be transmitted to the server device 1 (an uninputted column is shown). The "latest geomagnetic data registration date" in display R6 is a field for allowing the user or instructor to specify geomagnetic data that can be used when analyzing detected information in the server device 1 (unfilled fields are indicated). ). "Latest geomagnetic data registration date" and "Geomagnetic data CSV (required if there is no registered geomagnetic data)" in display R6 indicate geomagnetic data that can be used by the server device 1 when analyzing detection information. Or this is a field for the instructor to specify (unfilled fields are shown).

If "Register" in display R7 is selected, the data management unit 213 reads out the detection information and the first video specified in these columns from the analysis target data 222 and creates an analysis request. The created analysis request is then transmitted to the server device via the communication unit 25. On the other hand, when "Back" in display R8 is selected, the display control unit 214 causes the display unit 24 to display a display screen that displays a practice diary as shown in FIG.

<Display analysis results>
Next, the display screen from when the user terminal 2 receives the analysis result from the server device 1 to when the received analysis result is displayed on the display unit 24 will be described using FIGS. 12 to 14.

FIG. 12 is an example of a display screen when the practice diary for January 9 is displayed after receiving the analysis results from the server device 1. Unlike the display screen where the practice diary for January 9th is being created (see FIG. 10), "Serve 1" of display R9 is displayed. When “Serve 1” of display R9 is selected, the display control unit 214 reads the analysis result from the analysis result 223 and causes the display unit 24 to display the display screens shown in FIGS. 13 and 14.

FIG. 13 is a diagram showing a part of the display screen that displays the analysis results. In FIG. 13, a first moving image (or a thumbnail of the first moving image) that is an analysis target is displayed in display R10. The display R11 displays the detection information ("sensor data" in the diagram), geomagnetic data, and the original of the first video ("captured video" in the diagram) sent together with the first video at the time of the analysis request on the display unit 24. A button for display may be displayed. For example, when the user selects "photographed video", the display control unit 214 starts a predetermined video playback application and plays the first video.

FIG. 14 is a diagram showing a part of the display screen that displays the analysis results for each target event detected by the server device 1. Here, an example is shown in which two or more target events "hit" are detected from the first video and the detection information. Therefore, FIG. 14 is a display screen that includes at least a display regarding "Hit No. 1" and a display regarding "Hit No. 2".

Displays R12 to R15 in FIG. 14 display analysis results regarding "Hit No. 1." A second moving image is displayed on the display R12 ("racquet animation image" in the figure). Display R13 displays a graph showing the movement of the racket 6 analyzed based on the detected information. FIG. 14 shows a graph showing "speed by rotation direction" and a graph showing "swing speed." A second still image and a second related image group are displayed in display R14 (“racquet posture continuous images” in the figure). In display R14, the image corresponding to the "hit time" is the second still image, and the image group corresponding to "0.3 seconds ago" to "0.1 seconds ago" and "0.1 seconds later" is the second still image. This is a group of 2 related images.

A first still image and a first related image group are displayed in display R15 (“skeleton extraction continuous images (0.1 second interval)” in the figure). In display R15, the images corresponding to "No. 04" to "No. 05" are the first still images, and the other image groups are the first related image group. In this way, the display control section 214 may display the first still image and the second still image on the display section 24 at the same time. According to this configuration, the motion analysis system 100 allows the user to check the first still image and the second still image simultaneously, thereby making it easier for the user to understand the characteristics of the user's own target event and points for improvement.

The server device 1 may be able to generate skeletal data of the skeletal model based on the first video. In this case, the display control unit 214 may display the generated skeletal model on the first still image on the display unit 24, as shown in display R15. The display control unit 214 may display the generated skeletal model on the display unit 24 by superimposing it on the first related image group. The user's body movements can be clearly shown by using a skeletal model. Therefore, as shown in display R15, by displaying the first still image (and the first related image group) on which the skeletal model is superimposed, the motion analysis system 100 allows the user to understand the characteristics of his or her own motion and points for improvement. can be clearly recognized. Thereby, the user can practice to improve his/her skill while being aware of the characteristics of his/her own movements and areas for improvement.

As shown in FIG. 14, the display control unit 214 may display at least one first still image and a first related image group in chronological order in the first moving image. Thereby, the motion analysis system 100 can make the user understand the analysis result regarding the target event in association with the flow of a series of motions performed by the user.

The display control unit 214 may display at least one first still image and first related image group in chronological order for each detected target event. For example, when "Hit No. 2" is detected as a target event in addition to "Hit No. 1", the display control unit 214 displays "Hit No. 2" in the same display mode as display R14 and R15. The first still image (and the first related image group) and the second still image (and the second related image group) may be displayed on the display unit 24.

(Another Embodiment 1)
Although FIGS. 13 and 14 show an example in which the first moving image and the second moving image are displayed on the display section 24, the present invention is not limited thereto. For example, the display control unit 214 may display only the second still image and the second related image group without displaying the second moving image. That is, the display control unit 214 may display at least one of the first video and the second video.

(Another Embodiment 2)
Although the display R15 in FIG. 14 displays an image in which a skeletal model is superimposed on the first still image and the first still image group, the display R15 is not limited to such a display mode. For example, the display control unit 214 may display the moving image of the skeletal model generated by the server device 1 based on the first moving image on the display R15.

[Embodiment 2]
The motion analysis system 100 according to Embodiment 1 of the present disclosure has a configuration in which analysis results are transmitted from the server device 1 to the user terminal 2 (see FIG. 6). However, the motion analysis system 100 is not limited to this configuration. For example, the server device 1 may be configured to create a dedicated web page for each acquired analysis request, and allow the analysis results to be viewed on this web page. In this case, the server device 1 may transmit information for accessing the web page to the user terminal 2 that is the source of the analysis request. According to this configuration, the transmission load between the server device 1 and the user terminal 2 can be reduced.

[Embodiment 3]
The motion analysis system 100 according to Embodiment 1 of the present disclosure has a configuration in which the server device 1 acquires an analysis request from the user terminal 2 and executes motion analysis (see FIGS. 4 and 5). However, the motion analysis system 100 is not limited to this configuration. For example, the user terminal 2 may include all or part of the functions of the server device 1 (that is, the control unit 11 and the storage unit 12). For example, if the user terminal 2 has all the functions of the server device 1, the user terminal 2 can acquire the first video and detection information, detect the target event, extract the first still image, is possible. In this case, the user terminal 2 functions as a motion analysis device.

The invention according to the present disclosure has been described above based on the drawings and examples. However, the invention according to the present disclosure is not limited to each embodiment described above. That is, the invention according to the present disclosure can be modified in various ways within the scope shown in the present disclosure, and the invention according to the present disclosure also applies to embodiments obtained by appropriately combining technical means disclosed in different embodiments. Included in technical scope. In other words, it should be noted that those skilled in the art can easily make various changes or modifications based on the present disclosure. It should also be noted that these variations or modifications are included within the scope of this disclosure.

〔summary〕
The motion analysis system 100 according to aspect 1 of the present disclosure includes a first acquisition unit (analysis request reception unit 111) that acquires a first video capturing a user's motion, and a first acquisition unit (analysis request reception unit 111) that acquires detection information regarding the motion detected by a motion sensor. A second acquisition unit (analysis request reception unit 111) that acquires information, a target event detection unit 112 that detects a target event that occurs during operation based on the detection information, and at least one second acquisition unit that corresponds to the detected target event. The first extraction unit 113 extracts one still image from the first moving image.

In the motion analysis system 100 according to Aspect 2 of the present disclosure, in Aspect 1, the first extraction unit 113 extracts a first related image group, which is a plurality of still images before and after the first still image, from the first moving image. It may be possible.

In the motion analysis system 100 according to aspect 3 of the present disclosure, in the

aspect

1 or 2, when a plurality of target events are detected, the first extraction unit 113 extracts at least one first still image from the detected target event. It may be extracted separately.

The motion analysis system 100 according to Aspect 4 of the present disclosure, in Aspect 2, may include a display control unit 214 that displays at least one first still image and a first related image group in chronological order in the first moving image. good.

In the motion analysis system 100 according to aspect 5 of the present disclosure, in aspect 4, the display control unit 214 displays at least one first still image and the first related image group in chronological order for each detected target event. You can.

In the motion analysis system 100 according to aspect 6 of the present disclosure, in any of the aspects 1 to 5, the motion is a motion in which the user uses a tool (racquet 6), and based on the detection information, the motion analysis system a second video generation unit 114 that generates a second video showing a change in attitude of the racket 6); and a second extraction unit that extracts at least one second still image corresponding to the detected target event from the second video. 115.

In the motion analysis system 100 according to aspect 7 of the present disclosure, in the aspect 6, when a plurality of target events are detected, the second extraction unit 115 extracts a plurality of second still images corresponding to each of the detected target events. may be extractable.

In the motion analysis system 100 according to aspect 8 of the present disclosure, in the

aspect

6 or 7, the second video generation unit 114 can generate a plurality of second videos corresponding to each of the detected target events, and The second extraction unit 115 may be able to extract at least one second still image from each of the generated second moving images.

In the motion analysis system 100 according to aspect 9 of the present disclosure, in any of the aspects 6 to 8, the second extraction unit 115 extracts a second related image group that is a plurality of still images before and after the second still image. It may be possible to extract it from the second video.

The motion analysis system 100 according to Aspect 10 of the present disclosure may include a display control unit 214 that simultaneously displays the first still image and the second still image in any of Aspects 6 to 9 above.

The motion analysis system 100 according to aspect 11 of the present disclosure may include a display control unit 214 that displays at least one of the first video and the second video in any of the above aspects 6 to 10.

In the motion analysis system 100 according to aspect 12 of the present disclosure, in any of the aspects 6 to 11, the target event detection unit 112 detects the attitude change of the equipment (racquet 6) during the motion, which is calculated based on the detection information. A target event may be detected based on at least one of speed and movement speed.

A motion analysis system 100 according to an aspect 13 of the present disclosure includes a display control unit 214 that superimposes and displays a skeletal model generated based on data of a first video on a first still image in any of the aspects 1 to 12 above. may be provided.

A motion analysis system 100 according to an aspect 14 of the present disclosure includes at least one user terminal 2 capable of capturing a first moving image when acquiring detection information, and a first acquisition unit (analysis at least one motion analysis device (server device 1) comprising a request reception section 111), a second acquisition section (analysis request reception section 111), a target event detection section 112, and a first extraction section 113; The analysis device (server device 1) detects the target event and extracts the first still image in response to acquiring the analysis instruction including the detection information and the first video from the user terminal 2. good.

In the motion analysis system 100 according to aspect 15 of the present disclosure, in the aspect 14, the user terminal 2 transmits the motion analysis device (server device 1) to the motion analysis device (server device 1) on the date when the first video transmitted to the motion analysis device (server device The first still images acquired from 1) may be managed in association with each other.

The motion analysis system 100 according to aspect 16 of the present disclosure is configured such that, in

aspect

14 or 15, the first still image acquired from the motion analysis device (server device 1) is associated with a day and a day with which the first still image is not associated. It may also include a display control unit 214 that distinguishes and displays the .

In the motion analysis system 100 according to aspect 17 of the present disclosure, in any of the aspects 1 to 16, the target event is that the tool (racquet 6) used by the user comes into contact with a predetermined moving body, or the tool ( The racket 6) may be in a predetermined posture.

The motion analysis method according to aspect 18 of the present disclosure includes an acquisition step (step S17), a target event detection step (step S18) in which the computer (server device 1) detects a target event that occurred during operation based on the detection information; The method includes a first extraction step (step S18) of extracting one first still image from the first moving image.

1 Server device (behavior analysis device)
2 User terminal 3 User terminal (motion analysis device)
4 Network 5 Motion sensor device (motion sensor)
6 Racket 8 Table tennis table 21

Control units

22, 52 Storage unit 23 Input unit 24 Display unit 26 Imaging unit 53 Angular velocity sensor 54 Acceleration sensor 100 Motion analysis system 111 Analysis request reception unit (first acquisition unit, second acquisition unit)
112 Target event detection section 113 First extraction section 114 Second moving image generation section 115 Second extraction section 211 Imaging control section 212 Measurement value acquisition section 213 Display control section 214 Data management section

Claims

a first acquisition unit that acquires a first video capturing an image of the user's actions;
a second acquisition unit that acquires detection information regarding the motion detected by a motion sensor;
a target event detection unit that detects a target event occurring during the operation based on the detection information;
a first extraction unit that extracts at least one first still image corresponding to the detected target event from the first video;
Equipped with
Motion analysis system.
The first extraction unit is capable of extracting a first related image group, which is a plurality of still images before and after the first still image, from the first moving image.
The motion analysis system according to claim 1.
The first extraction unit extracts the at least one first still image for each detected target event when a plurality of target events are detected.
The motion analysis system according to claim 1 or 2.
comprising a display control unit that displays the at least one first still image and the first related image group in chronological order in the first moving image;
The motion analysis system according to claim 2.
The display control unit displays the at least one first still image and the first related image group in chronological order for each detected target event.
The motion analysis system according to claim 4.
The operation is an operation in which the user uses a tool,
a second video generation unit that generates a second video showing a change in posture of the tool during the operation based on the detection information;
a second extraction unit that extracts at least one second still image corresponding to the detected target event from the second moving image;
The motion analysis system according to any one of claims 1 to 5.
When a plurality of target events are detected, the second extraction unit is capable of extracting a plurality of second still images corresponding to each of the detected target events.
The motion analysis system according to claim 6.
The second video generation unit is capable of generating a plurality of second videos corresponding to each of the detected target events,
The second extraction unit is capable of extracting at least one second still image from each of the generated second moving images.
The motion analysis system according to claim 6 or 7.
The second extraction unit is capable of extracting a second related image group, which is a plurality of still images before and after the second still image, from the second moving image.
The motion analysis system according to any one of claims 6 to 8.
comprising a display control unit that simultaneously displays the first still image and the second still image;
The motion analysis system according to any one of claims 6 to 9.
comprising a display control unit that displays at least one of the first moving image and the second moving image;
The motion analysis system according to any one of claims 6 to 10.
The target event detection unit detects the target event based on at least one of a speed of posture change and a movement speed of the tool in the operation, which are calculated based on the detection information.
The motion analysis system according to any one of claims 6 to 11.
comprising a display control unit that displays a skeletal model generated based on data of the first moving image superimposed on the first still image;
The motion analysis system according to any one of claims 1 to 12.
at least one user terminal capable of capturing the first video when acquiring the detection information;
at least one motion analysis device comprising the first acquisition unit, the second acquisition unit, the target event detection unit, and the first extraction unit;
The motion analysis device includes:
Detecting the target event and extracting the first still image in response to acquiring an analysis instruction including the detection information and the first video from the user terminal;
The motion analysis system according to any one of claims 1 to 13.
The user terminal manages the first still image obtained from the motion analysis device by associating it with the date on which the first video transmitted to the motion analysis device was captured.
The motion analysis system according to claim 14.
comprising a display control unit that distinguishes and displays days to which the first still image acquired from the motion analysis device is associated and days to which it is not associated;
The motion analysis system according to claim 14 or 15.
The target event is that the tool used by the user comes into contact with a predetermined moving object, or that the tool takes a predetermined posture.
The motion analysis system according to any one of claims 1 to 16.
an acquisition step in which the computer acquires a first moving image of the user's motion and detection information regarding the motion detected by the motion sensor;
a target event detection step in which the computer detects a target event that occurred during the operation based on the detection information;
a first extraction step in which the computer extracts at least one first still image corresponding to the detected target event from the first video;
including,
Motion analysis method.