WO2020241738A1 - Training assisting method and device - Google Patents

Abstract

The problem is how to calculate and compare motion sequences, and communicate the motion sequences to a learner with respect to a physical activity in which the motion sequences are considered important in particular. A training assisting method comprises: preparing a first motion sequence of a physical activity as a reference; acquiring a second motion sequence from activity data of the physical activity of a learner; and presenting a difference between the first motion sequence and the second motion sequence to the learner as acoustic information.

Description

Training support methods and equipment

The present invention relates to training support methods and devices.

In sports instruction, there are many methods in which physical exercise is measured and photographed, and the instructor proposes improvement points based on the results. Taking golf instruction as an example, a method is often adopted in which a ballistic measuring instrument is used to measure a hit ball, and the instructor proposes points for improvement based on the result and the posture during a swing taken by a video camera.

However, many of the proposals often depend on the experience and intuition of the instructor, and due to the difference in feelings between the instructor and the learner, the improvement points may not be well communicated to the learner, and even more. , There is a possibility that the communication will be contrary to the intention of the leader.

Golf is one of the sports for which a sequence of movements has been established, and research on physical exercise in golf swings is being conducted. Hogari et al. (Non-Patent Document 1) attempted to measure the position and angle of joints in a swing and to quantify the norms and skills expressed in the golf instruction book (Non-Patent Document 2). In addition, Jorgensen (Non-Patent Document 3) modeled the arm and golf club during a golf swing as a double pendulum, and Inoue et al. (Non-Patent Document 4) investigated the relationship between the joint torque of the arm and wrist and the swing. The mode analysis of the double pendulum model was performed for the purpose of. Tinmark et al. (Non-Patent Document 5) focused on three parts, the hand, shoulder, and waist, and investigated the relationship between the order in which the rotation speed of each part was maximized and the speed of the head. Vena et al. (Non-Patent Document 6) investigated the relationship between the timing at which the rotational speeds of the left arm, shoulder, and hip are maximized and the skill level.

In a golf swing, the difference in the order and timing of movements in multiple body parts is called Kinematic Sequence, and is recognized as an important factor for efficiently increasing head speed (non-patented). Document 7). As shown in FIG. 20, the maximum value of the rotational speed of each part during the swing occurs in the order of the waist (pelvis), chest (upper body), arms, and golf club. That is, in the downswing, it is shown that the movements are chained in the order of the lower body (legs), the waist, the shoulders, the arms, and the club.

A golf swing is a movement that is completed while a plurality of body parts interact with each other in a short time, and it is difficult to visually grasp the chain of movements of the body parts even if the swing movie is played. In addition, it is already advanced people who can recognize whether the timing of movement of each body part is performed in a desirable manner and can consciously correct it. Based on these facts, it is important to convey to the learner the evaluation of the motion sequence, mainly the Kinematic Sequence, in an easy-to-understand manner.

It has been conventionally known to use sound for golf practice, and it is described in, for example, Patent Documents 1 to 3. The methods described in these documents allow the learner to swing in time with the sound, and do not evaluate the learner's swing.

JP 2001-299966 JP 2013-39357 U.S. Pat. No. 5,743,807

The present inventors have tackled the problem of how to calculate, compare, and convey this to the learner, especially in physical exercise in which the sequence of movement is important.

The technical means adopted by the present invention is
Prepare the first movement sequence of physical exercise as a reference,
Obtain the second motion sequence from the exercise data of the learner's physical exercise,
Presenting the difference (difference) between the first motion sequence and the second motion sequence to the learner as acoustic information.
Training support method.
One embodiment comprises obtaining the difference between the first and second motion sequences.
The acquired difference is presented to the learner as acoustic information.

In one aspect, the motion sequence is defined by a plurality of events that occur at different times.
One embodiment comprises obtaining the difference between the first and second motion sequences.
The difference includes the difference in the order in which the events occur and / or the timing in which the events occur.

In one aspect, each of the events is assigned a sound.
The first operation sequence and the second operation sequence can be converted into acoustic information (sound time series information).

In one embodiment, presenting the learner with acoustic information the difference between the first and second motion sequences
Presenting the first acoustic information corresponding to the first motion sequence,
Including presenting the second acoustic information corresponding to the second motion sequence,
This includes making the learner recognize the difference by comparing the first acoustic information and the second acoustic information.
In one aspect, when the order of the events in the first action sequence and the order of the events in the second action sequence are different, the sound of the event in the wrong order is changed when the second acoustic information is presented.

In one aspect, the learner is provided with a means for converting the difference into acoustic information, and the difference converted into acoustic information is presented to the learner.

In one embodiment, the difference between the first motion sequence and the second motion sequence is presented to the learner in a visually recognizable manner.
For example, a diagram comparing the first operation sequence and the second operation sequence (for example, the difference may be emphasized) may be displayed.
Alternatively, the analysis result of the difference between the first operation sequence and the second operation sequence may be illustrated.
The frame image corresponding to the event with a large difference may be displayed.
Further, the swing image may be presented at the same time as the presentation of the sound sequence.

The training support device adopted by the present invention is
A means of storing the first motion sequence of physical exercise as a reference,
A means of acquiring a second motion sequence from the learner's physical motion data,
It is provided with a means for presenting the difference (difference) between the first operation sequence and the second operation sequence to the learner as acoustic information.
In one embodiment, the means for obtaining the difference between the first operation sequence and the second operation sequence is included.
The presenting means presents the acquired difference to the learner as acoustic information.

In one aspect, the sequence of actions is defined by multiple events that occur at different times.
It is equipped with a means for acquiring the event occurrence time.
In one embodiment, the difference includes a difference in the order in which the events of the first operation sequence and the second operation sequence occur, and / or the timing of occurrence of the events.
Includes means for obtaining the event occurrence order and / or the difference in event occurrence timing.

In one aspect, each of the events is assigned a sound.
A means for converting the first operation sequence and the second operation sequence into acoustic information (time-series information of sound) is provided.

In one aspect, the learner is provided with a means for converting the difference into acoustic information, and the difference converted into acoustic information is presented to the learner.

The present invention can also be provided as a computer program that causes a computer to function as each of the above means.

In one aspect, the physical exercise is a golf swing.

The present invention is a method for quantitatively evaluating a learner's movement, and can be used as a learner's self-training support means and a training guidance support means.
In the present invention, the learner can perform his / her own movement by presenting the difference between the reference movement sequence and the learner's movement sequence acquired from the movement data to the learner as acoustic information in real time or near real time. It provides an intuitive way to be aware and make corrections.
According to the present invention, the accuracy and validity of one's own physical exercise (for example, golf swing) can be fed back to the learner in real time or near real time.
In one example, the present invention can be used to support training of a golf swing, but the present invention is not limited to the golf swing, and the training effect can be enhanced in other exercise training, dance training, rehabilitation and the like.

It is an overall view of the training support system which concerns on this embodiment. It is a schematic diagram of the training support method which concerns on this embodiment. It is a figure which shows the step which outputs the motion data as the time series information of a sound. It is a figure which shows the step which a learner is made to compare a self-movement and a reference movement as acoustic information. It is a figure which shows the step which presents the difference between a self-movement and a reference movement to a learner as acoustic information. It is a figure which shows the part and the number (1 to 20) of a feature point of a body. It is a figure which shows the center (a to j) of each body part. It is a figure which shows the axis (k, l) of a body. It is a figure which shows the shoulder line segment, the waist line segment, both arms, and a golf club. The definitions of the left thigh and left lower limb are shown. It is a figure which shows the definition of the left forearm vector and the club shaft vector. It is a figure which shows a series of movements of a golf swing. The figure on the left shows the first event (start of left knee extension: time when the angle of the left knee starts to increase), and the figure on the right shows the second event (release of left wrist fixation: time when the left wrist is returned). The figure on the left shows the third event (waist brake: time when the rotation angular velocity of the waist reaches the maximum), and the figure on the right shows the fourth event (maximum hand speed: time when the center position of the wrist reaches the maximum speed). is there. The figure on the left shows the fifth event (maximum head speed: time when the maximum speed of the club head), and the figure on the right shows the sixth event (shoulder brake: time when the rotation angular velocity of the shoulder becomes maximum). It shows the conversion of the time series of events into a time series of sounds, and the audio intervals are evenly spaced. Shows the operation sequence converted into acoustic information, the model is the reference operation data, Example 1 is when only the occurrence timing of some events is shifted (no order error), Example 2 is the order of some events It corresponds to the case where is replaced. It is a figure exemplifying the deviation information obtained from the deviation of the order, and the position of the wrong event, the phase difference of each event, and the event of the maximum phase difference are obtained. It is a figure which shows the presentation of acoustic information paying attention to the event of the maximum phase difference. It is a figure which shows the Kinematic Sequence (Kinematic sequence) in a golf swing (see Non-Patent Document 7).

[A] Outline of the training support system [A-1] Hardware configuration of the system The training support system according to the present embodiment will be described with reference to FIGS. 1 and 2. The training support system according to the present embodiment includes a moving image acquisition unit, a storage unit for storing various data, a processing unit for executing various calculations, an image output unit (display), and an audio output unit. Typically, the video acquisition unit is composed of a video camera, and the storage unit, processing unit, image output unit, and audio output unit include one or more computers (input unit, arithmetic unit consisting of CPU, RAM, ROM). It is composed of a storage unit and an output unit).

[A-2] Motion capture The means for acquiring time-series data of physical exercise is so-called motion capture, and in one aspect, motion capture is a motion acquisition unit (video camera) that acquires moving image data of a target motion. ) And an action acquisition unit that acquires time-series data representing the target action based on the moving image data. The moving image data acquired by the camera and the time-series data of the posture of the target acquired by motion capture are stored in the storage unit.

The movement of the target is defined by the time series data of the posture of the target acquired by motion capture. The posture of the target is specified by a plurality of feature points (typically joints) on the target body, and by acquiring three-dimensional coordinate values of the plurality of feature points in each frame, three of the plurality of feature points are obtained. The target operation is defined from the time-series data of the dimensional coordinate values. The type of motion capture used in the present invention is not limited, and an optical motion capture using an optical marker that identifies a feature point, a so-called inertial sensor such as an acceleration sensor, a gyroscope, or a geomagnetic sensor is attached to the target body. Therefore, a method of acquiring the target motion data, a so-called markerless motion capture in which an optical marker or a sensor is not attached, and the like can be exemplified. In the present embodiment, motion measurement is performed by optical motion capture using an optical marker, but markerless motion capture is advantageous from the viewpoint of not interfering with the natural motion of the target. As markerless motion capture, motion capture using a system equipped with a camera and a depth sensor (represented by Kinect) or deep learning is used to analyze RGB images from one or more viewpoints and motion data. A motion capture to acquire can be exemplified.

[A-3] Motion sequence of physical exercise The motion sequence of physical exercise is an event in whole body exercise in which a predetermined body part or a plurality of linked body parts express as a change in the relative positional relationship with other body parts. The group is described as having an order relationship and a time relationship (timing). That is, the operation sequence is defined by a plurality of events that occur at different times. The event is defined by the velocity (maximum velocity, maximum angular velocity, minimum angular velocity, etc.), position (angle, etc.) of one or more body parts. The operation sequence acquisition means includes means for acquiring the occurrence time of each event. The event occurrence time acquisition means acquires the occurrence time of each event that defines the learner's operation sequence, and stores the occurrence time of each event and the order of the events obtained from the occurrence time in the storage unit.

For example, in a physical exercise in which _five events E ₁ , E ₂ , E ₃ , E ₄ , and E ₅ occur in this order, the reference motion sequences are E ₁ , E ₂ , E ₃ , E ₄ , E _{5 and so on.} It can be specified by the order of occurrence and the timing of occurrence. By acquiring the occurrence time of each event E ₁ , E ₂ , E ₃ , E ₄ , and E ₅ , the order of the events can be known from the occurrence time. In addition, the occurrence timing (relative timing) of each event can be acquired from the occurrence time of each event. Even for the same physical exercise (for example, golf swing), the time between the occurrence of events in the physical exercise is relative because the time from the start to the end varies from person to person. Therefore, it is possible to compare the occurrence timings of each event by acquiring the start time point and the end time point of the physical exercise, sharing the entire time of the physical exercise, and relativizing the occurrence timing.

[A-4] Event Occurrence Time Acquisition Means In the event occurrence time acquisition means, what kind of information is used and how the event occurrence time is detected may differ depending on the event. The event occurrence time acquisition means uses time-series data of the target motion, and uses a predetermined angle, angular velocity, angular acceleration, velocity, rotation speed, acceleration, and position (relative to other parts) of one or more parts that define the event. Acquire the occurrence time of a predetermined event using the relative position, relative position with respect to an object other than the body, etc.), posture, center of gravity, etc. as conditions.

[A-5] Sound source data is stored in the sound source data storage unit. As for how to use the sound source data, when sound is assigned to each event that defines the operation sequence (reference operation sequence), or when sound is assigned to the difference between the reference operation sequence and the target person's operation sequence. There is. In the former, for example, a sound stored as sound source data is assigned to each event in advance, and the operation sequence is converted into time-series information of the sound by the acoustic information conversion means of the operation sequence and output from the audio output unit. To do. It is desirable that each sound assigned to each event is a sound that can be distinguished from each other and that the sequence of sounds can be easily identified. Further, a familiar and known sound source may be used as a sound sequence (sound time series information) corresponding to the reference operation sequence. In the latter, the sound assigned to each event is independent of the sound assigned to each event (including the case where no sound is assigned to each event), and the sound is assigned to the difference, or the sound assigned to each event is assigned. An example is the case where the sound is presented in a changed manner (for example, the pitch is raised or lowered by a semitone).

[A-6] The reference operation sequence storage unit stores the time and order of occurrence of each event that defines the reference operation sequence. The reference motion sequence is obtained from ideal motion data that serves as a model / model. For the physical exercise for which the motion sequence has been established, the physical exercise of the target to be a model is videotaped to acquire the exercise data, and the time-series data of the exercise data to be the model is acquired. Using the acquired time-series data, the occurrence times of a plurality of events that define the movement sequence of physical exercise are acquired, and the occurrence time and the occurrence order of each event are stored in the storage unit as a reference movement sequence. Video of physical exercise and time-series data of posture are also stored in the storage unit.

It is also possible to prepare a plurality of different reference operation data (event set that defines the operation sequence and the occurrence time and order of events), and select an operation sequence to be used as a reference from a plurality of candidates. Since the model physical exercise and the ideal physical exercise are not necessarily one, a plurality of reference motion sequences may be acquired and stored in the storage unit. In addition, the set of events that define a certain motion sequence is not always one set, and multiple types of motion sequences (at least some events are different) are acquired from the same physical exercise data as a reference and stored in the storage unit. You may keep it.

[A-7] Conversion of operation sequence to music information The step of converting an operation sequence into acoustic information is illustrated in FIG. First, a moving image of the movement when the target performs physical exercise is acquired, and time-series data of the movement of the target is acquired from the moving image data. Based on the time-series data of the target motion, the occurrence time and the occurrence order of each event that defines the motion sequence of the physical exercise are acquired. By assigning a sound to each event, the operation sequence is converted into acoustic information (sound time series information). Immediately after the subject performs physical exercise, the motion sequence can be converted into acoustic information and the acoustic information can be presented to the subject.

As the sound interval in the sound sequence (sound time series information), the same interval as the actual event may be used, or the relative interval may be maintained and the time may be lengthened. When presenting a sound sequence, if it is difficult to distinguish the sound at the same timing as the actual event sequence, it is not necessary to present the sound at the same timing as the event. That is, for example, in a golf swing, the time width of each event that defines the swing sequence and the time width of the presented sound sequence may be different (not swinging according to the sound). Alternatively, when the event occurrence intervals obtained from the exercise data as a model are converted into equal intervals and the model is expressed by voice, predetermined sounds may be presented in a predetermined order at equal intervals.

As support for training using such acoustic information, it can be exemplified that the learner listens and compares the acoustic information corresponding to the reference motion sequence and the acoustic information corresponding to his / her own motion sequence. As shown in FIG. 4, first, the learner is presented with acoustic information of a reference motion sequence (for example, a golf swing). Next, the learner's physical exercise (for example, golf swing) is performed, a motion sequence is acquired from the learner's motion, and the acquired motion sequence is converted into acoustic information and presented to the learner. When presenting the acoustic information corresponding to the learner's motion sequence, the pitch of the sound corresponding to the event having a different order from the reference motion sequence may be changed (FIG. 17 “Example 2”).

In this example, the learner is made to recognize the difference by comparing the acoustic information corresponding to the reference motion and the acoustic information corresponding to the own motion to the learner. The timing and number of times that the acoustic information corresponding to the reference motion and the acoustic information corresponding to the own motion are presented to the learner are not limited. For example, by comparing the acoustic information obtained by converting the learner's motion sequence with the reference acoustic information for each swing, it is intuitive to know which swing is closer to or far from the reference swing depending on the degree of difference in hearing. Can be judged.

[A-8] Comparison of Action Sequences The action sequence comparison means compares the occurrence time and order of each event that defines the learner's action sequence with the occurrence time and order of each event that defines the reference action sequence. , The difference (difference) in the operation sequence is acquired, and the information on the difference in the operation sequence is stored in the storage unit. The difference in the operation sequence includes, but is not limited to, the difference in the event occurrence order and the difference in the event occurrence timing. The difference in the occurrence timing (occurrence interval) can be obtained, for example, by comparing the ratio of each occurrence interval to the total time of physical exercise. Depending on the physical exercise, it may be desirable to compare both the event occurrence order and the event occurrence timing, and in some cases it may be necessary to compare only the event occurrence order, or only the event occurrence timing. In some cases it may be good. It should be noted that items other than the motion sequence may be compared between the reference motion and the learner's motion. Various items can be compared by using the time-series data of the posture that defines the movement as a reference and the time-series data of the posture that defines the movement of the learner. For example, the speeds of reference movements and learner movements (eg, head speed in a golf swing) may be compared.

In FIG. 5, the reference motion sequence and the learner's motion sequence are compared to obtain the difference, the difference is converted into output information (for example, acoustic information), and the output information (for example, acoustic information) is presented. Shows the steps. First, an image of the learner's movement is acquired, and time-series data of the learner's movement is acquired from the image data. Using the time-series data of the learner's movements, the occurrence time and the occurrence order of each event that defines the learner's movement sequence are acquired. By comparing the occurrence time and order of each event that defines the learner's action sequence with the occurrence time and order of each event that defines the reference action sequence, the difference in the action sequence is obtained. The difference in the operation sequence is, for example, a difference in the event occurrence order and a difference in the event occurrence timing. The information on the difference in the order of occurrence (displacement information obtained from the difference in the order) includes, but is not limited to, the position of the wrong event, the phase difference of each event, and the event of the maximum phase difference (see FIG. 18).

[A-9] Conversion of difference in motion sequence to music information The difference between the learner's motion sequence and the reference motion sequence is converted into acoustic information by, for example, an acoustic information conversion means of the difference in motion sequence, and is output as voice. It can be output from the unit. There is no limitation on how to convert the acquired difference (part or all) into output information (acoustic information), but for example, when presenting a sound sequence to a learner, the difference is related to the sound sequence. Adding information can be mentioned.

For example, the order in which the sound sequence is presented after the learner's action is the order corresponding to the actual learner's action sequence, and the sounds corresponding to the events whose order is different from the reference action sequence are changed and presented. More specifically, in listening to the sound corresponding to the reference action sequence and listening to the sound corresponding to the own action sequence, the sequence of events and the self action that define the reference action sequence are specified. When there is a difference in the order of the events that define the sequence, the order of the events is the order of the learner's action sequence (wrong order), and the sound corresponding to the wrong event is presented in a different manner. The change in sound can be exemplified by raising or lowering a semitone by typically changing the pitch (Fig. 17, Example 2). For example, by selecting each sound so that it becomes a chord when multiple sounds corresponding to the reference action sequence are played at the same time, a dissonant sound (dissonance) when multiple sounds corresponding to the own action sequence are played at the same time. By changing the pitch), the sound corresponding to each event is played at the same time after the self-motion is completed, so that the presence or absence of a difference between the reference motion sequence and the self-motion sequence is immediately recognized. Can be done.

Alternatively, the order in which the sounds are presented after the learner's actions is the order corresponding to the reference action sequence, and the event that occurs in a different order from the reference action sequence (for example, the event that occurs first among a plurality of events in different orders). ) May be changed (for example, pitch bend) and presented. Alternatively, in the sound sequence, the pitch of the event with the maximum phase difference may be changed and presented by using the order corresponding to the reference operation sequence (see FIG. 19). That is, the event of the maximum phase difference and the phase difference thereof can be used as the difference information. By selecting the event with the largest error, the learner can be made aware of what the event with the largest error was. Specifically, the position of the maximum phase difference in the acoustic information is used, and the value of the phase difference is added as information to the sounds in the same place as the ideal order. If there are multiple events with the maximum phase difference, the one that is the first in the obtained order is adopted. By always giving priority to the one in front of you, you can fix what you are conscious of. Further, the pitch may be changed according to the magnitude of the phase difference.

The sound sequence corresponding to the reference operation sequence may be converted into equal intervals and presented to the learner (see FIG. 16). When the order of the events that define the learner's action sequence is the same as the reference action sequence, and the timing of occurrence of one or more events is different, the sound sequence corresponding to the reference action sequence is generated. It may be changed with respect to the timing (Fig. 17, Example 1). For example, the sound sequence of the model data is sounded at equal intervals (0.2 seconds), while the sound sequence of the evaluation target data is delayed at the timing of a certain event, for example, at equal intervals. It may be delayed by 0.1 seconds, and if it is early, it may be sounded 0.1 seconds earlier than the equal interval. If there is a difference in the event occurrence order, it may be set not to evaluate the difference in the event occurrence interval, or even if there is a difference in the event occurrence order, it occurs. The difference in intervals may be evaluated.

The conversion of the difference in the motion sequence into acoustic information is not limited to the above example, and for example, during the learner's movement (for example, swing), the event of the motion sequence is referred to as the event of the motion sequence. The sound may be played in real time at a time different from the order of. Further, for the event in which the deviation of the event occurrence order is the largest as compared with the reference operation sequence, the name for specifying the event and the corresponding body part may be presented by voice, and the event may be presented. An image of the frame at the time of occurrence may be presented.

[A-10] Presentation of visual information It is useful to visually present various information to the learner. For example, moving image data of the learner's physical exercise or moving image data as a reference may be presented from the image output unit, or time-series data of the learner's posture may be applied to a musculoskeletal model and displayed. The frame image corresponding to each event may be displayed, or the frame image corresponding to each event may be displayed in comparison with the reference frame image and the learner's frame image. The image corresponding to each event may be displayed alone or in comparison with the reference as an image having skeletal information and musculoskeletal information.

A system that displays the musculoskeletal model obtained by performing inverse kinematics calculation, inverse kinematics calculation, and muscle tension estimation in real time when the time-series data of the learner's posture is applied to the musculoskeletal model and displayed. (Akihiko Murai, Kosuke Kurosaki, Katsu Yamane and Yoshihiko Nakamura. “Musculoskeletal-see-throuhg mirror: Computational modeling and algorithm for whole-body muscle activity visualization in physiology in real time” 310-317, 2010) may be used to present a series of skeletal movements in a swing. In this way, by visualizing the movement of the skeleton of the whole body in a series of swing movements, it is possible to encourage the player to understand his / her swing.

For the event in which the deviation of the event occurrence order is the largest as compared with the reference motion sequence, an image of the frame at that time may be presented to encourage the user to pay close attention to the motion of the body part. The contrast or difference between the learner's motion sequence and the reference motion sequence may be visually presented to the learner using a diagram or a graph. The learner's physical exercise motion analysis results (which may include comparison with reference exercise data) may be displayed using figures and graphs. Prepare a correspondence table of the change pattern of the event order acquired from the learner's motion data and the swing habit and the correction method, and present the correction method to the learner according to the obtained motion sequence (event order). It may be (visually or audibly).

[A-11] Target physical exercise The target physical exercise is not limited, but a sequence of golf swing, baseball bat swing, baseball pitching motion, karate kata, dance, rehabilitation, and other motions is established. It is possible to exemplify exercises that are being performed and exercises that have a fixed routine. For example, golf is one of the sports for which a sequence of movements has been established. In dance practice and rehabilitation, training starts from the point of acquiring a sequence of movements.

As shown in the embodiment described later, for example, taking a golf swing as an example, the operation sequences are (1) start of left knee extension, (2) release of left wrist fixation, (3) waist brake, and (4) hand. It consists of 6 events: maximum speed of (5) maximum speed of head, (6) brake of shoulder, and each uses the time of angle change of a predetermined part and the time of maximum speed of a predetermined part. Get the time of the event. In the embodiment described later, the time of each event is calculated from the time series data of the three-dimensional position information of each marker measured by the optical motion capture, using the time of the angle change and the time of the maximum speed. In a golf swing, the correct answer for a set of events of interest is not necessarily one, and the above six events are examples. The reference operation sequence may be composed of some of these events, or the reference operation sequence may be composed of other events. It will be appreciated by those skilled in the art that a set of other events that identify the kinematic sequence may be selected. Further, as long as the time series data of the posture of the target is obtained, it is possible to specify the occurrence time of any event.

[B] Application to golf swing Golf is one of the sports in which a sequence of movements is established. In golf, the sequence of swing movements is known as the Kinematic Sequence (Non-Patent Document 7, FIG. 20). In the explanation described later, the explanation will be given according to the right-handed swing, but the left-handed swing may be discussed by exchanging the left and right sides.

[B-1] Definition of feature points and each time In this embodiment, the body movement during the swing is measured by optical motion capture, and the swing movement data is based on the time change of the marker position attached to the whole body and the golf club. To get. In the present embodiment, the feature points of interest in the whole body and the golf club and the golf swing process are defined as follows. The selection and definition of these feature points are examples and do not limit the contents of the present invention.

[B-1-1] Definition of whole body and golf club feature points Professional golf instructors focus on the factors that are generally noted in evaluating exercise in golf coaching when selecting specific physical feature points from the whole body. We conducted an interview survey. In the present embodiment, with respect to the feature points on the body (measured marker positions (FIG. 6)) shown in Table 1, the center positions (FIGS. 7 and 8) of the body feature points are defined as follows, and they are defined as follows. Body parts were defined by connecting (Tables 1 and 2).

[shoulder]
The average of the right shoulder front position x _{rf, sh} and the right shoulder rear position x _{rb, sh} is taken and defined as the right shoulder center position x _{r, sh} . Similarly, the left shoulder center position x _{l, sh} is defined based on the left shoulder front position x _{lf, sh} and the left shoulder rear position x _{lb, sh} . The shoulder line segment connects these two points (Fig. 9).

[Waist]
The left and right positions of the pelvis are defined as the left hip position x _{l, hi} and the right hip position x _{r, hi, respectively,} and the line segment connecting these two points is defined as the waist line segment (Fig. 9).

[Upper body axis]
The upper body axis is the line segment connecting the two points of the midpoint x _{u, ax} of the shoulder line segment and the midpoint x _{l, ax} of the waist line segment (Fig. 9).

[Both arms]
The right wrist outside x _{ro, wr} , inside x _{ri, wr} , left wrist outside x _{lo, wr} , inside x _{li, wr} are averaged, and the value is defined as the center position of the wrist x _wr ( FIG. 8). The line segment connecting the right shoulder center x _{r, sh} and the wrist center x _wr is the right arm, and the line segment connecting the left shoulder center x _{l, sh} and the wrist center x _wr is the left arm (Fig. 9).

[Golf club]
The positions of the two heads of the golf club (outside the face x _{o, fa} , inside x _{i, fa} ) are averaged, and the value is defined as the center position x _{fa of the} face. Then, the line segment connecting the two points of the wrist center x _wr and the golf club face center x _fa is defined as the golf club (Fig. 9).

[Left leg]
The positions of the lateral x _{lo, kn} and medial x _{li, kn} of the left knee are averaged, and the positions are defined as the central position x _{l, kn} of the left knee. The positions of the lateral x _{lo, an} and medial x _{li, an} of the left ankle are averaged, and the positions are defined as the central position of the left ankle x _{l, an} . The line segment connecting the left hip x _{l, hi} and the left knee center x _{l, kn} is the left thigh, and the line segment connecting the left knee center x _{l, kn} and the left ankle center x _{l, an} . Is the left lower leg (Fig. 10).

[Left forearm]
Hidarihijisotogawa x _{lo, el} and the inner x _li, averaged position of _el, defines its position center position x _l left _{elbow, el} a. The positions of the outer side x _{lo, wr} and the inner side x _{li, wr} of the left wrist are averaged, and the position is defined as the center position x ^{l, wr} of the left wrist. The line segment connecting the two points of the center of the left elbow x _{l, el} and the center of the left wrist x _{l, wr} is the left forearm (Fig. 11).

[Club shaft]
For the shaft part of a golf club, the position x _{rg, sh} on the right side of the shaft and the position x _{lg, sh} on the left side of the shaft are averaged and defined as the shaft position x _sh . From this point, the line segment connecting the center position of the club face x _fa is _defined as the direction of the club shaft (FIG. 11).

[B-1-2] Definition of each time in the golf swing Based on the face center position x _fa of the golf club, a series of movements of the golf swing is divided into a plurality of steps, and the start time, the end time, and the boundary time of each step are divided. Was defined as follows. The following four times are defined as backswing, downswing, and follow-through, respectively.

(1) Address time t _add
The time to determine the initial posture and start swinging the golf club backwards. That is, it is defined as the time when the face center position x _fa starts to move from the stationary state in the opposite direction to the hit ball.

(2) Top time t _top
The time when the golf club finishes swinging backward and the golf club starts swinging down toward the ball.

(3) Impact time t _imp
When the golf club has been swung down and the ball collides with the front of the golf club head. The face center position is defined as the time when the distance from the address x _fa (t _add ) is the minimum.

(4) Finish time t _fin
The end time of a series of swing movements. It is defined as the time when the movement speed of the face center position x _fa after impact is the smallest.

[B-3] Motion analysis focusing on body parts A golf swing motion measurement experiment was conducted for one golf instructor using optical motion capture. In this embodiment, in addition to the optical motion capture, a golf swing motion measurement experiment was carried out using an electromyogram and a floor reaction force meter. The electromyogram and the floor reaction force meter are not used to acquire the motion sequence during a golf swing, but are used, for example, when a musculoskeletal model is to be displayed on a display. Table 3 shows the parts where the markers and myoelectric potential sensors used for optical motion capture are attached.

The golf ball was actually hit and the marker position of the whole body in the movement was measured. By attaching a marker to the driver to be used, the movement of the club was also measured at the same time. In addition, the number of shots measured was 12, and the number of hit balls was also measured using a ballistic measuring device for each shot. After each shot, the shot feeling and swing posture were self-evaluated, and the one with a high self-evaluation score was regarded as a successful shot, and the one with a low self-evaluation score was regarded as a failed shot. The trial with the highest score was used as a model shot. In golf instruction, we focused on the body parts that are most noticeable when visually evaluating shots, and performed motion analysis. After that, the success swing and the failure swing are compared, and the factors that strongly affect the swing performance are adopted.

For each body part defined as described above, the relationship between the body movement of the golf swing and the movement of each body part was examined. In the motion analysis, the height change from the address posture of the upper right corner of the pelvis, the height change from the address posture of the upper left corner of the pelvis, the angle change from the address posture of the shoulder, the angle change from the address posture of the waist, the upper body axis. Angle change from address posture, tilt of upper body axis from vertical direction, tilt of golf club shaft from vertical direction, change from address posture of distance between right shoulder and wrist, address of distance between left shoulder and wrist A total of 10 items were adopted: the difference from the posture and the difference between the angle change from the address posture of the shoulder and the waist.

The movement of each body part during the swing was investigated. Immediately before the impact, the hips move vertically upward and follow through to move in the hitting direction, and the upper body axis falls in the opposite direction to the hitting direction (to the right with respect to the learner). Going forward, the angle between both arms and the golf club was maintained at an almost constant value from the address to a certain point in the downswing, but it was observed that the angle changed sharply from that point. The rise of the hips is due to the extension of the left knee. Immediately before the impact, the left leg steps on the floor to brake the rotational movement of the hips, and the center of gravity of the whole body is moved upward to move the wrist and golf club. It is thought that it plays a role in increasing the speed of the head. The collapse of the upper body axis in the opposite direction to the hitting ball is a change brought about by the braking and the movement of the waist to the rotational movement of the whole body, and it also has a meaning to prevent the center of gravity of the whole body from excessive movement in the hitting direction by the rotational movement. it seems to do. The change in the angle between the hand and the golf club also suggests the implication of increasing the speed of the golf club head. By fixing the wrist in a bent state until just before the impact, the radius of gyration can be kept small and the rotation speed can be increased. It is considered that by releasing the fixation of the left wrist immediately before the impact in that state, the radius of gyration increases while the rotation speed is high, and the speed of the head increases.

[B-2] Evaluation of motion sequence in golf swing Regarding the temporal changes in the motion of a body part and the motion of another body part causing the motion, the interval and order of turning points (events) in each motion. Conduct a survey. In this analysis, we investigated the time-series relationships of the rotational movements of the waist, shoulders, arms, and golf clubs, which are the focus of attention in the Kinematic Sequence. In addition, the same investigation was conducted on the movements of fixing and releasing the wrist and the movement of extending the left knee and kicking up the floor, which were suggested to be important by the analysis in the previous section. 6 items of timing evaluation of the turning point (event) in the movement of a specific part (time when the angle of the left knee becomes maximum, time when the fixation of the left wrist is released, time when the rotation angular velocity of the waist becomes maximum, rotation angular velocity of the shoulder The time when the maximum speed, the time when the wrist speed is maximum, and the time when the head speed of the golf club is maximum) are adopted. These six events were selected based on an interview survey with a learner who is a golf instructor and a golf instruction (Non-Patent Document 2), but are examples and do not limit the present invention. .. Each event is calculated from the three-dimensional position coordinates of the body attached to the learner and the markers on the golf club.

(1) Waist rotation For the waist line segment defined in the previous section (for example, the line segment connecting the markers attached to the left side of the pelvis and the right side of the pelvis), at an arbitrary time based on the data (line segment) at the time of addressing. Each of the waist line segment and the reference line segment is projected onto the horizontal plane, the angle formed by the two line segments at each time is calculated, and the value is defined as the rotation angle of the waist. In the present embodiment, the time when the rotational angular velocity of the waist becomes maximum is set as the attention event.

(2) Rotation of the shoulder Similar to the rotation of the waist, the line segment of the shoulder defined in the previous section (the average of the front position of the right shoulder and the rear position of the right shoulder is the center position of the right shoulder, and the front position of the left shoulder and the rear position of the left shoulder The average is the center position of the left shoulder. Regarding the line segment connecting the center position of the right shoulder and the center position of the left shoulder), the shoulder is formed from the angle formed by the reference (line segment at the address time) and the projection of the line segment on the horizontal plane at any time. Define the rotation angle of. In the present embodiment, the time when the rotational angular velocity of the shoulder becomes maximum is set as the event of interest.

(3) Arm exercise Calculate the magnitude of speed at any time point for the center position of the wrist defined in the previous section (the average of the four positions on the outside and inside of the right wrist and the outside and inside of the left wrist is the center position of the wrist). However, the time when the wrist movement speed is maximized is the notable event.

(4) Fixing and releasing the wrist Maintaining the angle between both arms and the golf club during the swing is realized by fixing the left wrist. Therefore, consider the angle between the left forearm and the club shaft as a parameter that expresses the fixation of the wrist. Calculate the angle between the direction of the left forearm and the direction of the golf club at any point in time, and define it as the angle of the left wrist. Regarding this angle, the moment when the sign of the angle change changes immediately before the impact is defined as the time when the wrist is released from the fixation, and the time when the wrist is released is defined as the event of interest.

(5) Extension of the left knee The angle between the left thigh and the left lower leg is defined as the angle of the left knee. For the left knee, the state in which the knee is bent most immediately before kicking up the floor, that is, the time when the angle of the left knee is minimized is set as a noteworthy event.

(6) Golf club head movement For the golf club head position (face center position) defined in the previous section, calculate the magnitude of the speed at any time point, and set the time when the head movement speed becomes maximum. Make it a noteworthy event.

The timing of the event of interest is the start of extension of the left knee, the release of the left wrist, the brake of the waist, the maximum speed of the hand, the maximum speed of the golf club head, the impact, and the brake of the shoulder. The moment of impact almost coincided with the time when the speed of the golf club head was maximum. In each shot, the time elapsed from the start of extension of the left knee to the return of the shoulder was almost the same, but the timing of releasing the fixation of the left wrist was uneven. Especially in successful shots, the event occurred at almost the same timing except for the timing of releasing the fixation of the left wrist, while in the unsuccessful shot, the interval from the start of extension of the left knee to the return of the hip was shortened. It was. From this result, it was confirmed that the investigation of the timing of the extracted events is effective for the evaluation of the golf swing.

[B-3] Evaluation of Learner's Golf Swing For the six events that define the movement sequence of the golf swing, the results of analyzing the time of occurrence and the order of occurrence are compared with the model data, and the sound is heard with respect to hearing. Give feedback. First, let t ^ref _i and n ^ref _i be the occurrence times and order of each event in the model swing.

Next, the following matrix M is set in order to make the occurrence interval of each event of the model shot equal to t, and even in the swing to be evaluated, the processing by M is performed at the occurrence time t _i of each event. Apply (see FIG. 16).

However, M is a matrix represented by the following.

Next, for the model swing, each event is associated with a different frequency f ^ref _i sound. Here, h ^ref _i expresses the pitch difference from the sound of A (frequency 440 Hz) by the number of semitones.

In the present embodiment, when the above six sounds are played at the same time, a chord called C11 is set. Table 4 shows the assignment of sounds to each event.

Next, for the swing to be evaluated, the sound is associated with each event. Here, as compared with the model swing, if the event occurrence order is earlier, the pitch is shifted up by a semitone, and if it is late, the pitch is shifted down by a semitone. Using the above data, the timing evaluation of the swing of the model is presented by playing the sound of the frequency f ^ref _i in order for the time t, and after the time t ´ _i from the end time of the presentation of the timing evaluation of the model. _Make the sound of frequency f _i play.

In this way, by changing the order in which the sounds are reproduced and the reproduction time for the model and the swing to be evaluated, the deviation of each timing is intuitively presented. After that, the six sounds of the swing to be evaluated are simultaneously reproduced. As a result, when the event occurrence order is different from the model, the acoustic information is presented as a dissonance, and the learner can intuitively present the order deviation.

In FIG. 17, a "model" is a time series of events in a reference motion sequence. "Example 1" is a case where the left wrist is delayed and the shoulder brake is early, and there is no order error. The second note is delayed and the sixth note is earlier. "Example 2" is a case where the order of the wrist and the head is reversed, and the event intervals are set to be the same. The 4th note is a semitone lower and the 5th note is a semitone higher, and the order is reversed (all intervals remain 0.2 seconds).

By measuring and analyzing the movement of a golf swing for a professional golf instructor, the characteristic body parts and their movements in the movement are confirmed, and the evaluation items for evaluating the golf skill are selected. Ten items of posture evaluation and six items of timing evaluation of the turning point in the movement of a specific part were adopted. Based on the above selection, the evaluation results for improving golf skills were presented in comparison with the model using auditory information. In the present invention, the difference from the ideal kinematic sequence is calculated based on the analysis result (time-series information of the whole body skeletal movement of all movements), and it is presented as acoustic information immediately after the movement. We devised an intuitive method that allows the learner to be aware of his or her own movements and make corrections.

The present invention calculates the difference between its own motion sequence and the ideal motion sequence based on human motion measurement and motion analysis in real time or near real time, converts it into acoustic information, and immediately conveys it. It is intended to intuitively grasp the points to be corrected in the movement of the learner and enhance the training effect of the learner, and can be used for exercise training, golf training, dance training, rehabilitation, exercise support, and the like.

Claims (16)

1. Prepare the first movement sequence of physical exercise as a reference,
   Obtain the second motion sequence from the exercise data of the learner's physical exercise,
   Presenting the difference between the first motion sequence and the second motion sequence to the learner as acoustic information,
   Training support method.
2. Including acquiring the difference between the first operation sequence and the second operation sequence
   Present the acquired difference to the learner as acoustic information,
   The training support method according to claim 1.
3. The operation sequence is defined by multiple events that occur at different times.
   The training support method according to any one of claims 1 and 2.
4. Including acquiring the difference between the first operation sequence and the second operation sequence
   The difference includes a difference in the order of occurrence of events and / or a difference in the timing of occurrence of events.
   The training support method according to claim 3.
5. Sounds are assigned to each of the above events.
   The first operation sequence and the second operation sequence can be converted into acoustic information.
   The training support method according to any one of claims 3 and 4.
6. Presenting the difference between the first motion sequence and the second motion sequence to the learner with acoustic information is
   Presenting the first acoustic information corresponding to the first motion sequence,
   Including presenting the second acoustic information corresponding to the second motion sequence,
   Including making the learner recognize the difference by comparing the first acoustic information and the second acoustic information.
   The training support method according to claim 5.
7. It is equipped with a means for converting the difference into acoustic information.
   The difference converted into acoustic information is presented to the learner,
   The training support method according to any one of claims 1 to 6.
8. Including presenting the learner with a visually recognizable difference between the first and second motion sequences.
   The training support method according to any one of claims 1 to 7.
9. The training support method according to any one of claims 1 to 8, wherein the physical exercise is a golf swing.
10. A means of storing the first motion sequence of physical exercise as a reference,
    A means of acquiring a second motion sequence from the learner's physical motion data,
    A training support device including a means for presenting the difference between the first motion sequence and the second motion sequence to the learner as acoustic information.
11. Including means for obtaining the difference between the first operation sequence and the second operation sequence.
    The presenting means presents the acquired difference to the learner as acoustic information.
    The training support device according to claim 10.
12. The operation sequence is defined by multiple events that occur at different times.
    It is equipped with an event occurrence time acquisition means for acquiring the occurrence time of each event.
    The training support device according to any one of claims 10 and 11.
13. The difference includes a difference in the event occurrence order between the first operation sequence and the second operation sequence, and / or a difference in the event occurrence timing.
    Including means for obtaining the difference in the order of occurrence of events and / or the difference in the timing of event occurrence,
    The training support method according to claim 12.
14. Sounds are assigned to each of the above events.
    A means for converting the first operation sequence and the second operation sequence into acoustic information is provided.
    The training support method according to any one of claims 12 and 13.
15. It is equipped with a means for converting the difference into acoustic information.
    The difference converted into acoustic information is presented to the learner,
    The training support method according to any one of claims 10 to 14.
16. A computer program that causes a computer to function as each means according to claims 10 to 15.

Images