JP6283231B2 - Proficiency assessment method and program - Google Patents

Description

  The present invention relates to a proficiency level evaluation method and a program.

  In recent years, support for exercise acquisition by computer vision technology has been actively performed. Exercise learning is necessary in many scenes such as school physical education, sports clubs, dance classes, and rehabilitation centers. For example, taking sports as an example, a motion capture system is used to visually compare the movements of the model person and the trainer in the virtual reality space, and the difference between the movements of the skilled person and the trainer is calculated as a numerical value. Systems have been proposed (see, for example, Non-Patent Documents 1 and 2).

Chua, PT, Crivella, R., Daly, B., Hu, N., Schaaf, R., Ventura, D., Camill, T., Hodgins, J. and Pausch, R., "Training for Physical Tasks in Virtual Environments: Tai Chi ", Proceedings of the IEEE Virtual Reality 2003, 2003, p. 87-94 Choi, W., Mukaida, S., Sekiguchi, H. and Hachimura, K., “Quantitative Analysis of Iaido Proficiency by Using Motion Data”, ICPR 2008, 2008, p. 1-4

  Prior arts that support exercise acquisition, including the systems of Non-Patent Documents 1 and 2, are mainly aimed at comparing differences in movement between skilled and trainees. However, if a beginner trainee trains from the beginning with the movement of a skilled person as a target, there is a risk that the motivation will be reduced due to deviation from the target, or injury may occur due to an unreasonable posture. Moreover, there is also a problem that it is difficult for a beginner to understand intuitively what is wrong simply by presenting a difference in attitude or numerical value from an expert.

  In view of the above circumstances, an object of the present invention is to provide a proficiency level evaluation method and program for evaluating a proficiency level for a target action from the movement of the trainee himself.

  One aspect of the present invention is a proficiency level evaluation method executed by the proficiency level evaluation apparatus, based on three-dimensional posture data indicating the three-dimensional posture of the trainee when the trainee has tried the operation a plurality of times. Based on the relationship between the variance calculation process for quantitatively representing the variance of the motion among a plurality of trials and the value representing the variance of the motion and the proficiency level, the variance of the motion calculated in the variance calculation process is calculated. A proficiency level calculating process for calculating a proficiency level corresponding to the value to be expressed.

  Further, one aspect of the present invention is the proficiency level evaluation method described above, wherein the variance calculation process changes the position of a predetermined part obtained from the three-dimensional posture data, using the three-dimensional posture data of each trial. For each phase, a time length normalizing process for normalizing the duration of the phase in the three-dimensional posture data of a plurality of trials, and for each phase. A resampling process in which the three-dimensional posture data normalized by the time length normalization process is resampled at predetermined intervals; a dispersion of durations for each phase in the three-dimensional posture data of a plurality of trials; The variation of the trajectory for each phase of the predetermined part obtained from the three-dimensional posture data resampled by the sampling process, and the relocation for each phase. A posture variation obtained from a posture matrix obtained by arranging the sampled three-dimensional posture data, and an inter-trial variance calculation process for calculating a value that quantitatively represents the variance of motion among a plurality of trials from one or more of them. It is characterized by that.

  In one embodiment of the present invention, the computer quantifies the variance of the motion among a plurality of trials based on three-dimensional posture data indicating the three-dimensional posture of the trainee when the trainee tries the motion a plurality of times. A variance calculation unit for calculating a value representatively, and a proficiency for calculating a proficiency level corresponding to the value representing the variance of the motion calculated by the variance calculation unit based on a relationship between the value representing the variance of the motion and the proficiency level This is a program for functioning as a degree calculation unit.

  According to the present invention, it becomes possible to evaluate the proficiency level for the target motion from the trainer's own motion.

It is a block diagram which shows the structure of the skill level evaluation apparatus by embodiment of this invention. It is an example of the processing flow of the dispersion | distribution calculation part by the embodiment. It is a conceptual diagram of the phase division | segmentation by the same embodiment. It is a conceptual diagram of time normalization and resampling by the embodiment. It is a figure which shows the example of the function which calculates a skill level from dispersion | distribution of the operation | movement by the embodiment. It is a figure which shows the other example of the function which calculates a skill level from dispersion | distribution of the operation | movement by the embodiment. It is a figure which shows an example of the image which the learning assistance information display part by the embodiment displays.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
In general, it is said that there are the following three stages in learning of motion. A trial and error stage, an intentional adjustment stage, and an automation stage. The trial-and-error stage is a stage in which good results are occasionally obtained and performed well by chance because trial-and-error has been performed on the operation. The stage of deliberate adjustment is the stage of knowing where to pay attention and how to correct it. The stage of automation is a stage where exercise is refined and exercise is possible without consciousness. This includes, for example, the literature “WIKIBOOKS,“ Speech-Language Pathology / Stuttering / Fluency-Shaping Therapy / Motor Learning and Control ”, Internet <http://en.wikibooks.org/wiki/Speech-Language_Pathology/Stuttering/Fluency- Shaping_Therapy / Motor_Learning_and_Control> ”.

  In the trial and error stage described above, since the operation is repeated while searching for an appropriate operation, it is estimated that the variation in the operation becomes large. Also, in the intentional adjustment stage, since the task is conscious and adjusted, the operation varies to some extent although it is smaller than the trial and error stage. And if the operation is refined and automated, the variation in the operation becomes smaller. Based on this knowledge, the proficiency level evaluation apparatus of the present embodiment evaluates the proficiency level based on the variance between trials when the same operation is tried a plurality of times. That is, the proficiency level evaluation apparatus according to the present embodiment estimates the proficiency level of the operation based on the trainer's three-dimensional posture acquired from image data or distance data captured by the camera, and displays the estimated proficiency level in a feedback manner. . At this time, the proficiency level evaluation apparatus of the present embodiment calculates the proficiency level for the target motion as a score from the trainee's own movement indicated by the three-dimensional posture, and supports the acquisition of exercise by feeding back the score. Do.

FIG. 1 is a block diagram showing the configuration of the proficiency level evaluation apparatus according to the present embodiment.
As shown in the figure, the proficiency level evaluation apparatus 100 includes an image input unit 101, a three-dimensional posture acquisition unit 102, a variance calculation unit 103, a proficiency level calculation unit 104, an acquisition support information storage unit 105, an advice generation unit 106, and A learning support information display unit 107 is provided.

  The image input unit 101 receives operation data that is an image obtained by capturing a person's operation from an image input device such as a camera. For example, the image input unit 101 can be configured by a commercially available motion capture system or a multi-viewpoint camera. It is also possible to use a single camera or a distance camera. This operation data includes an image obtained by trying the same operation a plurality of times. The three-dimensional posture acquisition unit 102 generates three-dimensional posture three-dimensional posture data from the motion data acquired by the image input unit 101. The three-dimensional posture data is described as a three-dimensional rotation angle matrix of one or more human body joints.

  The variance calculation unit 103 calculates the variance of the motion in the multiple trials from the three-dimensional trial data of the multiple trials acquired by the three-dimensional posture acquisition unit 102. The variance calculation unit 103 performs this calculation by using general variance. The proficiency level calculation unit 104 calculates the proficiency level of the trainee from the variance calculated by the variance calculation unit 103. The proficiency level is obtained by substituting the variance calculated by the variance calculation unit 103 as a parameter for an arbitrary proficiency level calculation function defined based on the upper and lower limits of variance and the proficiency level at those values. Calculated. The learning support information storage unit 105 stores the variance calculated by the variance calculation unit 103 and the proficiency level calculated by the proficiency level calculation unit 104.

  The advice generation unit 106 generates a support message indicating advice to the trainee using the distribution and the proficiency level recorded in the acquisition support information storage unit 105. The advice generation unit 106 gives advice to the trainee by adding a support message to the text or image. The learning support information display unit 107 outputs the learning support information stored in the learning support information storage unit 105 and the support message generated by the advice generation unit 106 to the image output apparatus. The image output device is, for example, a display or projector that displays an image, but may be a printer that prints an image.

Next, the operation of the proficiency level evaluation apparatus 100 according to this embodiment will be described.
First, the image input unit 101 receives, from the image input device, motion data obtained by photographing the same motion that the target person (trainer) tried a plurality of times. The number of trials is M (M is an integer of 2 or more). The three-dimensional posture acquisition unit 102 generates three-dimensional posture data from the motion data received by the image input unit 101. Even when only one camera is used for the image input device, the three-dimensional posture acquisition unit 102 can predict three-dimensional posture data by the conventional technique.

FIG. 2 is an example of a processing flow of the variance calculation unit 103.
The variance calculation unit 103 divides the three-dimensional posture data obtained by performing the same motion of the target person M times for each trial (step S201). This division may be performed manually, or the variance calculation unit 103 may perform division using a motion estimation technique.

  The variance calculation unit 103 divides the target person's trial m = 1,..., M three-dimensional posture data into N (N is an integer of 1 or more) motion phases (step S202). The variance calculation unit 103 performs phase division based on displacement information and velocity information of an arbitrary joint in the three-dimensional posture data. As an example, a golf swing can generally be divided into three phases: backswing, downswing, and follow-through. In this case, the variance calculation unit 103 can divide the three-dimensional posture data into phases using the displacement information of the position of the wrist joint.

FIG. 3 is a conceptual diagram of phase division by the variance calculation unit 103. FIG. 3A is a conceptual diagram when the state (State) of the person indicated by the three-dimensional posture data of Trial 1 is phase-divided into Phase 1 (Phase 1) to Phase 3 (Phase 3). Show. FIG. 3B shows a concept when the state (State) of the person indicated by the three-dimensional posture data of Trial 2 is phase-divided into Phase 1 (Phase 1) to Phase 3 (Phase 3). The figure is shown. The state (State) is a coordinate position of the joint used in the phase division, and is calculated using the value of the three-dimensional posture data or the value thereof. For example, any coordinate position such as a three-dimensional coordinate position of a wrist or a foot or a y coordinate position can be used. In the case of the golf swing example described above, the variance calculation unit 103 divides the three-dimensional posture data into three phases of back swing, down swing, and follow-through according to the sign change of the differential value of the wrist position displacement indicated by the coordinate position. To do. Each phase n (n = 1, ..., N) , respectively, have a time length (duration) ^{d n.} Thus, a detailed analysis can be performed by dividing each trial into phases. The variance calculation unit 103 repeats the phase division for the three-dimensional posture data of trials m = 1,. The three-dimensional posture data divided into each phase is described as phase motion data.

  In the processing flow of FIG. 2, the variance calculation unit 103 selects the smallest phase that has not yet been selected as the processing target from among the divided phases 1 to N as the processing phase n that is the processing target phase (step S203). ).

The variance calculation unit 103 normalizes the time length of the three-dimensional posture data of trials m = 1,..., M of phase n (step S204).
FIG. 4 is a conceptual diagram of time normalization by the variance calculation unit 103. In general, the duration varies with each trial even in the same phase. A time length ^{d n} in trial processing phase n m (Trial m) and _{d m.} As shown in FIG. 4 (a), when the processing phase n is phase 1 (Phase 1), and _{d 1} is the time length ^{d 1} of the trial 1 (Trial 1), the time length of the trial 2 (Trial 2) It is different from d ₂ which is d ¹ . Therefore, the variance calculation unit 103, before calculating the variance, trial m = 1, ..., align the time length of M d _m a _{(continuation} time width) constant for each phase. For example, the dispersion calculation unit 103 tries m = 1 the processing phase n, ..., based on the time length d ⁿ of M, to determine the time length after normalization processing phase n. As an example, the dispersion calculation unit 103, the processing phase n trials m = 1, ..., those longest among the time length d ⁿ of M, the shortest, or average, etc., time after normalization optionally Decide. Variance calculating unit 103 attempts m = 1 the processing phase n, ..., as the time length d ⁿ of M is the time length after normalization, attempts m = 1, ..., 3-dimensional M Convert time of posture data.

In the processing flow of FIG. 2, the variance calculation unit 103 resamples the time-normalized three-dimensional posture data of the phase n at regular intervals s to obtain resampling posture data (step S205).
FIG. 4B is a conceptual diagram of resampling by the variance calculation unit 103. The variance calculation unit 103 calculates the resampling attitude data by a technique such as linear interpolation. Similar to the three-dimensional posture data, the resampling posture data indicates a three-dimensional rotation angle matrix of the human joint. p _m (m = 1, 2 in the figure) is a division point of the phase of trial m, and t _m is a trajectory formed from the resampled result of trial m.

In the processing flow of FIG. 2, the variance calculation unit 103 calculates variances of the duration V _d , the trajectory variation V _t , and the posture variation V _p of the processing phase n (step S206). First, the dispersion calculation unit 103, a variance _V d (n) duration, trial m = 1 the processing phase n, ..., the duration _{d 1} is the time length of ^{d n} in M each of the three-dimensional orientation data , ..., d Calculate by the variance of _M.

Next, the variance calculation unit 103 calculates the locus variation V _t (n) from the resampling posture data obtained in step S205. Here, as the trajectory, for example, a three-dimensional coordinate value of a predetermined part arbitrarily determined such as a wrist position can be used.
The variance calculation unit 103 reads a three-dimensional coordinate value of a predetermined part from the resampling posture data of each of the trials m = 1,..., M in the processing phase n, and the trajectory t ₁ (i),. Obtain _M (i). i (1 = 1,..., I) is a sample number of resampling in the same trial, and t _m (i) is a predetermined sampling data obtained from the resampling attitude data of the sample number i of trial m. It is a three-dimensional coordinate value of a part. The variance calculation unit 103 calculates the variance of the trajectories t ₁ (i),..., T _M (i) for each of the sample numbers i = 1,. The variation is V _t (n).

It should be noted that K trajectories can be calculated by selecting K arbitrary locations in each phase. When the locus is obtained by selecting K locations, the variance calculation unit 103 calculates the variation V _t (n) of the locus for each of the K locations. The trajectory variation V _t (n) calculated for the k (k = 1,..., K) th place is defined as V _t (n, k).

The variance calculation unit 103 calculates the attitude variation V _p (n) by obtaining the variance of the three-dimensional attitude matrix Y ₁ ,..., Y _M after resampling at the division point of phase n. 3D posture matrix _{Y m (m = 1 ...,} M) , the sample number i = 1 phases n, ..., lined with resampling position data to I, the resampling position data pause _{p n} Matrix.
Note that, when the processing phase n is 1, the variance calculation unit 103 also calculates V _p (N + 1). V _p (N + 1) is the variance of the three-dimensional posture matrix Y ₁ ,..., Y _M at the start of phase 1, and each three-dimensional posture matrix Y _m (m = 1 ..., M) is Resampling posture data at the start of phase 1.

The variance calculating unit 103 calculates the posture variation V _p (n), in addition to using the three-dimensional posture matrix Y ₁ ,..., Y _M as it is, the three-dimensional posture matrix Y ₁ ,. ., Y _M variance may be calculated using only the data of the upper and lower body parts included in _M. In this way, by using only a part of the data, it is possible to calculate the variance for each part such as the variance of the upper body and the variance of the lower body. When L sites are used, L V _p (n) are calculated for each phase n. The posture variation V _p (n) calculated for the l (l = 1,..., L) th place is _defined as a posture variation V _p (n, l).
The variance calculation unit 103 writes the calculated variance V _d (n) of the duration of the processing phase n, locus variation V _t (n), and posture variation V _p (n) in the learning support information storage unit 105.

The variance calculation unit 103 determines whether all of the phases up to phase N have been selected as processing phases (step S207). If the variance calculation unit 103 determines that there is an unselected phase (step S207: YES), the variance calculation unit 103 repeats the processing from step S203.
If the variance calculation unit 103 determines that all of the phases up to phase N have been selected as the processing phase (step S207: NO), the variance calculation unit 103 calculates the variance of the operation in a plurality of trials (step S208). The variance calculation unit 103 calculates the variance V _d (n) of the duration of n = 1,..., N, the variation V _t (n) of the trajectory, and n = 1,. Posture variation V _p (n) is read from the acquisition support information storage unit 105. The variance calculation unit 103 calculates the values of the weighted average V _ave of a plurality of variances such as the variance V _d (n) of the duration calculated above, the locus variation V _t (n), and the posture variation V _p (n). The following equation (1) is used for calculation. The average V _ave is a value that quantitatively represents the dispersion of motion among a plurality of trials.

In Expression (1), w _d , w _t , and w _p are weights of the dispersion of the duration, the variation of the trajectory, and the variation of the posture, and can be arbitrarily set. Except in special cases, it is possible to use 1 for this weight. The variance calculation unit 103 writes the calculated average V _ave in the acquisition support information storage unit 105. It should be noted that the motion variance among a plurality of trials is quantitatively expressed by using a part of the variance V _d (n) of the duration, the trajectory variation V _t (n), and the posture variation V _p (n). A value may be calculated.

The proficiency level calculation unit 104 reads the average V _ave of variance calculated by the variance calculation unit 103 from the acquisition support information storage unit 105 and substitutes it into the function f (V _ave ), so that the proficiency level P is 0 or more and 100 or less. Calculate by value.
5 and 6 are diagrams illustrating examples of the function f (V _ave ) for calculating the proficiency level from the variance of actions. As shown in the figure, the function _{f (V ave)} becomes _{100, V} maximum value max _{(V ave)} at 0 in _ave at the minimum value min of the _{V ave} the system designer has determined _{(V ave),} the slope Is a function of 0 or less. As described above, the function f (V _ave ) is defined by an arbitrary function that passes through min (V _ave ) and max (V _ave ) determined by the system designer, and the proficiency level P increases as the variance _decreases. It is possible. In this way, the proficiency level calculation unit 104 writes the proficiency level P calculated using the function that represents the relationship between the value indicating the variance of the motion and the proficiency level in the acquisition support information storage unit 105.

The advice generation unit 106 reads the values calculated by the variance calculation unit 103 and the proficiency level calculation unit 104 from the learning support information storage unit 105, and generates information indicating practice advice based on these values. For example, it is possible to determine levels such as beginner level, intermediate level, and advanced level according to the value of proficiency level P. As an example, the upper limit and the lower limit of the proficiency level P at each level are determined in advance and stored in the learning support information storage unit 105 (or a storage unit (not shown) provided in the advice generation unit 106). The advice generation unit 106 reads a level corresponding to the proficiency level P calculated by the proficiency level calculation unit 104 from the acquisition support information storage unit 105. Further, the advice generation unit 106 detects the largest point among the variances V _d (n), V _t (n), and V _p (n) calculated by the variance calculation unit 103, and pays attention based on the location. It is possible to generate an urging support message. In addition, for example, a support message indicating advice associated with each of the largest types of variance is stored in advance in the acquisition support information storage unit 105 (or a storage unit (not shown) included in the advice generation unit 106). The advice generation unit 106 acquires a support message corresponding to the largest type of variance among the variances V _d (n), V _t (n), and V _p (n) calculated by the variance calculation unit 103, and acquires the support message information support unit 105. Read from.

The learning support information display unit 107 outputs the results obtained by the variance calculation unit 103, the proficiency level calculation unit 104, and the advice generation unit 106.
FIG. 6 is a diagram illustrating an example of an image that the learning support information display unit 107 displays on the image output apparatus. As shown in the figure, the acquisition support information display unit 107 displays the acquisition support information such as the trainer's three-dimensional posture moving image data, the proficiency score, the beginner / intermediate / advanced level, and the support message on the display. . The trainee's 3D posture moving image data is 3D posture moving image data received by the image input unit 101 from the image input device. The proficiency score is a value of the proficiency level P calculated by the proficiency level calculation unit 104. The levels are beginner / intermediate / advanced levels determined by the advice generation unit 106, and the support message is a support message generated by the advice generation unit 106 based on various variance values calculated by the variance calculation unit 103.

  According to the present embodiment, it is possible to calculate the proficiency level as a score from a plurality of trial operations of the trainee and display intuitive feedback to the trainee. The technical point that produces this effect is that the proficiency level is calculated using variance from multiple trials of the same action.

  You may make it implement | achieve the function of the skill evaluation apparatus 100 in embodiment mentioned above with a computer. In that case, a program for realizing this function may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read into a computer system and executed. Here, the “computer system” includes an OS and hardware such as peripheral devices. The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Furthermore, the “computer-readable recording medium” dynamically holds a program for a short time like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. In this case, a volatile memory inside a computer system serving as a server or a client in that case may be included and a program that holds a program for a certain period of time. The program may be a program for realizing a part of the functions described above, and may be a program capable of realizing the functions described above in combination with a program already recorded in a computer system.

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes designs and the like that do not depart from the gist of the present invention.

  It can be used to support exercise acquisition.

DESCRIPTION OF SYMBOLS 100 ... Proficiency evaluation apparatus, 101 ... Image input part, 102 ... Three-dimensional attitude | position acquisition part, 103 ... Dispersion calculation part, 104 ... Proficiency degree calculation part, 105 ... Acquisition support information storage part, 106 ... Advice generation part, 107 ... Acquisition support information display

Claims (2)

A proficiency evaluation method executed by the proficiency evaluation device,
A variance calculation process for calculating a value that quantitatively represents the variance of the motion between a plurality of trials based on three-dimensional posture data indicating the three-dimensional posture of the trainee when the trainee has tried the motion a plurality of times;
A proficiency level calculation process for calculating a proficiency level corresponding to the value indicating the variance of the behavior calculated in the variance calculation process based on the relationship between the value level indicating the variance of the motion and the proficiency level;
Have
The variance calculation process includes:
A phase division process of dividing the three-dimensional posture data of each trial into phases of motion based on a change in position of a predetermined part obtained from the three-dimensional posture data ;
For each phase, a time length normalization process for normalizing the duration of the phase in the three-dimensional posture data of a plurality of trials;
For each phase, a resampling process of resampling the three-dimensional posture data normalized by the time length normalization process at predetermined intervals;
Dispersion of duration for each phase in the three-dimensional posture data of a plurality of trials, variation in trajectory for each phase of a predetermined portion obtained from the three-dimensional posture data resampled by the resampling process, and phase An inter-trial variance calculation process for calculating a pose variation obtained from a posture matrix obtained by re-sampling the three-dimensional posture data resampled every time and a value that quantitatively represents a variance of motions among a plurality of trials from one or more. And having
A proficiency evaluation method characterized by that. Computer
A variance calculating unit that calculates a value that quantitatively represents the variance of the motion between a plurality of trials based on three-dimensional posture data indicating the three-dimensional posture of the trainer when the trainee has tried the motion a plurality of times;
A proficiency level calculating unit that calculates a proficiency level corresponding to the value representing the variance of the behavior calculated by the variance calculating unit, based on a relationship between the value indicating the variance level of the motion and the proficiency level;
Function as
The variance calculation unit
A phase division process that divides the three-dimensional posture data of each trial into phases of motion based on a change in position of a predetermined part obtained from the three-dimensional posture data ;
For each phase, a time length normalization process for normalizing the duration of the phase in the three-dimensional posture data of a plurality of trials;
A resampling process for resampling the 3D posture data normalized by the time length normalization process at predetermined intervals for each phase;
Dispersion of duration for each phase in the three-dimensional posture data of a plurality of trials, variation in trajectory for each phase of a predetermined part obtained from the three-dimensional posture data resampled by the resampling process, and phase Inter-trial variance calculation processing that calculates a value that quantitatively represents the variance of the motion between one or more trials from one or more of the posture variations obtained by arranging the three-dimensional posture data resampled every time. A program that performs

Images