JP2020175047A - Operation analyzing device, operation analyzing method and program - Google Patents

Abstract

To easily perform comparison of images at a specific operation timing.SOLUTION: An operation analyzing device 1 comprises: extracts an image (frame image) G1 at a specific operation timing periodically appearing from a video obtained by continuously shooting a user who is running, extracts an image (frame image) G2 at the specific operation timing from a prescribed model video that is continuously shot when extracting the image at the specific operation timing, and displays the images G1, G2 on the display unit 15 in a comparable manner.SELECTED DRAWING: Figure 12

Description

The present invention relates to a motion analysis device, a motion analysis method and a program.

Conventionally, while the practitioner performs physical movements related to skill (for example, golf) (for example, golf swing), the movement of oneself (practitioner) and the movement of the normative technician are compared and examined, and the movement of the normative technician is compared. A physical skill acquisition support device that allows oneself to confirm whether or not it can be accurately reproduced is disclosed (see, for example, Patent Document 1).

Japanese Unexamined Patent Publication No. 2011-152333

However, in the physical skill acquisition support device disclosed in Patent Document 1, when an attempt is made to compare and examine movements at a specific movement timing, the specific movement information and skill information are used based on the operation of the practitioner. There is a problem that it is troublesome because each operation timing image must be specified.

The present invention has been made in view of such a problem, and an object of the present invention is to easily compare images at a specific operation timing.

In order to solve the above problems, the motion analysis device according to the present invention
A first extraction means for extracting a frame image of a specific operation timing that appears periodically from a continuously captured image of a user performing an exercise consisting of repeated periodic operations, and a first extraction means.
When the frame image of the specific operation timing is extracted by the first extraction means, the second extraction means for extracting the frame image of the specific operation timing from the predetermined model video continuously shot. ,
A display control means for causing the display means to display the frame images extracted by the first extraction means and the second extraction means in a manner comparable to each other.
It is characterized by having.

According to the present invention, it is possible to easily compare images at a specific operation timing.

It is a block diagram which shows the schematic structure of the motion analysis apparatus. It is a figure which shows the state which the user wears the sensor device 2. It is a flowchart which shows the flow of a measurement process. It is a flowchart which shows the flow of the landing timing setting process. It is a graph which shows by exemplifying a part of the waveform of the acceleration signal of each of the front-rear direction, the left-right direction, and the up-down direction. It is a flowchart which shows the flow of the takeoff timing setting process. It is a graph which shows by exemplifying a part of the waveform of the acceleration signal in each of the front-rear direction and the up-down direction. Of the graph shown in FIG. 7, a part of the graph between 28.9 seconds and 29.1 seconds is enlarged. It is a graph which shows by exemplifying a part of the waveform of the acceleration signal in each of the front-rear direction and the up-down direction. It is a graph which shows by exemplifying a part of the waveform of the acceleration signal in each of the front-rear direction and the up-down direction. It is a graph which shows by exemplifying a part of the waveform of the acceleration signal in each of the front-rear direction and the up-down direction. It is a flowchart which shows the flow of the motion analysis processing. It is a figure which shows an example of a running form comparison screen.

Hereinafter, embodiments according to the present invention will be described in detail with reference to the accompanying drawings. The present invention is not limited to the illustrated examples.

<< Configuration of motion analysis device 1 >>
The motion analysis device 1 of the present embodiment is a device for analyzing the motion of a user who is continuously taking an image of a user who is performing an exercise consisting of repeating periodic motions. In the following, running will be described as an example of an exercise consisting of repeated periodic movements.

FIG. 1 is a block diagram showing a schematic configuration of an operation analysis device 1 according to an embodiment to which the present invention is applied.
As shown in FIG. 1, the motion analysis device 1 includes a CPU (Central Processing Unit) 11, a RAM (Random Access Memory) 12, a storage unit 13, an operation unit 14, a display unit 15, and a communication unit 16. To be equipped.

The CPU (first extraction means, second extraction means, display control means, comparison means, notification means, correction means) 11 controls each part of the motion analysis device 1. The CPU 11 reads a designated program among the system programs and application programs stored in the storage unit 13 and expands them in the RAM 12, and executes various processes in cooperation with the program.

The RAM 12 is a volatile memory and forms a work area for temporarily storing various data and programs.

The storage unit 13 is composed of, for example, a flash memory, an EEPROM (Electrically Erasable Programmable ROM), an HDD (Hard Disk Drive), or the like. The storage unit 13 stores system programs and application programs executed by the CPU 11, data necessary for executing these programs, and the like.

Further, the storage unit 13 is provided with a video storage unit (not shown) for storing predetermined video data. The predetermined video data is, for example, video data related to a video continuously captured by a user who is running using a treadmill. When the video data is stored in the video storage unit, the acceleration signal data (sensing) acquired from the sensor device 2 (described later) attached to the user at the time of shooting the video of the video data and synchronized with the video. Data) and data of each phase (landing timing, lowest point timing, takeoff timing, highest point timing, etc.) of the cycle in the user's running operation are stored in association with each other. The data of each phase of the cycle in this running motion is the data calculated by executing the measurement process (described later) by the motion analysis device 1.

Further, the storage unit 13 is provided with a model video storage unit (not shown) for storing predetermined model video data. Here, the predetermined model video data is video data related to a model video in which a runner as a model is continuously photographed while running using a treadmill. The predetermined model video data is acquired from the sensor device 2 (described later) attached to the runner as a model at the time of shooting the video of the model video data, and the video is obtained in the same manner as the predetermined video data described above. The acceleration signal data (sensing data) synchronized with the above and the data of each phase of the cycle in the running operation of the runner are associated with each other and stored in the model video storage unit.

Further, the storage unit 13 is provided with a comment table (not shown) that stores comments to be notified as a result of the motion analysis when the motion analysis process (see FIG. 12) described later is executed. In this comment table, for example, "difference between the posture of the runner to be analyzed and the posture of the runner as a model", "comment", etc. for each phase (motion timing) of the cycle in the running motion. Items are provided, and data related to these items are stored in association with each other.

Here, the sensor device 2 will be described with reference to FIG. FIG. 2 is a diagram showing a state in which the sensor device 2 is worn by the user.
As shown in FIG. 2, the sensor device 2 has a main body portion 2A and a belt portion 2B, and the main body portion 2A is fixed at the position of the user's waist by the belt portion 2B. Here, the left-right direction is the X-axis, the front-back direction is the Y-axis, and the up-down direction is the Z-axis. On the X-axis, the left-hand direction is positive and the right-hand direction is negative. On the Y-axis, the opposite direction of travel is positive and the direction of travel is negative. On the Z axis, the upward direction is positive and the downward direction is negative. The sensor device 2 includes an acceleration sensor (not shown), and sequentially records acceleration signal data (sensing data) in the three-axis (X-axis, Y-axis, Z-axis) directions output by the acceleration sensor during running operation. It is possible.

The operation unit 14 is provided with, for example, a touch panel, receives a touch input from a user, and outputs the operation information to the CPU 11.
The touch panel is formed integrally with the display unit 15, and for example, the XY coordinates of the contact position on the display unit 15 by the user can be set by various methods such as a capacitance method, a resistance film method, and an ultrasonic surface acoustic wave method. To detect. Then, the touch panel outputs a position signal related to the XY coordinates of the contact position to the CPU 11.

The display unit (display means) 15 is composed of an LCD (Liquid Crystal Display), an EL (Electro Luminescence) display, or the like, and performs various displays according to display information instructed by the CPU 11.

The communication unit 16 is composed of a modem, a router, a network card, and the like. The communication unit 16 transmits / receives data to / from an external device connected via a communication network.

<< Operation of motion analysis device 1 >>
[Measurement processing]
Next, the measurement process executed by the motion analysis device 1 will be described with reference to FIGS. 3 to 11.

FIG. 3 is a flowchart showing the flow of the measurement process.
5 and 7 to 11 are graphs showing a part of the waveforms of the acceleration signals in the front-rear direction, the left-right direction, and the up-down direction by exemplifying them. In the following description, how the waveform of the acceleration signal is used in the measurement process will be described with reference to FIGS. 5 and 7 to 11. Further, in the present embodiment, the case where the above measurement process is applied only to the portions shown in FIGS. 5 and 7 to 11 will be described as an example, but the measurement process is performed in the front-rear direction, the left-right direction, and the up-down direction. It is executed for the entire waveform of each acceleration signal.

As shown in FIG. 3, when the measurement process is executed, the CPU 11 first performs the landing timing setting process (step S1). Here, the landing timing is one aspect of the cycle in the running motion, and refers to the timing when the running user's foot (one foot) touches the ground. The details of the landing timing setting process will be described later.

Next, the CPU 11 sets the lowest point timing (step S2). Specifically, the CPU 11 sets the timing indicating the minimum value T _min (see FIG. 5) of the height position waveform T, which will be described later, as the lowest point timing. Here, the lowest point timing is one phase of the cycle in the running motion, and is the timing when the running user's foot (one foot) passes the lowest position while touching the ground. Point to.

Next, the CPU 11 performs a takeoff timing setting process (step S3). Here, the takeoff timing is one aspect of the cycle in the running motion, and refers to the timing when the running user's foot (one foot) is off the ground. The details of the takeoff timing setting process will be described later.

Next, the CPU 11 sets the highest point timing (step S4). Specifically, the CPU 11 sets the timing indicating the maximum value T _max (see FIG. 5) of the height position waveform T, which will be described later, as the highest point timing. Here, the highest point timing is one aspect of the cycle in the running motion, and refers to the timing when the running user's foot (one foot) passes the highest position of the waist after being separated from the ground.

Next, the CPU 11 calculates the difference time between the landing timing set by the landing timing setting process and the takeoff timing set by the takeoff timing setting process as the ground contact time (step S5).
As a result, the data (landing timing, lowest point timing, takeoff timing, highest point timing, and ground contact time) of each phase of the cycle in the running operation are calculated, and the CPU 11 ends the measurement process.

Next, the details of the landing timing setting process (step S1) in the above measurement process will be described.
FIG. 4 is a flowchart showing the flow of the landing timing setting process.
FIG. 5 is a graph showing a part of the waveforms of the acceleration signals in the front-rear direction, the left-right direction, and the up-down direction as an example. In the following description, how the waveform of the acceleration signal is used for the landing timing setting process will be described with reference to FIG. Further, in the present embodiment, a case where the landing timing setting process is applied only to the portion shown in FIG. 5 will be described as an example, but the landing timing setting process is performed by accelerating signals in the front-rear direction, the left-right direction, and the up-down direction. It is executed for the entire waveform of.

As shown in FIG. 4, when the landing timing setting process is executed, the CPU 11 first performs a well-known smoothing process such as a moving average on the vertical acceleration signal (second acceleration signal) AccZ1 ( Step S11).
Next, the CPU 11 obtains the maximum value Z _max of the waveform of the vertical acceleration signal AccZ2 after smoothing, and divides the time axis with the timing showing the maximum value Z _max as the reference timing (fourth reference timing) (step). S12). In FIG. 5, the time axis is divided by lines P1, P2, and P3 based on the maximum value Z _max . The division region between the line P1 and the line P2, and the division region between the line P2 and the line P3 are referred to as a first division region R1.

Next, the CPU 11 obtains the height position waveform T representing the height position of the sensor device 2 by integrating the vertical acceleration signal AccZ1 before smoothing twice for each first division region R1 (step S13). ..
Next, the CPU 11 divides the time axis with the timing indicating the maximum value T _max of the height position waveform T in each first division region R1 as the reference timing (first reference timing). In FIG. 5, the time axis is divided by lines P4, P5, and P6 based on the maximum value T _max . The lines P4, P5, and P6 are set at the breaks between steps (for example, the boundary between odd-numbered steps and even-numbered steps) (step S14). The area divided by the lines P4, P5, P6 is referred to as a second divided area. For convenience of explanation, in the following, of the two consecutive second division regions, the earlier one (before in time) is referred to as the "first second division region R21", and the latter (after in time). This is referred to as "later second division region R22".

Next, the CPU 11 determines the maximum value position (line P2,) of the waveform of the acceleration signal AccZ2 in the vertical direction after smoothing from the first half of each of the second division regions R21 and R22 (for example, the step break (lines P4 and P5)). In P3)), the positive maximum value of the waveform of the acceleration signal (first acceleration signal) AccY in the front-rear direction is searched for (step S15).
FIG. 5 illustrates a case where the first second division region R21 has one positive maximum value Y _max and the second second division region R22 has two positive maximum values Y _max .

Next, the CPU 11 determines whether or not there is one positive maximum value Y _{max in} the first half of each of the second division regions R21 and R22, and if it is one, the positive maximum value Y _max. Is specified and the process proceeds to step S18, and if there are two or more, the process proceeds to step S17 (step S16).
Here, at the time of landing, the waveform of the acceleration signal AccY in the front-rear direction has a peak (maximum value) at a positive value because the vehicle decelerates due to the impact of the landing operation. This positive maximum value Y _max is an extreme value indicating deceleration in the traveling direction.
However, multiple similar peaks may occur depending on the running style and speed. In other words, at the time of landing, at least one positive maximum value is generated in the waveform of the acceleration signal AccY in the front-rear direction. Therefore, in step S16, it is determined whether or not there is one positive maximum value. In the present embodiment, since the acceleration signal AccY in the front-rear direction has a positive direction in the opposite direction of travel and a negative direction in the direction of travel, the extreme value indicating deceleration in the waveform of the acceleration signal AccY in the front-rear direction is a positive maximum value. Although it is Y _max , when the positive and negative values are opposite, the extreme value indicating deceleration becomes the negative minimum value.

In step S17, the CPU 11 identifies one from a plurality of positive maximum values Y _max based on the waveform of the acceleration signal AccX in the left-right direction. For example, when landing by running, only one foot lands, so the upper body tilts to the left and right. At this time, the upper body swings in the left-right direction in order to unknowingly adjust the posture. That is, in the waveform of the acceleration signal AccX in the left-right direction, a waveform showing left-right blurring appears at the time of landing. Based on this waveform, it is possible to specify the time of landing.
Specifically, the CPU 11 has three extreme values X _m1 , X _m2 , and X _m3 of the waveform of the acceleration signal AccX in the left-right direction within a predetermined interval in the first half portion of the second division region R22. Further, if the difference between the adjacent extreme values X _m1 , X _m2 , and X _m3 among the three extreme values X _m1 , X _m2 , and X _m3 is equal to or more than a predetermined value, it is recognized as a waveform showing left and right blurring. Then, the CPU 11 specifies a positive maximum value Y _max that is close to the first extreme value X _m1 that occurs among the three extreme values X _m1 , X _m2 , and X _m3 .
For the "predetermined interval" and the "predetermined value", the values obtained by experiments and simulations are used. Specifically, the predetermined interval is preferably a value within the range of 40 to 100 ms, more preferably 70 ms. Further, preferably a value within the range of 5 to 15 m / ^{s 2} as the predetermined value, 10 m / ^{s 2} is more preferable.

Next, the CPU 11 sets the position (time) of the specified positive maximum value Y _max as the landing timing (step S18), and ends the landing timing setting process.

Next, the details of the takeoff timing setting process (step S3) in the above measurement process will be described.
FIG. 6 is a flowchart showing the flow of the takeoff timing setting process.
7, 9 and 10 are graphs showing a part of the waveform of the acceleration signal in the front-rear direction and the up-down direction by exemplifying.
FIG. 8 is an enlarged graph of a part of the graph shown in FIG. 7 between 28.9 seconds and 29.1 seconds.
In the following description, how the waveform of the acceleration signal is used for the takeoff timing setting process will be described with reference to FIGS. 7 to 10. Further, in the present embodiment, a case where the takeoff timing setting process is applied only to the portions shown in FIGS. 7 to 10 will be described as an example, but the takeoff timing setting process is performed in the front-rear direction and the up-down direction, respectively. It is executed for the entire waveform of the acceleration signal.

As shown in FIG. 6, when the takeoff timing setting process is executed, the CPU 11 first performs a smoothing process using a moving average for the acceleration signal (first acceleration signal) AccY in the front-rear direction (step). S21).
Specifically, the CPU 11 is the number of samples when the number of taps is 11 to 15 (here, the sampling frequency is 200 Hz and the sampling cycle is 5 ms) with respect to the acceleration signal AccY in the front-rear direction. Therefore, the triangular moving average (first moving average) of the period (corresponding to the case where the period for calculating the moving average is 25 to 35 ms before and after the target signal (first period)) is calculated, and the triangle is concerned. The first signal (hereinafter referred to as FAccY_2) after the smoothing process by the moving average is obtained, and the number of taps is 3 to 5 (the number of taps is the number of samples when the sampling frequency is 200 Hz, and the moving average is calculated. After calculating the triangular moving average (second moving average) of the period (corresponding to the case where the period is 5 to 10 ms before and after the target signal (second period)) and smoothing with the triangular moving average. Second signal (hereinafter referred to as FAccY_1) is obtained. Here, the triangular moving average has the effect of reducing the noise of the target signal, and is a well-known method. In this smoothing process using the triangular moving average, the larger the number of taps (number of samples), the greater the effect of smoothing the signal.

Next, the CPU 11 assigns the variable n to the sampling timing for each sampling cycle (for example, 5 ms) in sequence according to the passage of time, as a second division region (between lines P4 and P5 in FIG. 5 described above). Alternatively, in the latter half of the region (the region between the lines L1 and L2 in FIG. 7) which is set in the same manner as the second division region between the lines P5 and P6, FAccY_2 changes from negative to positive, that is, from acceleration to deceleration. Search for the timing (plus zero cross timing; n = ZC) that changes to (step S22). Then, the CPU 11 sets the above-mentioned timing (ZC) searched and acquired as the second reference timing. In FIGS. 7 to 10, the second reference timing is indicated by the line _LZC .
Here, when searching for plus-zero cross timing (ZC), as shown in the area circled by the broken line in FIG. 8, the value of AccY changes from negative to positive depending on noise and body movement even before takeoff. The timing of sudden changes may occur. Then, in FAccY_1 having a relatively small number of taps, this steep change remains, and similarly, the steep change changes from negative to positive. On the other hand, as shown in FIG. 8, in FAccY_2 having a relatively large number of taps, even when AccY or FAccY_1 changes from negative to positively steep, this steep change is alleviated and becomes negative. The timing of the positive change will not appear. Therefore, in the present embodiment, the plus zero cross timing (ZC) can be appropriately searched by using FAccY_2 in which the number of taps is relatively large. The number of taps is when the sampling frequency is 200 Hz and the sampling period is 5 ms, and the number of taps changes depending on the sampling frequency of the acceleration signal. Further, the number of samplings and the sampling period are not limited to the above values, and may be in the vicinity of the above values.

Next, the CPU 11 searches for a timing (LMX) at which FAccY_2 reaches a maximum value in the latter half of the second division area (area divided by lines L1 and L2 in FIG. 7) (step S23). Then, the CPU 11 sets the above timing (LMX) as the third reference timing. In FIGS. 7 to 10, the third reference timing is indicated by the line L _LMX .

Next, the CPU 11 first sets the sampling timing (n =) one sampling cycle after the second reference timing in the search period divided between the second reference timing and the third reference timing, that is, the lines L _ZC and L _LMX. Calculate the difference (FAccY_1 (n) -FAccY_1 (n-1)) between the value of FAccY_1 for each sampling cycle between ZC + 1, step S24) and the third reference timing (LMX) with respect to the value of FAccY_1 one sampling cycle before. Then, it is determined whether or not the value of the difference is less than the predetermined threshold value TH (step S25). Here, in the threshold processing in step S25, for example, when the maximum value appears at once after the second reference timing due to the influence of noise or the peculiar movement of the body, or after the waveform in which the inflection point is seen in the middle. The purpose is to enable accurate identification of takeoff timing when a maximum value appears. Further, in the threshold value processing, FAccY_1 is used in order to suppress the influence of noise contained in the AccY data. Here, the threshold value TH is set to a value close to 0 m / s ² (for example, 0.5 m / s ² ). The threshold value TH may be set stepwise according to the maximum value of FAccY_1. For example, if the maximum value of FAccY_1 is less than 5 m / s ² , it is set to 0.5 m / s ^2, and if the maximum value is 5 m / s ² or more, it is set to 1.0 m / s ² .

When it is determined in step S25 that the difference is not less than the threshold value TH (step S25; NO), the CPU 11 increments the variable n by 1 (step S26), and the sampling timing indicated by the variable n at this time is the third. It is determined whether or not the timing is the reference timing (LMX) (step S27).
In step S27, if it is determined that the sampling timing indicated by the variable n at this time is not the third reference timing (LMX) (step S27; NO), the process returns to step S25, and the CPU 11 repeats the processes after step S25. ..
On the other hand, in step S27, when it is determined that the sampling timing indicated by the variable n at this time is the third reference timing (LMX) (step S27; YES), that is, the second reference timing (ZC) and the third reference. When the condition that the difference is less than the threshold value TH is not satisfied with the timing (LMX), the CPU 11 sets the third reference timing (LMX) as the takeoff timing as shown in FIG. (Step S28), the takeoff timing setting process is completed.

Further, when it is determined in step S25 that the difference is less than the threshold value TH (step S25; YES), the CPU 11 determines that the value of the acceleration signal AccZ in the vertical direction at the sampling timing indicated by the variable n at this time is a predetermined value. It is determined whether or not the value is less than C1 and the value of FAccY_1 is equal to or more than the specific value C2 (step S29).
In the determination process of step S29, whether or not the AccZ value is less than the predetermined value C1 is determined because the AccZ value is on the positive side, that is, upward while the foot is placed on the ground. , AccZ is greater than or equal to the predetermined value C1 (see, for example, the area circled by the broken line in FIG. 9), because it is assumed that the foot is still in contact with the ground. The predetermined value C1 is slightly larger than the gravitational acceleration 9.80665 m / ^{s 2} (e.g., 12m / ^{s 2)} to.
Further, in the determination process of step S29, whether or not the value of FAccY_1 is equal to or higher than the specific value C2 is determined when the value of FAccY_1 is very small with respect to the maximum value of FAccY_1 (broken line in FIG. 10). (Refer to the area circled in) is because even if the rate of increase of AccY is 0 at that timing, it is assumed that the foot is not yet off the ground, and it is relatively large at the takeoff timing. This is because it is assumed that a change (decrease) in speed will occur. Here, the specific value C2 is a predetermined coefficient (for example, for example) with respect to the maximum value of the value of FAccY_1 in the range from the second reference timing (ZC) to the rear end of the second division region (see, for example, line L2 in FIG. 7). It is a value multiplied by 40 to 60%).

If it is determined in step S29 that the above determination conditions are not satisfied (step S29; NO), the process proceeds to step S26, and the CPU 11 performs subsequent processing.
On the other hand, when it is determined in step S29 that the above determination condition is satisfied (step S29; YES), the CPU 11 uses the sampling timing (see FIGS. 8 and 10) indicated by the variable n at this time as the takeoff timing. The setting (step S30) is performed, and the takeoff timing setting process is completed.

As described above, the CPU 11 performs the landing timing setting process (step S1), the lowest point timing setting process (step S2), the takeoff timing setting process (step S3), and the highest point timing setting process (step S4). By executing, for example, as shown in FIG. 11, when one foot touches the ground and touches the ground (landing timing), and when the foot passes the lowest position while touching the ground (bottom). Point timing), when the foot takes off from the ground (takeoff timing), and when the foot passes the highest position after leaving the ground (highest point timing), and When the reverse foot touches the ground (landing timing), and when the reverse foot passes the lowest position while the foot touches the ground (lowest point timing), the foot separates from the ground. It is possible to detect (extract) the time of reverse foot takeoff (takeoff timing) and the time of the highest point of reverse foot (highest point timing) when the foot passes the highest position after leaving the ground. ..

[Motion analysis processing]
Next, the motion analysis process executed by the motion analysis device 1 will be described with reference to FIG. FIG. 12 is a flowchart showing the flow of the motion analysis process.

As shown in FIG. 12, first, the CPU 11 determines whether or not the video data desired by the user has been specified via the operation unit 14 (step S31).

If it is determined in step S31 that the video data desired by the user has not been specified (step S31; NO), the CPU 11 performs the determination process in step S31 until the video data desired by the user is specified. Repeat.
On the other hand, when it is determined in step S31 that the user-desired video data has been designated (step S31; YES), the CPU 11 acquires the designated video data from the video storage unit (not shown) of the storage unit 13. (Step S32).

Next, the CPU 11 acquires the model video data from the model video storage unit (not shown) of the storage unit 13 (step S33).

Next, the CPU 11 corrects the size of the subject of the image based on the image data acquired in step S32 to obtain the size of the subject of the image and the subject of the model image of the model image data acquired in step S33. Performs video size correction processing to match the size (step S34). In the image size correction process, for example, by correcting the size of the subject of the model image based on the model image data acquired in step S33, the size of the subject of the above image and the subject of the model image are corrected. You may try to match the size. Further, the size of the subject of the image and the size of the subject of the model image may be matched by correcting each other's size of the subject of the image and the model image.

Next, the CPU 11 determines whether or not a specific operation timing has been specified via the operation unit 14 (step S35). Here, the specific operation timing is a timing (phase) that appears periodically during running, and specifically, when the left foot touches the ground, which is detected by the above-mentioned measurement process. (Landing timing), at the lowest point when the foot passes the lowest position while touching the ground (lowest point timing), when the foot is off the ground (takeoff timing), and this foot When the highest point (highest point timing) when the foot passes the highest position after leaving the ground, and when the right foot touches the ground with the reverse foot (landing timing), the waist is the highest while the foot touches the ground. At the lowest point of the reverse foot that passed the low position (lowest point timing), at the time of the reverse foot taking off from the ground (takeoff timing), and after this foot is off the ground, the waist is the highest. It is the time of the highest point of the reverse foot (highest point timing) that has passed the high position.

If it is determined in step S35 that the specific operation timing has not been specified (step S35; NO), the CPU 11 repeats the determination process in step S35 until the specific operation timing is specified. ..
On the other hand, when it is determined in step S35 that a specific operation timing has been specified (step S35; YES), the CPU 11 acquires an image (frame image) of the specified specific operation timing in step S32. Extract from the video of the data (step S36). For example, when the landing timing by the left foot is specified as the specific operation timing, the CPU 11 detects the landing timing by the left foot from the acceleration signal data stored in association with the video data acquired in step S32. The timing is specified, and the image (frame image) taken at this timing is extracted from the video of the video data.

Next, the CPU 11 extracts an image (frame image) of the specified specific operation timing from the model video of the model video data acquired in step S33 (step S37). For example, when the landing timing by the left foot is specified as the specific operation timing, the CPU 11 detects it as the landing timing by the left foot from the acceleration signal data stored in association with the model video data acquired in step S33. The timing is specified, and the image (frame image) taken at this timing is extracted from the model video of the model video data.

Next, the CPU 11 includes an image G1 of a specific operation timing (see FIG. 13) extracted in step S36 (or step S41), an image G2 of a specific operation timing extracted in step S37 (see FIG. 13), and The motion analysis is performed based on (step S38). Specifically, the CPU 11 extracts the joint points of the respective subjects from the images G1 and G2, and estimates the posture of each subject from the positions of the extracted joint points. Then, the CPU 11 derives the difference in the posture of the subject of the image G1 with respect to the posture of the subject of the image G2.

Next, the CPU 11 displays the running form comparison screen 20 on the display unit 15 as a result of the operation analysis in step S38 (step S39).

FIG. 13 is a diagram showing an example of the running form comparison screen 20.
As shown in FIG. 13, on the running form comparison screen 20, the image G1 of the specific operation timing (landing timing by the left foot) extracted in step S36 (or step S41) and the specific operation timing extracted in step S37. The image G2 (landing timing by the left foot) is displayed side by side. Further, in the comment column C below each of the images G1 and G2, a comment corresponding to the difference in the posture of the subject of the image G1 with respect to the posture of the subject of the image G2, which is the result of the motion analysis in step S38 (for example, " The center of gravity of the landing timing is the back center of gravity. Try to keep the center of gravity directly above your left foot. ") Is displayed.

Next, the CPU 11 determines whether or not an instruction to continue the motion analysis process has been given via the operation unit 14 (step S40).

When it is determined in step S40 that the instruction to continue the motion analysis process has been given (step S40; YES), the CPU 11 has acquired another image (frame image) of the specified specific motion timing in step S32. Extracting from the video of the video data (step S41). Then, the CPU 11 returns the process to step S38, and repeats the subsequent processes.
On the other hand, if it is determined in step S40 that the instruction to continue the motion analysis process has not been given (step S40; NO), whether or not the CPU 11 has been instructed to end the motion analysis process via the operation unit 14. (Step S42).

If it is determined in step S42 that the instruction to end the motion analysis process has not been made (step S42; NO), the CPU 11 returns the process to step S40 and repeats the subsequent processes.
On the other hand, when it is determined in step S42 that the instruction to end the motion analysis process has been given (step S42; YES), the CPU 11 stores the images G1 and G2 and the comments displayed on the running form comparison screen 20 in the storage unit 13. (Step S43), and the operation analysis process is completed.

As described above, according to the motion analysis device 1, an image (frame image) G1 of a specific motion timing that appears periodically is extracted from a continuously captured image of a running user, and the specific motion analysis device 1 is used. When an image of the operation timing is extracted, an image (frame image) G2 of the specific operation timing is extracted from a predetermined model image continuously shot, and the respective images G1 and G2 can be compared with each other. Since the image is displayed on the display unit 15, the images G1 and G2 can be easily compared at a specific operation timing.

Further, according to the motion analysis device 1, if there is acceleration signal data acquired from the sensor device 2 worn by the user at the time of shooting the image and synchronized with the image, it appears periodically from the acceleration signal data. Since the image G1 captured at the specific operation timing is extracted from the video, the image G1 at the specific operation timing can be accurately extracted.

Further, according to the motion analysis device 1, the images G1 and G2 displayed on the running form comparison screen 20 are compared, and the comparison result is displayed (notified) as a comment on the running form comparison screen 20, so that the motion analysis It is possible to easily understand the difference between the target runner and the model runner.

Further, according to the motion analysis device 1, by correcting the size of the subject of the image, the size of the subject of the image and the size of the subject of the model image are matched, and the size of the subject of the image and the subject of the model image are matched. Since the image G1 at a specific operation timing is extracted from the video after the size is adjusted to the above, it is possible to easily compare the images G1 and G2 on the running form comparison screen 20.

Although the present invention has been specifically described above based on the embodiments, the present invention is not limited to the above embodiments and can be modified without departing from the gist thereof.
For example, in the above embodiment, running has been described as an example of an exercise consisting of repeated periodic movements, but the exercise is not limited to running, and the exercise is not limited to running, but walking, swimming, skiing, skating, and biking. It may be targeted for exercise such as.

Further, in the above embodiment, the motion analysis device 1 extracts the images G1 and G2 of a specific motion timing. For example, when the image G1 is extracted in step S36 of the motion analysis process, the image G1 is extracted. A predetermined number of images taken before and after are extracted, and when the image G2 is extracted in step S37, a predetermined number of images taken before and after the image G2 are extracted, and the running form comparison screen 20 displays. These images may be displayed in a manner comparable to each other.

Further, in the above embodiment, for example, it corresponds to the acceleration signal data associated with the video data stored in the video storage unit of the storage unit 13 and the model video data stored in the model video storage unit. When comparing the attached acceleration signal data with the data at a specific operation timing, if there is a significant difference in each data, the image at this operation timing is the image of the video data and the model video data. Each of the extracted images may be extracted from the model image and displayed on the display unit 15 in a comparable manner.

Further, in the above embodiment, when the acceleration signal data is not associated with the video data for which the motion analysis is performed by the motion analysis device 1, the posture of the subject is estimated for each image (frame image) of the video of the video data. By performing the processing, the image at a specific operation timing may be extracted.

Further, in the above embodiment, the motion analysis device 1 displays the running form comparison screen 20 on the display unit 15. For example, the running form comparison screen 20 is displayed on the display unit (not shown) of an external device. You may let it.

In the above description, an example in which a flash memory, EEPROM, and HDD are used in the storage unit 13 as a computer-readable medium for the program according to the present invention has been disclosed, but the present invention is not limited to this example. A portable recording medium can be applied as another computer-readable medium. A carrier wave is also applied to the present invention as a medium for providing data of a program according to the present invention via a communication line.

Although the embodiments of the present invention have been described, the scope of the present invention is not limited to the above-described embodiments, but includes the scope of the invention described in the claims and the equivalent scope thereof.
The inventions described in the claims originally attached to the application of this application are added below. The claim numbers mentioned in the appendix are the scope of the claims originally attached to the application for this application.
[Additional Notes]
<Claim 1>
A first extraction means for extracting a frame image of a specific operation timing that appears periodically from a continuously captured image of a user performing an exercise consisting of repeated periodic operations, and a first extraction means.
When the frame image of the specific operation timing is extracted by the first extraction means, the second extraction means for extracting the frame image of the specific operation timing from the predetermined model video continuously shot. ,
A display control means for causing the display means to display the frame images extracted by the first extraction means and the second extraction means in a manner comparable to each other.
A motion analysis device characterized by being equipped with.
<Claim 2>
The first extraction means is acquired from a motion sensor attached to the user at the time of shooting the image, and when there is sensing data synchronized with the image, the specificity that periodically appears from the sensing data. The operation timing of the above is extracted, and the frame image taken at the specific operation timing is extracted from the video.
The motion analysis apparatus according to claim 1.
<Claim 3>
Each of the first extraction means and the second extraction means extracts a predetermined number of frame images taken before and after the frame image in addition to the frame image of the specific operation timing.
The display control means causes the display means to display each frame image extracted by the first extraction means and the second extraction means in a manner comparable to each other.
The motion analysis apparatus according to claim 1 or 2.
<Claim 4>
A comparison means for comparing the frame image of the video displayed on the display means with the frame image of the model video, and
A notification means for notifying the result of comparison by the comparison means, and
The motion analysis apparatus according to any one of claims 1 to 3, further comprising.
<Claim 5>
A correction means for matching the size of the subject of the image with the size of the subject of the model image by correcting the size of at least one of the subject of the image and the subject of the model image is provided.
The first extraction means extracts a frame image of the specific operation timing from the image after the size of the subject of the image and the size of the subject of the model image are matched by the correction means.
The motion analysis device according to any one of claims 1 to 4, wherein the motion analysis device is characterized by the above.
<Claim 6>
The first extraction step of extracting a frame image of a specific operation timing that appears periodically from a continuously captured image of a user who is performing an exercise consisting of repeating periodic operations, and
When the frame image of the specific operation timing is extracted by the first extraction step, the second extraction step of extracting the frame image of the specific operation timing from the predetermined model video continuously shot. ,
A display control step of displaying the frame images extracted by the first extraction step and the second extraction step on the display means in a manner in which they can be compared with each other.
A motion analysis method characterized by including.
<Claim 7>
Computer,
A first extraction means for extracting a frame image of a specific operation timing that appears periodically from a continuously captured image of a user who is performing an exercise consisting of repeating periodic operations.
When the frame image of the specific operation timing is extracted by the first extraction means, the second extraction means for extracting the frame image of the specific operation timing from a predetermined model image continuously captured.
A display control means for causing a display means to display each frame image extracted by the first extraction means and the second extraction means in a manner comparable to each other.
A program characterized by functioning as.

1 Motion analysis device 11 CPU (first extraction means, second extraction means, display control means, comparison means, notification means, correction means)
15 Display unit (display means)

Claims (7)

A first extraction means for extracting a frame image of a specific operation timing that appears periodically from a continuously captured image of a user performing an exercise consisting of repeated periodic operations, and a first extraction means.
When the frame image of the specific operation timing is extracted by the first extraction means, the second extraction means for extracting the frame image of the specific operation timing from the predetermined model video continuously shot. ,
A display control means for causing the display means to display the frame images extracted by the first extraction means and the second extraction means in a manner comparable to each other.
A motion analysis device characterized by being equipped with. The first extraction means is acquired from a motion sensor attached to the user at the time of shooting the image, and when there is sensing data synchronized with the image, the specificity that periodically appears from the sensing data. The operation timing of the above is extracted, and the frame image taken at the specific operation timing is extracted from the video.
The motion analysis apparatus according to claim 1. Each of the first extraction means and the second extraction means extracts a predetermined number of frame images taken before and after the frame image in addition to the frame image of the specific operation timing.
The display control means causes the display means to display each frame image extracted by the first extraction means and the second extraction means in a manner comparable to each other.
The motion analysis apparatus according to claim 1 or 2. A comparison means for comparing the frame image of the video displayed on the display means with the frame image of the model video, and
A notification means for notifying the result of comparison by the comparison means, and
The motion analysis apparatus according to any one of claims 1 to 3, further comprising. A correction means for matching the size of the subject of the image with the size of the subject of the model image by correcting the size of at least one of the subject of the image and the subject of the model image is provided.
The first extraction means extracts a frame image of the specific operation timing from the image after the size of the subject of the image and the size of the subject of the model image are matched by the correction means.
The motion analysis device according to any one of claims 1 to 4, wherein the motion analysis device is characterized by the above. The first extraction step of extracting a frame image of a specific operation timing that appears periodically from a continuously captured image of a user who is performing an exercise consisting of repeating periodic operations, and
When the frame image of the specific operation timing is extracted by the first extraction step, the second extraction step of extracting the frame image of the specific operation timing from the predetermined model video continuously shot. ,
A display control step of displaying the frame images extracted by the first extraction step and the second extraction step on the display means in a manner in which they can be compared with each other.
A motion analysis method characterized by including. Computer,
A first extraction means, which extracts a frame image of a specific operation timing that appears periodically from a continuously captured image of a user who is performing an exercise consisting of repeating periodic operations.
When the frame image of the specific operation timing is extracted by the first extraction means, the second extraction means for extracting the frame image of the specific operation timing from a predetermined model image continuously captured.
A display control means for causing a display means to display each frame image extracted by the first extraction means and the second extraction means in a manner comparable to each other.
A program characterized by functioning as.

Images