JP2024016487A - Training movement counting program, training movement counting device, and training movement counting system - Google Patents

Abstract

An object of the present invention is to provide a high degree of freedom in photographing training motions and achieve high usability. A training motion counting device 10 of the present invention stores, in a storage device 15, a region in a body skeleton whose positional relationship with a predetermined joint changes with respect to a predetermined joint during a training motion. In advance, the arithmetic processing unit 16 acquires positional information indicating the positions of the joint Pj and the upper part P from the body skeletal information of the video taken of the training motion, and based on this positional information, determines the position of the upper part Pj with respect to the joint Pj. The vertical position H of P is detected, the vertical position H is compared with the upper determination position Sa, and the effectiveness of the training motion is determined based on the obtained comparison result. [Selection diagram] Figure 1

Description

The present invention relates to a training motion counting technique that analyzes videos of training motions and counts the number of effective motions.

When a person personally performs training movements for the purpose of maintaining or promoting health, it is necessary to easily and objectively evaluate the training movements. Conventionally, as a technology for evaluating such training movements, body skeletal information is extracted by analyzing videos taken of training movements, and changes in body posture detected from the obtained skeletal information are used for predetermined judgments. Training motion counting technology has been proposed that evaluates the effectiveness of training motions based on whether conditions are satisfied and automatically counts the number of motions that are recognized to be effective (for example, Patent Document (See 1 etc.)

Japanese Patent Application Publication No. 2020-195573

In the conventional technology described above, the angle of joint movement in a characteristic posture that is repeated in the training movement is set in advance as a criterion for determining the effectiveness of the training movement, and skeletal information obtained by analyzing the video is used. The angle of the target joint is obtained from the above and compared with the judgment conditions. For example, in the case of a squat motion, if the knee angle when bending the knee obtained from the video satisfies the judgment angle, it is considered to be a squatting posture, which is one of the important postures for determining whether or not counting is appropriate. It will be judged.

However, the angle of the target joint tends to change depending on the position of the camera that photographs the training motion. Therefore, in order to accurately determine whether or not the posture is valid, there is a problem in that it is necessary to precisely adjust the orientation of the camera and the body.

For example, in the case of a posture in which the knee angle is 60 degrees, the angle of 60 degrees can be correctly detected if the camera is positioned directly beside the knees. This is because the plane in which the three parts for specifying the knee angle are present is photographed by the camera from a direction perpendicular to the plane. However, if the camera position deviates from the lateral direction to the front-rear diagonal direction, the knee angle of 60 degrees is detected as an angle larger than 60 degrees. This is because due to the positional shift of the camera, the plane in which the three parts for specifying the knee angle are present will be photographed from the front and back diagonal directions.

Therefore, conventionally, in order to accurately determine whether a posture is valid or not, it is necessary to accurately adjust the camera position and body orientation according to the training content, that is, the joint motion to be determined. For this reason, the conventional technology has a poor degree of freedom in photographing training motions, and cannot provide high usability as a technique for counting training motions.

The present invention is intended to solve such problems, and aims to provide a training motion counting technique that has a large degree of freedom in photographing training motions and can provide high usability.

In order to achieve such an object, the training motion counting program according to the present invention uses a predetermined joint as a reference and identifies parts of the body skeleton whose positional relationship with the predetermined joint changes during a training motion. A processing device that counts the number of times the training motion is performed based on a change in the positional relationship between the predetermined joint and the body part acquired from body skeletal information of a video of the training motion. A training motion counting program used in a training motion counting device, the program comprising: a position information acquisition step of acquiring, in the arithmetic processing device, position information indicating the positions of the predetermined joint and the body part; a motion determination step of detecting a vertical position of the part with respect to a predetermined joint, comparing the vertical position with a preset determination position, and determining the effectiveness of the training motion based on the obtained comparison result; and a counting step of counting the number of training movements for which effectiveness has been recognized.

Further, in one configuration example of the training motion counting program according to the present invention, in the motion determination step, the arithmetic processing device detects a vertical position of the part with respect to the joint based on the position information, and determines the vertical position. The method is configured to include a step of comparing with a preset upper determination position and a lower determination position, and determining the effectiveness of the training motion based on the obtained comparison result.

Further, the training motion counting device according to the present invention includes a storage device that stores, with respect to a predetermined joint, a part in a body skeleton whose positional relationship with the predetermined joint changes during a training motion; an arithmetic processing device that counts the number of training movements based on a change in the positional relationship between the predetermined joint and the body part obtained from body skeletal information in a video shot of the a position information acquisition unit that acquires position information indicating the position of the predetermined joint and the part from skeletal information; and a position information acquisition unit that detects the vertical position of the part with respect to the predetermined joint based on the position information, and determines the vertical position in advance. a motion determining section that compares the training motion with a set determination position and determines the effectiveness of the training motion based on the obtained comparison result; and a counting section that counts the number of times the training motion has been found to be effective. It is configured to be equipped.

Further, a training motion counting system according to the present invention includes the above-described training motion counting device and a camera terminal that transmits an image of a training motion to the training motion counting device via a communication line, The counting device is configured to count the number of training movements based on the video received from the camera terminal.

According to the present invention, when a video of a training motion is analyzed to count the number of effective motions, the degree of freedom regarding the shooting of the training motion is large, and high usability can be obtained.

FIG. 1 is a block diagram showing the configuration of a training motion counting device according to a first embodiment. FIG. 2 is an explanatory diagram showing an example of the structure of part data. FIG. 3 is an explanatory diagram showing skeleton information. FIG. 4 is an explanatory diagram showing a configuration example of determination data according to the first embodiment. FIG. 5 is an explanatory diagram showing the relationship (side view) between the vertical position and determination data used in the squat motion according to the first embodiment. FIG. 6 is an explanatory diagram showing detection of a reference vertical position from a reference posture. FIG. 7 is a flowchart showing the operation of the training motion counting device according to the first embodiment. FIG. 8 is an explanatory diagram showing the relationship (front view) between the vertical position and the determination data used in the squat motion. FIG. 9 is an explanatory diagram showing the relationship (side view) between the vertical position and the determination data used in the push-up motion. FIG. 10 is an explanatory diagram showing the relationship (side view) between the vertical position and the determination data used in the abdominal muscle movement. FIG. 11 is an explanatory diagram showing a configuration example of determination data according to the second embodiment. FIG. 12 is an explanatory diagram showing the relationship (side view) between the vertical position and determination data used in the squat motion according to the second embodiment. FIG. 13 is a flowchart showing the operation of the training motion counting device according to the second embodiment.

Next, embodiments of the present invention will be described with reference to the drawings.
[First embodiment]
First, with reference to FIG. 1, a training motion counting device according to a first embodiment of the present invention will be described. FIG. 1 is a block diagram showing the configuration of a training motion counting device according to a first embodiment.

This training motion counting device 10 is made up of a computer such as a PC or a server, and is configured to automatically count the number of effective training motions by analyzing video footage of training motions.

[Principle of the present invention]
Generally, when a certain joint is moved during a training motion, the bones connected to that joint rotate with respect to the joint, so the positional relationship between the joint and the part of that bone located on the opposite side of the joint changes significantly. Here, if we limit the parts to the upper part that is mainly located above the joint during rotation, the change in the positional relationship between these joints and the upper part will be the vertical position of the upper part with respect to the joint, that is, the main part. can be detected as a change in vertical position that shows a positive value. Furthermore, since this vertical position can be specified if skeletal information is obtained, the degree of change in the vertical position during joint movement is not easily affected by changes in the camera position or body orientation.

The present invention focuses on the characteristics of the vertical position of the upper part with respect to the joint, detects the joint and the position of the upper part with respect to the joint from the video of the training motion, and detects the vertical position of the upper part with respect to the joint. Then, the effectiveness of the training motion is determined based on the comparison result between the obtained vertical position and a preset upper determination position, and the number of training motions that are found to be effective is automatically counted. It is composed of

The training motion counting device 10 mainly includes a communication I/F 11, an operation input device 12, a screen display device 13, a camera 14, a storage device 15, and an arithmetic processing device 16.

[Communication I/F]
The communication I/F 11 connects to a camera terminal 20 such as a smart phone via a communication line LN via a communication network NW such as the Internet, and performs various data communications to exchange various data such as videos of training movements. is configured to do so. In this embodiment, the training motion counting device 10 receives a video of a training motion captured by the camera terminal 20 via the communication I/F 11, and counts the number of training motions based on the obtained video. A case will be explained as an example.

A system including such a training motion counting device 10 and a camera terminal 20 is referred to as a training motion counting system. In the training motion counting system, it is also possible to transmit count results such as the number of motions obtained by the training motion counting device 10 to the camera terminal 20.

[Operation input device]
The operation input device 12 is composed of an operation input device such as a keyboard, a mouse, a touch key, etc., and has a function of detecting an operation by an operator or a user and outputting the detected operation to the arithmetic processing unit 16.

[Screen display device]
The screen display device 13 is composed of a screen display device such as an LCD, and is configured to display various screens outputted from the arithmetic processing device 16, such as a menu screen, a setting screen, and an output screen.

[camera]
The camera 14 is a photographing device using an image sensor, and is configured to photograph an image and output it to the arithmetic processing device 16. The camera 14 may be a camera pre-installed in the training motion counting device 10 such as a notebook PC, or an external camera such as a web camera may be connected to the training motion counting device 10 via a wired or wireless interface. It may also be used as

In the present embodiment, an example will be described in which the training motion counting device 10 acquires a video of a training motion captured by the camera terminal 20 via the communication I/F 11.

[Storage device]
The storage device 15 is composed of a storage device such as a hard disk or a semiconductor memory, and is configured to store processing data used for various calculation processes executed by the calculation processing unit 16 and programs 15P such as a training motion count program.

The program 15P realizes various processing units that execute training motion effectiveness determination processing, motion counting processing, etc. by cooperating with the CPU of the arithmetic processing unit 16. The program 15P is read in advance from an external device or a recording medium (both not shown) connected via the communication line LN, and is stored in the storage device 15.

The main processing data stored in the storage device 15 includes video data 15A, body part data 15B, and determination data 15C.
[Video data]
The video data 15A is video data showing a video of a training motion taken by the camera terminal 20.

[Part data]
Part data 15B is setting data indicating joints and upper parts used to determine the effectiveness of training motions. FIG. 2 is an explanatory diagram showing an example of the structure of part data. In the example of FIG. 2, for each training movement, there is a joint Pj that is the center of the movement, and it is connected to this joint Pj via a bone and rotates around the joint Pj. An upper region P located above is registered.

For example, in the case of a squat motion, the "knee", which is one of the joints that moves the most during the squat motion, is set as the joint Pj. Further, the "hip" connected to the "knee" via the femur is set as the upper part P.

FIG. 3 is an explanatory diagram showing skeleton information. Here, skeletal information indicating the squatting posture of a squat motion is shown. The skeletal information includes coordinate positions of main joints and line segments connecting each joint. This skeletal information is obtained using known methods such as human region estimation technology SSD (Single Shot Multibox Detector), skeletal extraction technology CPN (Cascaded Pyramid Network), and a sequential prediction process that combines Confidence Map and Part Affinity Fields (PAFs). Can be extracted from video. The upper part P registered in Fig. 2 is generally a joint included in the skeletal information, but it is a point set in the middle or on an extension of a line segment connecting each joint (thick line in Fig. 3). There may be.

[Judgment data]
The determination data 15C is setting data for determining the effectiveness of the training motion based on the vertical position H of the upper part P with respect to the joint Pj. FIG. 4 is an explanatory diagram showing a configuration example of determination data according to the first embodiment.

In the example of FIG. 4, an upper determination position Sa is registered for each training motion. For example, in the case of a squat motion, a value of "30 cm" is set in the upper determination position Sa. This upper determination position Sa indicates a position 30 cm vertically away from the joint Pj, that is, the reference vertical position S0. Note that conversion from pixels to length may be obtained using a known method.

A training motion can be considered as a motion that repeatedly creates a characteristic posture, and can only be called an effective training motion when this characteristic posture and other postures are sequentially confirmed.

In the example of FIG. 4, it is assumed that the training motion is composed of the characteristic posture and other postures, and when the vertical position H is less than or equal to the upper determination position Sa, it is determined that the characteristic posture has been detected, and the vertical position H is larger than the upper determination position Sa, it is determined that a posture other than the characteristic posture has been detected. The value of the upper determination position Sa may be determined based on the results obtained by detecting vertical positions H in characteristic postures from a plurality of people in advance and statistically processing them.

[Computational processing unit]
The arithmetic processing unit 16 includes a CPU and its peripheral circuits, and includes various processing units that read out the program 15P from the storage device 15 and execute training motion effectiveness determination processing, motion counting processing, etc. by cooperating with the CPU. Realize. The main processing units implemented by the arithmetic processing unit 16 include a video acquisition unit 16A, a position information acquisition unit 16B, a motion determination unit 16C, and a motion count unit 16D.

[Video acquisition section]
The video acquisition unit 16A performs data communication with the camera terminal 20 from the communication I/F 11 via the communication network NW, thereby acquiring the video of the training motion taken by the camera terminal 20, and stores the video data 15A in the storage device 15. It is configured to be saved as .

[Location information acquisition unit]
The position information acquisition unit 16B analyzes the video data 15A to extract body skeletal information, and from the obtained skeletal information, determines the position of the joint Pj and upper part P corresponding to the target training movement among the part data 15B. It is configured to acquire information (coordinate information).

[Operation determination section]
The motion determination unit 16C detects the vertical distance from the reference vertical position S0 of the joint Pj to the upper part P, that is, the vertical position H of the upper part P with respect to the joint Pj, based on the position information acquired by the position information acquisition part 16B. is configured to do so. FIG. 5 is an explanatory diagram showing the relationship (side view) between the vertical position and determination data used in the squat motion according to the first embodiment. In the case of a squat motion, the joint Pj corresponds to the knee, and the upper region P corresponds to the lower back.

In the standing posture (a posture other than the characteristic posture) in FIG. 5(a), the positional relationship between the joint Pj and the upper region P is far apart in the vertical direction, and In the posture), the positional relationship between the joint Pj and the upper part P is closer in the vertical direction than in the standing posture. Therefore, the vertical position H in FIG. 5(b) is lower (smaller) than the vertical position H in FIG. 5(a).

Further, the motion determining unit 16C compares the detected vertical position H with the upper determination position Sa of the determination data 15C, and determines whether or not it is a characteristic posture based on the comparison result, thereby distinguishing between the characteristic posture and other postures. The apparatus is configured to determine the effectiveness of a training motion comprised of postures.

When the determination data 15C is set as shown in FIG. 4 described above, for the squat motion shown in FIG. 5, if the vertical position H is higher than 30 cm (upper determination position Sa), the squat action shown in FIG. It is determined that a standing posture (a posture other than the characteristic posture) has been detected.

Furthermore, if the vertical position H is 30 cm or less (upper determination position Sa), it is determined that the crouching posture (characteristic posture) shown in FIG. 5(b) has been detected. These validity determination processes may be performed continuously or intermittently for each frame forming the video data 15A.

The vertical position H detected by the motion determining unit 16C varies to some extent depending on the person, and may cause an error in determining effectiveness. In order to suppress the influence of such variations, the user's height is input from the camera terminal 20 or the operation input device 12 before the start of the counting operation, and the motion determination unit 16C calculates the joint Pj and upper part P from the height. The reference vertical position S0 may be obtained by calculating the positional information, and the upper judgment position Sa may be estimated based on the obtained reference vertical position S0 and set in the judgment data 15C.

Further, regarding the reference vertical position S0, a reference posture such as an upright posture may be photographed in advance and included in the video data 15A, and the reference vertical position S0 may be extracted from the reference posture. FIG. 6 is an explanatory diagram showing detection of a reference vertical position from a reference posture.

As shown in FIG. 6, the position information acquisition section 16B acquires the position information of the joint Pj and the upper region P from the skeletal information of the reference posture consisting of the upright posture, and the motion determination section 16C determines the reference vertical position from this position information. The position S0 may be detected, and the upper judgment position Sa may be estimated based on the obtained reference vertical position S0 and set in the judgment data 15C.

[Operation count section]
When the motion determining section 16C detects postures other than the characteristic posture and the characteristic posture alternately, the motion counting section 16D determines that the series of motions constituting the training motion has been performed correctly, and counts up the number of motions. is configured to do so. Further, the operation counting unit 16D may display the obtained count result on the screen display device 13, and may notify the camera terminal 20 from the communication I/F 11 via the communication line LN, and It may also be displayed on the screen.

[Operation of the first embodiment]
Next, with reference to FIG. 7, the operation of the training motion counting device 10 according to this embodiment will be described. FIG. 7 is a flowchart showing the operation of the training motion counting device according to the first embodiment. Here, it is assumed that the video data 15A and the part data 15B are stored in the storage device 15 in advance before starting the counting operation.

Note that, before starting the counting operation, the determination data 15C may be set according to the user. In this case, as described above, the motion determination unit 16C obtains the user's height input from the camera terminal 20 or the operation input device 12 before starting the counting operation, and estimates the upper determination position Sa from the height. What is necessary is to set it in the judgment data 15C. Alternatively, as described above, the upper determination position Sa may be estimated based on the reference vertical position S0 detected from the reference orientation included in the video data 15A, and set in the determination data 15C.

In the counting operation, the position information acquisition unit 16B first analyzes any one frame of the video data 15A to extract body skeletal information, and from the obtained skeletal information, selects a target training motion from the part data 15B. (position information acquisition step) (step 100).

Subsequently, the motion determination unit 16C detects the vertical position H of the upper region P with respect to the joint Pj based on the position information acquired by the position information acquisition unit 16B (step 101).

After that, the motion determining unit 16C compares the detected vertical position H with the upper determination position Sa of the determination data 15C, and determines whether the posture is other than the characteristic posture based on the comparison result (step 102). Here, if the vertical position H is less than or equal to the upper determination position Sa (step 102: NO), the process returns to step 100 and processes regarding the subsequent frame are executed.

In step 102, if the vertical position H is higher than the upper determination position Sa (step 102: YES), the position information acquisition unit 16B determines that a posture other than the characteristic posture has been detected, and detects the subsequent characteristic posture. The subsequent frames of the video data 15A are analyzed to extract body skeletal information, and from the obtained skeletal information, the position information (coordinates information) (step 103).

Subsequently, the motion determination unit 16C detects the vertical position H of the upper region P with respect to the joint Pj based on the position information acquired by the position information acquisition unit 16B (step 104).

After that, the motion determination unit 16C compares the vertical position H with the upper determination position Sa of the determination data 15C, and determines whether or not it is a characteristic posture based on the comparison result (motion determination step) (step 105). Here, if the vertical position H is higher than the upper determination position Sa (step 105: NO), the process returns to step 103 and processes regarding the subsequent frame are executed.

In step 105, if the vertical position H is less than or equal to the upper determination position Sa (step 105: YES), the motion counting section 16D detects that the motion determining section 16C has alternately detected a posture other than the characteristic posture and a characteristic posture. , it is determined that the series of movements constituting the training movement has been performed correctly, and the number of movements is counted up (counting step) (step 106).

After this, the operation determination unit 16C determines whether to end the counting operation (step 107). At this time, if the number of operations reaches a predetermined number or an operation to stop the counting operation is performed, it may be determined that the counting operation has ended.

In step 107, if the counting operation is to be continued (step 107: NO), the process returns to step 100 and processes regarding subsequent frames are executed.

On the other hand, when the counting operation is to be completed (step 107: YES), the operation determination unit 16C outputs the counting result to the screen display device 13 and the camera terminal 20 (step 108), and the series of processes is completed.

In this way, the effectiveness of the training motion is determined based on the comparison result between the vertical position H of the upper part P with respect to the joint Pj and the determination data 15C. At this time, if skeletal information is obtained, the positions of the joint Pj and the upper part P can be specified, so the degree of change in the vertical position H during joint movement is independent of the influence of changes in camera position and body orientation. I know it's hard to accept.

FIG. 8 is an explanatory diagram showing the relationship (front view) between the vertical position and the determination data used in the squat motion. In the above-mentioned FIG. 5, the case where the training motion is photographed from the side of the person is explained as an example, but as shown in FIG. Can be detected.

In addition, although the squat motion has been described above as an example, other training motions can be performed in the same manner. FIG. 9 is an explanatory diagram showing the relationship (side view) between the vertical position and the determination data used in the push-up motion. FIG. 10 is an explanatory diagram showing the relationship (side view) between the vertical position and the determination data used in the abdominal muscle movement. It can be seen that in either case, the vertical position H of the upper region P with respect to the joint Pj can be easily detected.

In addition, for these as well, if skeletal information is obtained, the positions of the joints Pj and the upper part P can be identified, so the degree of change in the vertical position H during joint movement can be determined even if the camera position or body orientation changes. It can be seen that it is not easily affected by this.

[Effects of the first embodiment]
In this way, in the present embodiment, the storage device 15 rotates with respect to a predetermined joint Pj, and also stores the upper part P that is mainly located above the joint Pj at the time of rotation, and performs calculations. The processing device 16 extracts body skeletal information from the video of the training motion, acquires positional information indicating the positions of the joint Pj and the upper part P from this skeletal information, and based on this positional information, determines the upper part of the joint Pj. The vertical position H of the part P is detected, the vertical position H is compared with the upper determination position Sa, and the effectiveness of the training motion is determined based on the obtained comparison result.

As a result, the effectiveness of the training motion is determined based on the vertical position H of the upper region P with respect to the joint Pj, so that it is less affected by changes in the camera position or body orientation. Therefore, compared to the case where the effectiveness of a training motion is determined based on joint angles, there is no need to strictly adjust the camera position or body orientation. Therefore, the degree of freedom regarding photographing training motions is large, and high usability can be obtained.

[Second embodiment]
Next, a second embodiment of the present invention will be described. In the first embodiment, on the premise that the upper part P is mainly located above the joint Pj when the joint rotates, it is determined whether or not it is a characteristic posture by comparing the vertical position H with the upper determination position Sa. We have explained the case of determining whether In this embodiment, in addition to the characteristic postures similar to those of the first embodiment, a case will be described in which a downward posture in which the upper part P is located below the joint Pj is detected.

[Judgment data]
FIG. 11 is an explanatory diagram showing a configuration example of determination data according to the second embodiment. In the example of FIG. 11, an upper determination position Sa and a lower determination position Sb are registered for each training motion. For example, in the case of a squat motion, a determination value of "30 cm" is set at the upper determination position Sa, and a determination range of "20 cm" is set at the lower determination position Sb.

The upper determination position Sa "30 cm" indicates a position 30 cm away from the joint Pj, that is, the reference vertical position S0 in the upward direction as a vertical distance. Further, the lower judgment position Sb "20 cm" indicates a position 20 cm away from the joint Pj, that is, the reference vertical position S0 in the downward direction as a vertical distance.

[Operation determination section]
The motion determination unit 16C compares the detected vertical position H with the upper determination position Sa and the lower determination position Sb of the determination data 15C, and detects the characteristic posture and the downward posture based on the comparison result, thereby determining the characteristic posture. The apparatus is configured to determine the effectiveness of training movements composed of other postures.

FIG. 12 is an explanatory diagram showing the relationship (side view) between the vertical position and determination data used in the squat motion according to the second embodiment. When the determination data 15C is set as in the example of FIG. 11 described above, for the squat motion shown in FIG. 12, if the vertical position H is higher than 30 cm above (upper determination position Sa), the squat motion shown in FIG. It is determined that a standing posture (a posture other than the characteristic posture) has been detected.

Furthermore, if the vertical position H is in the range from 30 cm above (upper judgment position Sa) to 20 cm below (lower judgment position Sb) across the joint Pj, that is, the reference vertical position S0, the squatting posture of FIG. 12(b) (characteristic posture) is detected. Further, when the vertical position H is lower than 20 cm downward (lower determination position Sb), it is determined that the sitting posture shown in FIG. 12(c) has been detected.

For example, as shown in FIG. 12, in the case of a squat motion, some users lower their hips further from the characteristic posture of the squatting posture shown in FIG. 12(b) to take a sitting posture as shown in FIG. may be taken. This sitting posture is an incomplete downward posture that occurs, for example, when the muscles become fatigued during repeated squat movements and the lower back can no longer support the body. Therefore, in order to determine that a motion including such a downward posture is not a regular squat motion, a downward determination position Sb may be provided and compared with the vertical position H, as in the present embodiment.
Note that the other configurations according to this embodiment are the same as those in the first embodiment, and description thereof will be omitted here.

[Operation of second embodiment]
Next, with reference to FIG. 13, the operation of the training motion counting device 10 according to this embodiment will be described. FIG. 13 is a flowchart showing the operation of the training motion counting device according to the second embodiment. Here, it is assumed that the video data 15A and the part data 15B are stored in the storage device 15 in advance before starting the counting operation.

Note that, before starting the counting operation, the determination data 15C may be set according to the user. In this case, as described above, the motion determination unit 16C obtains the height of the user input from the camera terminal 20 or the operation input device 12 before starting the counting operation, and uses the height to determine the upper determination position Sa and the lower determination position Sa. The position Sb may be estimated and set in the determination data 15C. Alternatively, as described above, the upper determination position Sa and the lower determination position Sb may be estimated based on the reference vertical position S0 detected from the reference orientation included in the video data 15A, and set in the determination data 15C.

In the counting operation, the position information acquisition unit 16B first analyzes any one frame of the video data 15A to extract body skeletal information, and from the obtained skeletal information, selects a target training motion from the part data 15B. The positional information (coordinate information) of the joint Pj and upper part P corresponding to is acquired (step 200).

Subsequently, the motion determination unit 16C detects the vertical position H of the upper region P with respect to the joint Pj based on the position information acquired by the position information acquisition unit 16B (step 201).

After that, the motion determination unit 16C compares the detected vertical position H with the upper determination position Sa of the determination data 15C, and determines whether the posture is other than the characteristic posture based on the comparison result (step 202). . Here, if the vertical position H is less than or equal to the upper determination position Sa (step 202: NO), the process returns to step 200 and processes regarding the subsequent frame are executed.

In step 202, if the vertical position H is higher than the upper determination position Sa (step 202: YES), the position information acquisition unit 16B determines that a posture other than the characteristic posture has been detected, and detects the subsequent characteristic posture. The subsequent frames of the video data 15A are analyzed to extract body skeletal information, and from the obtained skeletal information, the position information (coordinates information) (step 203).

Subsequently, the motion determination unit 16C detects the vertical position H of the upper region P with respect to the joint Pj based on the position information acquired by the position information acquisition unit 16B (step 204).

Thereafter, the motion determination unit 16C compares the detected vertical position H with the upper determination position Sa of the determination data 15C, and determines whether or not it is a characteristic posture based on the comparison result (step 205). Here, if the vertical position H is higher than the upper determination position Sa (step 205: NO), the process returns to step 203 and processes regarding the subsequent frame are executed.

In step 205, if the vertical position H is below the upper judgment position Sa (step 205: YES), the operation counting section 16D compares the vertical position H with the lower judgment position Sb of the judgment data 15C, and based on the comparison result. It is determined whether or not the robot is in a downward posture (step 206). Here, if the vertical position H is lower than the downward determination position Sb (step 206: YES), since a downward posture has been detected, the process returns to step 200 and processes regarding the subsequent frame are executed.

On the other hand, in step 206, if the vertical position H is equal to or higher than the lower determination position Sb (step 206: NO), the motion determination unit 16C detects a posture other than the characteristic posture and a characteristic posture alternately, so the training motion is not performed. It is determined that the series of operations have been performed correctly, and the number of operations is counted up (step 207).

After this, the operation determination unit 16C determines whether to end the counting operation (step 208). At this time, if the number of operations reaches a predetermined number or an operation to stop the counting operation is performed, it may be determined that the counting operation has ended.

In step 208, if the counting operation is to be continued (step 208: NO), the process returns to step 200 and processes regarding subsequent frames are executed.

On the other hand, when the counting operation is to be completed (step 208: YES), the operation determination unit 16C outputs the counting result to the screen display device 13 and the camera terminal 20 (step 209), and ends the series of processing.

In this way, the effectiveness of the training motion is determined based on the comparison result between the vertical position H of the upper part P with respect to the joint Pj, the upper determination position Sa, and the lower determination position Sb. Thereby, in addition to postures other than characteristic postures and characteristic postures, it is possible to detect an incomplete downward posture in which the upper region P is located below the joint Pj.

Furthermore, as in the first embodiment, the positions of the joint Pj and the upper part P can be specified if skeletal information is obtained, so the degree of change in the vertical position H during joint movement depends on the camera position and body orientation. It can be seen that it is not easily affected by changes in the

Further, in the above description, the squat motion was used as an example, but other training motions can be performed in the same manner. As shown in FIGS. 9 and 10 described above, the vertical position H of the upper part P with respect to the joint Pj can be easily detected even in the case of a push-up motion or a sit-up motion. In addition, for these as well, if skeletal information is obtained, the positions of the joints Pj and the upper part P can be identified, so the degree of change in the vertical position H during joint movement can be determined even if the camera position or body orientation changes. It can be seen that it is not easily affected by this.

[Effects of the second embodiment]
In this way, in the present embodiment, when determining the effectiveness of a training motion, the arithmetic processing unit 16 detects the vertical position H of the upper part P with respect to the joint Pj based on the position information of the joint Pj and the upper part P. However, the vertical position H is compared with a preset upper determination position Sa and a lower determination position Sb, and the effectiveness of the training motion is determined based on the obtained comparison results.

As a result, in addition to postures other than the characteristic postures and characteristic postures, it is possible to detect incomplete downward postures in which the upper part P is located below the joint Pj, so the number of training movements can be counted more accurately. be able to.

[Expansion of embodiment]
Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above embodiments. The configuration and details of the present invention may be modified in various ways within the scope of the present invention by those skilled in the art. Moreover, each embodiment can be implemented in any combination within the range not contradictory.

10...Training action counting device, 11...Communication I/F, 12...Operation input device, 13...Screen display device, 14...Camera, 15...Storage device, 15A...Video data, 15B...Part data, 15C...Judgment data, 16... Arithmetic processing unit, 16A... Video acquisition section, 16B... Position information acquisition section, 16C... Motion determination section, 16D... Motion counting section, 20... Camera terminal, LN... Communication line, NW... Communication network, Pj... Joint, P...Upper part, H...Vertical position, Sa...Upper determination position, Sb...Lower determination position, S0...Reference vertical position.

Claims (4)

A storage device that stores parts of the body skeleton whose positional relationship with the predetermined joints changes during training motions, with respect to the predetermined joints as a reference, and body skeletal information obtained from video footage of the training motions. A training motion counting program used in a training motion counting device comprising an arithmetic processing device that counts the number of training motions based on a change in the positional relationship between the predetermined joint and the body part,
The arithmetic processing device,
a position information acquisition step of acquiring position information indicating the positions of the predetermined joint and the part;
Detecting the vertical position of the part with respect to the predetermined joint based on the position information, comparing the vertical position with a preset determination position, and evaluating the effectiveness of the training motion based on the obtained comparison result. a motion determination step for determining;
A counting step of counting the number of training movements for which effectiveness has been recognized. A training movement counting program. In the motion determination step, the arithmetic processing device detects a vertical position of the part with respect to the joint based on the position information, and compares the vertical position with a preset upper determination position and a lower determination position, The training motion counting program according to claim 1, further comprising the step of determining the effectiveness of the training motion based on the obtained comparison result. A storage device that stores parts of the body skeleton whose positional relationship with the predetermined joints changes during training motions, with respect to the predetermined joints as a reference, and body skeletal information obtained from video footage of the training motions. an arithmetic processing device that counts the number of training movements based on a change in the positional relationship between the predetermined joint and the body part;
The arithmetic processing device is
a position information acquisition unit that acquires position information indicating the position of the predetermined joint and the part from the skeletal information;
Detecting the vertical position of the part with respect to the predetermined joint based on the position information, comparing the vertical position with a preset determination position, and evaluating the effectiveness of the training motion based on the obtained comparison result. a motion determination unit that determines;
A counting unit that counts the number of training movements for which the effectiveness of the training movement has been recognized. A training motion counting device according to claim 3;
and a camera terminal that transmits images of training movements to the training movement counting device via a communication line,
The training motion counting system is characterized in that the training motion counting device counts the number of training motions based on the video received from the camera terminal.

Images