KR102488560B1 - Apparatus and method for sarcopenia diagnosis and training - Google Patents

Abstract

Devices and methods for diagnosing and training sarcopenia are disclosed. An apparatus for diagnosing and training sarcopenia according to an embodiment includes an input unit for obtaining an input image obtained by capturing a test subject; a posture estimation unit extracting a skeleton of the test subject from the input image and estimating a posture of the test subject; a performance evaluation unit that classifies the performed motion of the test subject through motion recognition based on the extracted skeleton and evaluates the level of the performed motion; a level-by-level training determining unit for determining a training program to be performed by the test subject based on the level of the performing motion; And classifying the training motion of the test subject through action recognition based on a skeleton extracted from an input image of a test subject performing training according to the training program, and providing feedback to the test subject based on the accuracy of the training motion. Includes a training coaching unit that provides.

Description

Sarcopenia diagnosis and training apparatus and method {APPARATUS AND METHOD FOR SARCOPENIA DIAGNOSIS AND TRAINING}

The disclosed embodiments relate to a method for diagnosing and training sarcopenia and a system using the same.

Sarcopenia refers to a condition in which the muscles of the body are abnormally reduced or weakened, resulting in poor physical activity. More than 20% of the elderly population in Korea is confirmed to have sarcopenia, and recently, sarcopenia has been internationally recognized as a disease and a 'disease code' has been assigned. In order to effectively treat sarcopenia and minimize side effects of sarcopenia, early diagnosis of sarcopenia and effective treatment based thereon are very important. This patent relates to an image-based early diagnosis of sarcopenia and an evaluation of the level of sarcopenia, thereby enabling effective treatment for sarcopenia according to the degree of sarcopenia.

Korean Patent Registration No. 10-1582347 (2016.01.04. Notice) Korean Patent Registration No. 10-1782247 (Announced on September 26, 2017) Korean Patent Registration No. 10-1884418 (2018.06.12. Notice)

The present invention is a method for diagnosing and treating sarcopenia with only image input without any other additional tools, presenting appropriate exercises according to the level of the diagnosed sarcopenia and providing feedback thereto to enable effective diagnosis and treatment of sarcopenia, and It's about the device.

An apparatus for diagnosing and training sarcopenia according to an embodiment includes an input unit for obtaining an input image obtained by capturing a test subject; a posture estimation unit extracting a skeleton of the test subject from the input image and estimating a posture of the test subject; a performance evaluation unit that classifies the performed motion of the test subject through motion recognition based on the extracted skeleton and evaluates the level of the performed motion; a level-by-level training determining unit for determining a training program to be performed by the test subject based on the level of the performing motion; And classifying the training motion of the test subject through action recognition based on a skeleton extracted from an input image of a test subject performing training according to the training program, and providing feedback to the test subject based on the accuracy of the training motion. Includes a training coaching unit that provides.

The performance evaluation unit may determine a level of the performance motion based on a result of at least one of a plurality of tests and sarcopenia level evaluation.

In the plurality of tests, the change in the shape of a finger in a state in which the test subject holds the elastic body and does not apply force and the state in which the test subject applies maximum force to the elastic body changes in the shape of the finger and the elastic force of the elastic body Based on the test subject may include a grip strength test for testing the grip force applied to the elastic body.

The performance evaluation unit may evaluate the accuracy and level of the performed motion by comparing the performed motion with a preset reference posture.

The training coaching unit may compare the training motion with a preset reference training motion to evaluate the accuracy of the training motion.

The performance evaluation unit calculates the angle formed by each joint from the connection relationship between the movement of each joint included in the skeleton and the neighboring joint in the performing motion, and calculates the angle of each joint in the reference posture. Accuracy of the performed motion may be determined based on the difference between angles of the joints.

The posture estimation unit extracts the outline of the test subject along with the skeleton of the test subject from the input image, and the performance evaluation unit performs the operation based on matching accuracy between the contour of the subject's performed motion and the contour of the reference posture. Calibration can be performed for the accuracy of the operation.

The training coaching unit may provide the feedback so that the test subject performs a training motion matching a preset reference training motion.

A method for diagnosing and training sarcopenia according to an embodiment includes obtaining a first input image obtained by capturing a test subject; estimating a posture of the test subject by extracting a skeleton of the test subject from the first input image; classifying a performed motion of the test subject through motion recognition based on the extracted skeleton, and evaluating a level of the performed motion; determining a training program to be performed by the test subject based on the level of the performing motion; obtaining a second input image of the test subject performing training according to the training program; extracting the skeleton of the test subject from the second input image; classifying a training motion of the test subject through motion recognition based on a skeleton extracted from the second input image; and providing feedback to the test subject based on the accuracy of the training operation.

In the evaluating of the level of the performing motion, the level of the performing motion may be determined based on a result of at least one of a plurality of tests and sarcopenia level evaluation.

In the plurality of tests, the change in the shape of a finger in a state in which the test subject holds the elastic body and does not apply force and the state in which the test subject applies maximum force to the elastic body changes in the shape of the finger and the elastic force of the elastic body Based on the test subject may include a grip strength test for testing the grip force applied to the elastic body.

In the step of evaluating the level of the performed motion, the accuracy and level of the performed motion may be evaluated by comparing the performed motion with a preset reference posture.

In the providing of the feedback, accuracy of the training motion may be evaluated by comparing the training motion with a preset reference training motion.

In the step of evaluating the level of the performed motion, an angle formed by each joint is calculated from a connection relationship between the movement of each joint included in the skeleton and a neighboring joint in the performed motion, and the calculated angle and Accuracy of the performed motion may be determined based on a difference between angles of each joint in the reference posture.

The step of estimating the posture includes extracting the outline of the test subject along with the skeleton of the test subject from the input image, and the step of evaluating the level of the performed motion includes the comparison between the outline of the subject's performed motion and the reference posture. Correction of the accuracy of the performed operation may be performed based on matching accuracy between contour lines.

In the providing of the feedback, the feedback may be provided so that the test subject performs a training operation matching a preset reference training operation.

According to the disclosed embodiments, the presence or absence of sarcopenia is diagnosed based on an image, and the level of sarcopenia is evaluated if it is determined to be sarcopenia. Training suitable for each stage according to the presence and level of sarcopenia is performed, effective training is performed through feedback during training, and effective treatment for sarcopenia is provided through continuous re-evaluation to provide personalized, step-by-step training programs. make it happen

1 is a configuration diagram of a sarcopenia diagnosis and training device according to an embodiment
2 is a diagram showing an example of a process of classifying the level of the performed motion through the evaluation of the level of the performed motion;
3 is a diagram illustrating an example of user feedback according to an embodiment;

Hereinafter, specific embodiments will be described with reference to the drawings. The detailed descriptions that follow are provided to provide a comprehensive understanding of the methods, devices and/or systems described herein. However, this is only an example and is not limited thereto.

In describing the embodiments, if it is determined that a detailed description of related known technologies may unnecessarily obscure the subject matter of the invention, the detailed description will be omitted. In addition, terms to be described later are terms defined in consideration of functions, which may vary according to intentions or customs of users and operators. Therefore, the definition should be made based on the contents throughout this specification. Terminology used in the detailed description is only for describing the embodiments and should in no way be limiting. Unless expressly used otherwise, singular forms of expression include plural forms. Also, expressions such as “comprising” or “comprising” are intended to indicate certain characteristics, numbers, steps, operations, elements, parts or combinations thereof, and one or more other characteristics other than those described. , should not be construed to exclude the existence or possibility of any number, step, action, element, any part or combination thereof.

1 is a configuration diagram of an apparatus for diagnosing and training sarcopenia according to an embodiment.

Referring to FIG. 1 , the apparatus for diagnosing and training sarcopenia 100 includes an input unit 110, a posture estimation unit 120, a performance evaluation unit 130, a level-by-level training determination unit 140, and a training coaching unit 150. include

The input unit 110 obtains an input image obtained by continuously photographing a subject to be examined.

In this case, the image may be generated using, for example, an imaging means such as a camera, and may be an input image obtained by photographing the entire body or a part of the body of the test subject. Also, the input image may be a color image, but may be a black and white image or a depth image according to embodiments.

Meanwhile, according to an embodiment, the input unit 110 may obtain motion data measured using a motion measurement sensor capable of measuring a motion of an examination subject, such as, for example, a gyro sensor, in addition to an input image. there is. At this time, the motion measurement sensor is, for example, a smartphone (smartphone), a smart watch (smart watch), a smart band (smart band), such as possessed or wearable mobile device (mobile device) or wearable device to be provided can

The posture estimation unit 120 extracts the skeleton of the examination subject from the input image and estimates the posture of the examination subject.

Specifically, the posture estimator 120 may extract the skeleton of the test subject from the input image using a model based on a pre-learned artificial neural network. In this case, the artificial neural network-based model may be, for example, a model combining a Convolutional Neural Network (CNN) and Short-Term Memory (LSTM).

Meanwhile, when the input image is a 2D image, the posture estimator 120 may extract a 2D skeleton from the input image and convert the extracted 2D skeleton into a 3D skeleton according to an embodiment. there is. At this time, in order to transform the 2D skeleton into a 3D skeleton, the posture estimator 120 may use position or manifold based constraints and a separate conversion CNN. However, when the input image is a 3D image such as a depth image, the posture estimator 120 may directly extract a 3D skeleton from the input image.

Meanwhile, according to an additional embodiment, the posture estimator 120 may extract the silhouette of the test subject together with the skeleton of the test subject from the input image. At this time, the extracted contour may be used to determine the accuracy of an operation to be described later.

Specifically, when the input image is a 2D image, the posture estimator 120 may extract the contour of the test subject through foreground extraction based on background modeling. In addition, when the input image is a 3D image such as a depth image, the posture estimator 120 may extract an outline of the examination subject from a foreground image extracted through background separation from the input image.

The performance evaluation unit 130 classifies the motion of the test subject into a motion to be performed for evaluation of the motion level and a non-performed general motion through motion recognition based on the skeleton of the test subject extracted from the input image. Also, the performance evaluation unit 130 compares the classified performed motion with a preset reference posture to evaluate the accuracy and level of the performed motion. Based on the difference in joint angle, the accuracy and level of the performed motion are evaluated.

Specifically, the performance evaluation unit 130 may calculate the angle formed by each joint from the connection relationship between the movement of each joint included in the skeleton and the neighboring joints in the classified performing motions. Also, the performance evaluation unit 130 may calculate posture accuracy for the performed motion based on an angle difference between the calculated angle of each joint and the angle of each joint in the reference posture. At this time, the performance evaluation unit 130 may calculate the angular difference based on the Mahalanobis distance considering the range and change value (covariance) of the angle that varies for each joint in order to calculate the posture accuracy, and the angular characteristics. The distance between the angles can be measured using a quaternion to reflect . In addition, since there may be relatively important joints and unimportant joints to take the reference posture, the performance evaluation unit 130 may measure the distance between angles by multiplying weights according to the importance of each joint preset according to the reference posture. there is.

Meanwhile, when the posture estimator 120 extracts the outline of the test subject along with the skeleton of the test subject from the input image, the performance evaluation unit 130 performs matching between the outline of the motion performed by the test subject and the outline of the reference posture. Based on the accuracy, correction for the accuracy of the performed motion may be performed.

Meanwhile, FIG. 2 is a diagram illustrating an example of a process of classifying a performance level through performance level evaluation.

As in the example shown in FIG. 2 , the performance level evaluation may include a first test 210 , a second test, and sarcopenia level evaluation 230 .

In this case, the first test 210 may be a grip strength test. Specifically, the first test 210 may be performed using an elastic body (eg, a rubber ball) whose elastic force is already known. Specifically, the first test 210 is based on the change in the shape of the finger in a state where the test subject holds the elastic body and does not apply force and the state in which the test subject applies maximum force to the elastic body. It can be performed by calculating the amount of change and calculating the grip force applied to the elastic body based on the calculated amount of change and the elastic force of the elastic body.

According to an embodiment, when the calculated grip strength is greater than or equal to a preset reference value, the performance evaluation unit 130 may determine the level of the motion performed by the test subject as that of a normal elderly person, and the calculated grip strength is less than the preset reference value. In this case, it may be determined that the first test 210 has not passed.

Meanwhile, the second test 220 may be a stand-up test 220 from a chair. The second test 220 may be performed when the test subject does not pass the first test 210, and the performance evaluation unit 130 may perform the test subject's performance when the test subject passes the second test 220. The level of motion can be judged as a preliminary sarcopenia level.

Meanwhile, when the test subject does not pass both the first test 210 and the second test 220, the performance evaluation unit 130 performs the sarcopenia level evaluation 230 to determine the level of the test subject's performance. can judge

Specifically, the sarcopenia level evaluation 230 may be performed through analysis of the test subject's gait. In the gait analysis, the speed of the test subject's gait, the degree of balance, the stride length, the movement of the center of gravity, etc. are analyzed, and based on this, the level of the test subject's performance is determined based on the clustering of the acquired data. It can be classified as one of the stages. In this case, the plurality of pre-set sarcopenia stages may include a sarcopenia level and a severe sarcopenia level, but may further include more detailed stages according to embodiments. Meanwhile, according to an embodiment, in addition to gait analysis, an analysis of maintaining a balance posture according to various foot postures may be additionally performed for the sarcopenia level evaluation 230, and in this case, the performance evaluation unit 130 performs the gait analysis result. It is possible to determine the sarcopenia stage of the test subject using the time to maintain the balance posture.

The level-by-level training determining unit 140 determines a training program to be performed by the test subject based on the level of the performed motion evaluated by the performance evaluation unit 130, and allows the test subject to perform the determined training program.

Specifically, the level-by-level training determining unit 140 is a general elderly person level, a preliminary sarcopenia level, and a plurality of sarcopenia levels (eg, sarcopenia level and severe sarcopenia level) of a training program preset for each test subject's A training program set for the level of the performed motion may be determined as a training program to be performed by the test subject.

The training coaching unit 150 is based on action recognition based on the skeleton extracted by the posture estimating unit 120 from an input image of a test subject performing training according to the training program determined by the training determining unit 140 for each level. Classify the training motion performed by the test subject and evaluate the accuracy of the classified training motion.

Specifically, the training coaching unit 150 evaluates the training performance process (eg, the type of unit training being performed by the test subject, the number of times of execution, the execution time, etc.) based on the divided training motions, and the training motion and the dictionary Accuracy of the training motion may be evaluated by comparing the set reference training motion.

In addition, the training coach 150 may provide feedback so that the test subject performs a motion consistent with the reference training motion based on the angle and position of the joint that is different from the reference training motion in the training motion. In this case, the feedback may be provided by displaying the position change direction of the joint to be changed in the form of an arrow on the screen displaying the input image to match the training motion of the test subject with the reference training motion, as in the example shown in FIG. 3 . there is. At this time, the screen may be an additionally installed screen to easily receive feedback while training the test subject, but is not necessarily limited thereto.

Meanwhile, the form of feedback provided by the training coaching unit 150 is not limited to the above example, and may be provided in various forms using, for example, a virtual avatar or animation.

Meanwhile, according to an embodiment, the training coaching unit 150 may determine whether to re-evaluate the performance level of the test subject according to a preset criterion. Specifically, the training coach 150 determines the level of the motion performed by the test subject when the test subject performs the training motion according to the training program for a preset time or number of times or when the accuracy of the training motion continues at a certain level or more. A re-evaluation may be deemed necessary.

Meanwhile, when it is determined by the training coaching unit 150 that the level of the performed motion needs to be re-evaluated, the performance evaluation unit 130 re-evaluates the level of the motion performed by the test subject. At this time, the performance evaluation unit 130 performs re-evaluation in the same way as the method used to evaluate the level of the motion performed by the test subject, but the same level as the current level of the motion performed by the test subject, an improved level by one level, and a level of deterioration by one level. The level of sarcopenia can be re-evaluated through the evaluation of at least one of the levels identified. In addition, the performance evaluation unit 130 evaluates additional motion performance when reevaluating the level of the motion performed by the test subject in order to set up a training program considering the characteristics of each individual, based on the motion range, motion accuracy, and motion time. operating characteristics can be identified.

Meanwhile, the level-by-level training determining unit 140 may re-determine a training program to be performed by the test subject based on the performance level of the test subject re-evaluated by the performance evaluation unit 130 . At this time, the level-by-level training determining unit 140 may newly set a personalized training program considering the characteristics of the individual identified by the performance evaluation unit 130 when re-evaluating the level of the motion performed by the test subject, and changes in the sarcopenia stage may occur. If there is, the training program can be reset to fit the fluctuated phase.

Meanwhile, an embodiment of the present invention may include a computer readable recording medium including a program for performing the methods described in this specification on a computer. The computer readable recording medium may include program instructions, local data files, local data structures, etc. alone or in combination. The media may be specially designed and configured for the present invention, or may be commonly available in the field of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floppy disks, and ROMs, RAMs, flash memories, and the like. A hardware device specially configured to store and execute program instructions is included. Examples of program instructions may include high-level language codes that can be executed by a computer using an interpreter or the like as well as machine language codes generated by a compiler.

Although representative embodiments of the present invention have been described in detail above, those skilled in the art will understand that various modifications are possible to the above-described embodiments without departing from the scope of the present invention. . Therefore, the scope of the present invention should not be limited to the described embodiments and should not be defined, and should be defined by not only the claims to be described later, but also those equivalent to these claims.

100: sarcopenia diagnosis and training device
110: input unit
120: posture estimation unit
130: performance evaluation unit
140: training decision unit for each level
150: training coaching department

Claims (16)

an input unit for acquiring an input image of a subject to be examined;
a posture estimation unit extracting a skeleton of the test subject from the input image and estimating a posture of the test subject;
a performance evaluation unit that classifies the performed motion of the test subject through motion recognition based on the extracted skeleton and evaluates the level of the performed motion;
a level-by-level training determining unit for determining a training program to be performed by the test subject based on the level of the performing motion; and
The training motion of the test subject is classified through action recognition based on a skeleton extracted from an input image of a test subject performing training according to the training program, and feedback is provided to the test subject based on the accuracy of the training motion. Including a training coaching department that provides,
The performance evaluation unit,
determining a level of the performed motion based on a result of at least one of a plurality of tests and sarcopenia level assessment;
The plurality of tests,
Based on the change in the shape of the test subject's finger in a state where the test subject holds the elastic body and does not apply force and the state in which the test subject applies maximum force to the elastic body, and the elastic force of the elastic body, to the test subject A first test for testing the grip force applied to the elastic body by
The performance evaluation unit,
When the first test is passed, the level of the motion performed by the test subject is determined to be that of a general elderly person,
If the first test is not passed and the second test is passed, the level of the test subject's performance is determined as a preliminary sarcopenia level,
When the first test and the second test are not passed, the level of the test subject's performance is classified as one of a plurality of pre-set sarcopenia stages through the gait analysis of the test subject,
The training coaching department,
The input image provides the feedback so that the test subject performs a training motion matching a preset reference training motion, and the position change direction of a joint to be changed in order to match the training motion of the test subject with the reference training motion. An apparatus for diagnosing and training sarcopenia providing the feedback by displaying in the form of an arrow on the displayed screen.
delete delete delete The method of claim 1,
The training coach unit compares the training motion with a preset reference training motion to evaluate the accuracy of the training motion. Sarcopenia diagnosis and training apparatus.
The method of claim 1,
The performance evaluation unit calculates the angle formed by each joint from the connection relationship between the movement of each joint included in the skeleton and the neighboring joint in the performing motion, and the calculated angle of each joint and the preset reference posture. An apparatus for diagnosing and training sarcopenia for determining the accuracy of the performed motion based on the difference between the angles of each joint.
The method of claim 6,
The posture estimation unit extracts the outline of the test subject together with the skeleton of the test subject from the input image;
The performance evaluation unit performs a correction for the accuracy of the performed motion based on the matching accuracy between the contour of the motion performed by the test subject and the contour of the reference posture.
delete Obtaining a first input image of the test subject by the input unit;
estimating the posture of the test subject by extracting the skeleton of the test subject from the first input image in a posture estimation unit;
classifying the performed motion of the test subject through motion recognition based on the extracted skeleton and evaluating the level of the performed motion in a performance evaluation unit;
determining a training program to be performed by the test subject based on the level of the performed motion in a level-by-level training determination unit;
in the training coaching department. obtaining a second input image of the test subject performing training according to the training program;
In the training coaching department. extracting the skeleton of the test subject from the second input image;
In the training coaching department. classifying a training motion of the test subject through motion recognition based on a skeleton extracted from the second input image; and
In the training coaching department. Providing feedback to the test subject based on the accuracy of the training operation;
Evaluating the level of the performed operation,
The performance evaluation unit determines a level of the performance motion based on a result of at least one of a plurality of tests and sarcopenia level evaluation;
The plurality of tests,
Based on the change in the shape of the test subject's finger in a state where the test subject holds the elastic body and does not apply force and the state in which the test subject applies maximum force to the elastic body, and the elastic force of the elastic body, to the test subject a first test for testing a grip force applied to the elastic body by and a second test to test standing up from a chair;
Evaluating the level of the performed operation,
determining, by the performance evaluation unit, a level of the motion performed by the test subject as a level of a general elderly person when the first test is passed;
determining, by the performance evaluation unit, the level of the test subject's performing motion as a preliminary sarcopenia level when the first test is not passed and the second test is passed; and
In the performance evaluation unit, if the first test and the second test are not passed, classifying the level of the test subject's performing motion as one of a plurality of preset stages of sarcopenia through gait analysis of the test subject Including more steps,
The step of providing the feedback is,
In the training coaching unit, the feedback is provided so that the test subject performs a training motion consistent with a preset reference training motion, but the location of a joint to be changed to match the training motion of the test subject with the reference training motion. A method for providing information for diagnosis and training of sarcopenia, wherein the change direction is displayed in the form of an arrow on the screen on which the input image is displayed to provide the feedback.
delete delete delete The method of claim 9,
The providing of the feedback may include comparing the training motion with a preset reference training motion to evaluate accuracy of the training motion.
The method of claim 9,
In the step of evaluating the level of the performed motion, an angle formed by each joint is calculated from a connection relationship between the movement of each joint included in the skeleton and a neighboring joint in the performed motion, and the calculated angle and A method for providing information for diagnosis and training of sarcopenia for determining accuracy of the performed motion based on a difference between angles of each joint in a preset reference posture.
The method of claim 14,
In the step of estimating the posture, the outline of the test subject is extracted together with the skeleton of the test subject from the input image;
The step of evaluating the level of the performed motion is for diagnosing and training sarcopenia, which corrects the accuracy of the motion based on matching accuracy between the contour of the motion performed by the test subject and the contour of the reference posture. How to provide information.
delete

Images