CN114973048A

CN114973048A - Method and device for correcting rehabilitation action, electronic equipment and readable medium

Info

Publication number: CN114973048A
Application number: CN202111649835.6A
Authority: CN
Inventors: 唐东强; 彭飞; 邓竹立
Original assignee: Beijing 58 Information Technology Co Ltd
Current assignee: Beijing 58 Information Technology Co Ltd
Priority date: 2021-12-29
Filing date: 2021-12-29
Publication date: 2022-08-30

Abstract

The embodiment of the invention provides a method and a device for correcting rehabilitation actions, electronic equipment and a readable medium, wherein the method comprises the following steps: acquiring a rehabilitation training video of a user executing a rehabilitation action; identifying multi-dimensional action index data from the rehabilitation training video; determining target standard index thresholds of multiple dimensions corresponding to the rehabilitation action in the current rehabilitation stage of the user; when there is the motion indicator data that does not meet the target standard indicator threshold, generating corrective information for a rehabilitative motion performed by the user based on the motion indicator data. According to the embodiment of the invention, the evaluation of the action index data corresponding to a plurality of dimensions of the rehabilitation action executed by the user can be performed based on the rehabilitation training video of the rehabilitation action executed by the user, and the rehabilitation action executed by the user is corrected according to the evaluation result, so that a doctor does not need to evaluate the rehabilitation action one by one, and the workload and the medical cost of the doctor are reduced.

Description

Method and device for correcting rehabilitation action, electronic equipment and readable medium

Technical Field

The embodiment of the invention relates to the technical field of computers, in particular to a rehabilitation action correction method, a rehabilitation action correction device, electronic equipment and a computer readable medium.

Background

With the improvement of living standard of people, more and more people participate in the sports activities, but irregular actions or excessive actions in the sports activities are inevitable and are accompanied with injuries, and after the injuries or after the operations, scientific rehabilitation training plays a crucial role in recovering the body.

In particular, scientific rehabilitation usually refers to the use of appropriate, directed or targeted body movements to help the body return to a normal state, wherein the movements performed in rehabilitation are also referred to as rehabilitation movements. However, since the user is unaware of the rehabilitation standard, he may not know which rehabilitation actions need to be performed, or even know which rehabilitation actions should be performed, he may not know whether the rehabilitation actions performed by himself meet the standard, so that the body may not be restored to a normal state due to the non-standard execution of the rehabilitation actions during the rehabilitation training process, and even more injuries and diseases may be caused to the body.

Therefore, in the rehabilitation training process of the user, the problem to be solved urgently is solved by judging whether rehabilitation action is standard or not and further determining whether correction is needed or not.

Disclosure of Invention

The embodiment of the invention provides a rehabilitation action correcting method, a rehabilitation action correcting device, electronic equipment and a computer readable storage medium, and aims to solve the problem that whether rehabilitation actions executed by a user are standard or not cannot be judged, so that the rehabilitation actions executed by the user cannot be corrected correspondingly.

The embodiment of the invention discloses a method for correcting rehabilitation actions, which comprises the following steps:

acquiring a rehabilitation training video of a user executing rehabilitation action;

identifying multi-dimensional action index data from the rehabilitation training video;

determining target standard index thresholds of multiple dimensions corresponding to the rehabilitation action in the current rehabilitation stage of the user;

when there is the motion indicator data that does not meet the target standard indicator threshold, generating corrective information for a rehabilitative motion performed by the user based on the motion indicator data.

Optionally, before the identifying the motion index data of multiple dimensions from the rehabilitation training video, the method further includes:

acquiring a standard user posture skeleton diagram corresponding to the rehabilitation action;

and generating standard index thresholds corresponding to multiple dimensions of the rehabilitation action respectively according to the standard user posture skeleton diagram.

Optionally, the identifying of the motion index data of multiple dimensions from the rehabilitation training video includes:

identifying a user gesture skeleton map from video frames of the rehabilitation training video;

combining the user gesture skeletal maps into a set of user gesture skeletal maps;

and calculating action index data corresponding to the rehabilitation action in a plurality of dimensions according to the user posture skeleton diagram set.

Optionally, the calculating, according to the user posture skeleton diagram set, action index data corresponding to the rehabilitation action in multiple dimensions respectively includes:

acquiring joint point coordinates of joint points in the user posture skeleton diagram of the user posture skeleton diagram set;

and calculating action index data respectively corresponding to a plurality of dimensions according to the joint point coordinates.

Optionally, the determining the current rehabilitation stage of the user, the target standard indicator thresholds of multiple dimensions corresponding to the rehabilitation action, includes:

acquiring a rehabilitation starting date and an injury type;

determining the current rehabilitation stage of the user in the injury type according to the rehabilitation starting date and the current system date;

and acquiring standard index thresholds corresponding to multiple dimensions of the rehabilitation action of the injury type respectively, and adjusting the standard index thresholds according to the current rehabilitation stage to obtain a target standard index threshold.

Optionally, the rehabilitation stage has a corresponding rehabilitation action, the method further comprising:

and if the rehabilitation action does not belong to the current rehabilitation stage, carrying out danger reminding.

Optionally, the motion indicator data comprises at least a speed, an acceleration, a magnitude, a duration and a number of repetitions of the rehabilitative motion.

The embodiment of the invention also discloses a device for correcting the rehabilitation action, which comprises:

the video acquisition module is used for acquiring a rehabilitation training video for a user to execute a rehabilitation action;

the data identification module is used for identifying multi-dimensional action index data from the rehabilitation training video;

the standard acquisition module is used for determining target standard index thresholds of multiple dimensions corresponding to the rehabilitation action in the current rehabilitation stage of the user;

a motion correction module to generate correction information for a rehabilitation motion performed by the user based on the motion indicator data when there is the motion indicator data that does not meet the target standard indicator threshold.

Optionally, the apparatus further comprises: the standard index acquisition module is used for acquiring a standard user posture skeleton diagram corresponding to the rehabilitation action; and generating standard index thresholds corresponding to multiple dimensions of the rehabilitation action respectively according to the standard user posture skeleton diagram.

Optionally, the data recognition module is configured to recognize a user gesture skeleton map from a video frame of the rehabilitation training video; combining the user gesture skeletal maps into a set of user gesture skeletal maps; and calculating action index data corresponding to the rehabilitation action in a plurality of dimensions according to the user posture skeleton diagram set.

Optionally, the data recognition module is configured to obtain joint coordinates of joints in the user gesture skeleton diagrams of the user gesture skeleton diagram set; and calculating action index data respectively corresponding to multiple dimensions according to the joint point coordinates.

Optionally, the standard obtaining module is configured to obtain a rehabilitation starting date and a damage type; determining the current rehabilitation stage of the user in the injury type according to the rehabilitation starting date and the current system date; and acquiring standard index thresholds corresponding to multiple dimensions of the rehabilitation action of the injury type respectively, and adjusting the standard index thresholds according to the current rehabilitation stage to obtain a target standard index threshold.

Optionally, the rehabilitation phase has a corresponding rehabilitation action, the apparatus further comprising: and the danger reminding module is used for carrying out danger reminding if the rehabilitation action does not belong to the current rehabilitation stage.

The embodiment of the invention also discloses electronic equipment which comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory finish mutual communication through the communication bus;

the memory is used for storing a computer program;

the processor is configured to implement the method according to the embodiment of the present invention when executing the program stored in the memory.

Also disclosed are one or more computer-readable media having instructions stored thereon, which, when executed by one or more processors, cause the processors to perform a method according to an embodiment of the invention.

The embodiment of the invention also discloses a computer program product, which is stored in a storage medium and is executed by at least one processor to realize the method according to the embodiment of the invention.

The embodiment of the invention has the following advantages:

in the embodiment of the invention, a rehabilitation training video for a user to execute rehabilitation action is acquired, action index data of multiple dimensions are identified from the rehabilitation training video, then a target standard index threshold corresponding to the current rehabilitation stage of the user is determined, and when action index data which do not meet the standard index threshold exist, the rehabilitation action is corrected based on the action identification. According to the embodiment of the invention, the evaluation of the action index data corresponding to a plurality of dimensions of the rehabilitation action executed by the user can be performed based on the rehabilitation training video of the rehabilitation action executed by the user, and the rehabilitation action executed by the user is corrected according to the evaluation result, so that a doctor does not need to evaluate the rehabilitation action one by one, and the workload and the medical cost of the doctor are reduced.

Drawings

FIG. 1 is a flow chart illustrating the steps of a method for correcting rehabilitation movements according to an embodiment of the present invention;

FIG. 2 is a complete flow chart of a correction of rehabilitation motions provided in an embodiment of the present invention;

fig. 3 is a block diagram of a device for correcting rehabilitation motions according to an embodiment of the present invention.

Detailed Description

In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in further detail below.

In a related technical scheme, a rehabilitation action standard judgment method based on a sensor is provided, and specifically, a sample rehabilitation action sequence of rehabilitation training is firstly obtained, then a user wears the sensor on the body, so that the rehabilitation action sequence executed in the rehabilitation training process of the user can be obtained by using the sensor, the sum of standardized distances between the rehabilitation action sequence executed by the user and the sample rehabilitation action sequence is calculated through a Dynamic Time Warping (DTW) algorithm, and finally the sum of the standardized distances is input into a classifier to analyze the rehabilitation action sequence, so that the accuracy of the rehabilitation action sequence executed by the user is evaluated, and the user further corrects the executed action according to the feedback evaluation.

However, although the sensors can accurately collect data related to the rehabilitation action of the user, the user is generally not allowed to wear any equipment in the rehabilitation process, and if the user wears the sensors, the proprioception and muscle contraction and extension of the user are influenced to a certain extent, and then the rehabilitation of the user is influenced.

In view of the above problems, an embodiment of the present invention provides a method for correcting a rehabilitation action, which can determine, from a rehabilitation training video of the rehabilitation action performed by a user, the rehabilitation action performed by the user from the rehabilitation training video based on skeleton recognition, further recognize action index data corresponding to a plurality of dimensions, and then perform multi-dimensional evaluation on the action index data and a standard index threshold, and correct the rehabilitation action performed by the user according to an evaluation result, without requiring a doctor to perform the evaluation of the rehabilitation action one by one, thereby reducing the workload and the medical cost of the doctor. In addition, the rehabilitation action executed by the user is corrected based on the video, so that the user does not need to wear any equipment, the rehabilitation of the user can be prevented from being influenced by wearing the equipment, and a better rehabilitation effect is obtained.

Referring to fig. 1, a flowchart illustrating steps of a method for correcting a rehabilitation action provided in an embodiment of the present invention is shown, which may specifically include the following steps:

and 102, acquiring a rehabilitation training video for executing rehabilitation actions by a user.

In a specific implementation, a user may suffer from various injuries, including sports injuries and injuries caused by operations, and for different sports injuries or injuries types corresponding to operations, the user needs to perform a corresponding rehabilitation action to help the body to return to a normal state as soon as possible.

For example, for a user with a sports injury of a wrist joint, rehabilitation motions such as forward bending of the wrist joint, stretching of the wrist joint, and flexion and extension of the wrist joint related to the sports injury need to be performed to recover the flexibility of the wrist joint as soon as possible; for a user who has performed a knee joint anterior cruciate ligament reconstruction operation, rehabilitation actions such as leg bending, leg straightening and lifting, weight-bearing walking, ball clamping, leg lifting and the like related to the operation need to be performed after the operation, so that the problems of knee joint adhesion, knee joint stiffness and the like occurring after the operation are avoided. Of course, the examples of the rehabilitation action in the embodiment of the present invention are only examples, and the physician order should be regarded as the standard in practice.

The rehabilitation training video may be a video shot by the user in advance to execute a rehabilitation action, or may be a video shot in real time to execute a rehabilitation action, which is not limited in the embodiment of the present invention. Specifically, when the user can execute the rehabilitation action, the user shoots a corresponding rehabilitation training video, and then uploads the rehabilitation training video to a designated processing platform to correct the rehabilitation action, or when the user executes the rehabilitation action, the user shoots the rehabilitation training video through a terminal device such as a smart phone and a camera in real time to correct the rehabilitation action in real time.

And 104, identifying multi-dimensional action index data from the rehabilitation training video.

The motion index data of multiple dimensions may include at least the speed, acceleration, amplitude, duration, and repetition number of the rehabilitation motion performed by the user, and the like, but is not limited thereto in practice.

For example, in a rehabilitation training video in which the user performs wrist joint stretching, the motion index data that can be recognized from the rehabilitation training video may be: duration 10s, repeat 30 times.

And 106, determining target standard index thresholds of multiple dimensions corresponding to the rehabilitation action in the current rehabilitation stage of the user.

In specific implementation, the user can continuously recover along with the lapse of time, so that in practice, a plurality of corresponding recovery stages are set for each injury type, and corresponding standard index thresholds are formulated for different recovery stages, and the action index data of the recovery action executed by the user cannot exceed the standard index threshold corresponding to the current recovery stage, otherwise, the execution of the recovery action is regarded as non-standard and needs to be corrected.

For example, taking the knee joint anterior cruciate ligament reconstruction as an example, the rehabilitation stages of the injury type may include: 1. 1-2 weeks after surgery; 2. 2-5 weeks after surgery; 3. 5-6 weeks after surgery; 4. 6-8 weeks after surgery; 5. 8-12 weeks after surgery; 6. 12-20 weeks after surgery; 7. 20 weeks-1 year after surgery. Wherein, these rehabilitation stages are provided with the standard index threshold value that a plurality of dimensions correspond respectively. For example, assuming that the current rehabilitation stage of the user is determined to be 2-5 weeks after surgery, a multi-dimensional standard index threshold of the rehabilitation stage of 2-5 weeks after surgery may be obtained as the target standard index threshold.

And 108, when the action index data which does not meet the target standard index threshold exists, generating correction information aiming at the rehabilitation action performed by the user based on the action index data.

In the embodiment of the invention, the multi-dimensional action index data determined by the rehabilitation training video based on the rehabilitation action executed by the user can be respectively compared with the multi-dimensional target standard index thresholds one by one, so that the defects of the rehabilitation action executed by the user can be evaluated, and the corresponding correction information is generated based on the action index data so as to correct the rehabilitation action executed by the user, thereby obtaining a better rehabilitation training effect.

Illustratively, assuming that the target standard indicator thresholds are obtained as speed a1, acceleration b1, amplitude c1, duration d1 and number of repetitions f1, the motion indicator data determined based on the rehabilitation training video of rehabilitation motions performed by the user are speed a2, acceleration b2, amplitude c2, duration d2 and number of repetitions f2, wherein if a1 > a2, correction information that the speed of rehabilitation motions performed by the user is too high may be generated, if acceleration b1 > b2, correction information that the acceleration of rehabilitation motions performed by the user is too high may be generated, if c1 > c2, correction information that the amplitude of rehabilitation motions performed by the user is too high may be generated, if d1 < d2, correction information that the duration of rehabilitation motions performed by the user is insufficient may be generated, if f1 < f2, correction information that the number of times of rehabilitation motions performed by the user may be generated, and the user may correspondingly adjust the performed rehabilitation motions based on the correction information, realize the correction of rehabilitation action.

In the method for correcting the rehabilitation action, a rehabilitation training video for a user to execute the rehabilitation action is acquired, action index data of multiple dimensions are recognized from the rehabilitation training video, then a target standard index threshold corresponding to the current rehabilitation stage of the user is determined, and when action index data which do not meet the standard index threshold exist, the rehabilitation action is corrected based on the action identification. According to the embodiment of the invention, the evaluation of the action index data corresponding to a plurality of dimensions of the rehabilitation action executed by the user can be performed based on the rehabilitation training video of the rehabilitation action executed by the user, and the rehabilitation action executed by the user is corrected according to the evaluation result, so that a doctor does not need to evaluate the rehabilitation action one by one, and the workload and the medical cost of the doctor are reduced.

On the basis of the above-described embodiment, a modified embodiment of the above-described embodiment is proposed, and it is to be noted herein that, in order to make the description brief, only the differences from the above-described embodiment are described in the modified embodiment.

In an exemplary embodiment, the step 104 of identifying the motion index data of multiple dimensions from the rehabilitation training video may include the following steps:

In particular implementations, the rehabilitation action performed by the user may be identified through the skeleton. In particular, the amount of the solvent to be used, _{human being} The skeleton of the body is an internal frame of the human body, one skeleton can be abstracted into two elements which are respectively a joint point (joint) and a skeleton (bone), wherein the joint point is used for connecting two adjacent skeletons, on the basis, the skeleton can be simplified into a skeleton diagram (graph) formed by points and edges, the points in the skeleton diagram correspond to the joint points in the skeleton, and the edges in the skeleton diagram correspond to the skeletons in the skeleton. Putting a skeleton graph of a skeleton in a three-dimensional Euclidean space, wherein the attribute of a point is a coordinate point (x, y, z) in the corresponding three-dimensional space, and the side is a coordinate point in the three-dimensional spaceA line segment of (a).

In the motion recognition, a change of a human's continuous semantic posture (position) over a period of time is defined as a motion, for example, waving hands, sitting down, self-photographing, raising legs, etc., so that when the motion recognition is performed based on a skeleton diagram, another dimension, that is, time, needs to be added, and in practical applications, coordinate points of the skeleton diagram are discrete in the time dimension. In the field of computer vision, the definition of skeleton-based motion recognition is: a sequence composed of a plurality of skeleton maps of a skeleton is discriminated, and an action expressed by an executor semantically represented by the skeleton is recognized. In the embodiment of the present invention, the rehabilitation action performed by the user of the embodiment of the present invention may be recognized through the skeleton.

As an alternative example of the present invention, a Vision framework may be employed to recognize rehabilitative actions in a rehabilitation training video. Specifically, a Vision framework is a framework based on a coreML technology, which is introduced from the iOS 11 system, provides technologies such as face recognition, object detection, object tracking, and the like, and adds body posture detection to the iOS 14 system, so that in the embodiment of the present invention, a skeleton diagram of an image frame in a rehabilitation training video is recognized by using the Vision framework, and a rehabilitation action is recognized based on the skeleton diagram.

In the embodiment of the invention, user posture skeleton maps corresponding to video frames in a rehabilitation training video can be acquired based on a Vision frame, then the user posture skeleton maps are combined into a user posture skeleton map set, and further, action index data corresponding to rehabilitation actions executed by a user in multiple dimensions, for example, action index data of the rehabilitation actions for lifting shanks in several dimensions of speed, acceleration, amplitude, duration and repetition times, can be identified based on the user posture skeleton map set.

As an optional example of the present invention, a part of the rehabilitation training video has a longer duration, and accordingly, the number of video frames in the rehabilitation video may be very large, so in the embodiment of the present invention, it may be set that it is not necessary to perform skeleton recognition on all video frames in the rehabilitation training video to obtain the user posture skeleton map, but only a part of the video frames may be taken to perform skeleton recognition to obtain the user posture skeleton map, for example, one video frame may be obtained from the rehabilitation training video at regular intervals according to the playing time sequence of the video frames, so that the number of video frames that need to be subjected to skeleton recognition may be greatly reduced without affecting the recognition result, and the recognition efficiency may be improved.

Of course, when the embodiment of the present invention is specifically implemented, other manners may also be used to obtain the motion index data corresponding to the rehabilitation motion in the rehabilitation training video, which is not limited in the embodiment of the present invention.

In the above exemplary embodiment, the skeleton is used for identifying the action index data corresponding to the rehabilitation action in multiple dimensions, so that accurate data can be acquired, and further, when the rehabilitation action executed by the user is evaluated based on the action index data, the evaluation result can be more accurate.

In an exemplary embodiment, the calculating, according to the set of user gesture skeletons, motion index data corresponding to the rehabilitation motion in multiple dimensions, respectively, may include:

Specifically, joint point coordinates of joint points in the user posture skeleton diagram set, for example, joint point coordinates of a knee joint, joint point coordinates of a wrist joint, joint point coordinates of a neck joint, joint point coordinates of a head joint, and the like, may be acquired, and then motion index data respectively corresponding in a plurality of dimensions may be calculated based on the joint point coordinates.

In the embodiment of the invention, the joint point coordinates of the user posture skeleton diagram set are obtained to calculate the coordinate difference value, and the action index data such as the speed, the acceleration, the amplitude, the duration, the repetition times and the like of the rehabilitation action executed by the user in the rehabilitation training video are calculated based on the coordinate difference value.

For example, for the velocity calculation of rehabilitation motion, mainly evaluated is motion close to uniform training, and using the kinematic formula v ═ s/t, the corresponding execution code may be:

// speed calculation

func getSpeedWithTwoPoints(point1：CGPoint，point2：CGPoint，duration：CGFloat)->CGFloat{

let distance＝sqrt(pow((point1.x-point2.x)，2)+pow((point1.y-point2.y)， 2)

let speed＝distance/duration

return speed

}

Wherein point1 and point2 refer to joint point coordinates in two user gesture skeleton maps, respectively.

For the acceleration of the rehabilitation action, the calculation evaluates the type of injury requiring acceleration and deceleration, for example the rehabilitation action of standing fast and sitting slowly, using the kinematic formula of the acceleration: s is 0.5at ^2, and the corresponding execution code may be:

// acceleration calculation

func getAccelerationWithTwoPoints(point1：CGPoint，point2：CGPoint， duration：CGFloat)->CGFloat{

let distance＝sqrt(pow((point1.x-point2.x)，2)+pow((point1.y-point2.y)， 2)

let Acceleration＝sqrt(2*distance)

return acceleration

}

In addition, the amplitude can be calculated by the coordinate difference of the joint point coordinates in the two user posture skeleton diagrams, the duration can be calculated by the time difference in the two user posture skeleton diagrams, and the repetition times can be obtained by counting the times of executing the rehabilitation action in the user posture skeleton diagrams.

In the above exemplary embodiment, the motion index data corresponding to the rehabilitation motion in multiple dimensions is calculated according to the joint point coordinates of the user posture skeleton diagram set, so that accurate motion index data can be obtained, and further, when the rehabilitation motion is evaluated based on the motion index data, the evaluation result can be more accurate.

In an exemplary embodiment, before the step 104 of identifying the motion index data of multiple dimensions from the rehabilitation training video, the method may further include the steps of:

In the embodiment of the present invention, a standard user posture skeleton diagram corresponding to a rehabilitation action performed by a rehabilitation teacher or other professionals may be collected in advance, and then, a standard index threshold corresponding to each of multiple dimensions of the rehabilitation action may be generated according to the standard user posture skeleton diagram, where the standard index threshold may also include data of multiple dimensions such as speed, acceleration, amplitude, duration, and repetition number.

The standard index threshold may also be identified by using a Vision framework, and the comparison between the standard index threshold and the motion index data corresponding to the generated rehabilitation motion in multiple dimensions is similar, so that the detailed description is omitted, and the similar part is only required by referring to the calculation process of the motion index data.

In the above exemplary embodiment, the standard index thresholds of the rehabilitation action corresponding to the multiple dimensions are identified through the skeleton, so that accurate data can be acquired, and further, when the rehabilitation action executed by the user is evaluated based on the standard index thresholds, the evaluation result can be more accurate.

In an exemplary embodiment, the step 106 of determining the current rehabilitation stage of the user, and the target standard indicator thresholds of multiple dimensions corresponding to the rehabilitation action, may include the following steps:

acquiring a rehabilitation starting date and a damage type;

In the embodiment of the invention, the standard index threshold of the rehabilitation action of each injury type is fixed, but the standard index threshold can be correspondingly adjusted according to the current rehabilitation stage of the user, so that the target standard index threshold corresponding to the current rehabilitation stage is obtained.

Specifically, when the user starts to perform rehabilitation training, the user may input a rehabilitation starting date and a damage type in advance, for example, the user may input 2021 year 05, month 01 as the rehabilitation starting date and input the damage type as the knee joint exercise damage according to a prompt, then, the current rehabilitation stage of the user in the damage type may be determined according to a rehabilitation starting log input by the user and the current system date, and then, the standard index threshold of multiple dimensions of the rehabilitation action is adjusted based on the current rehabilitation stage, so as to obtain the target standard index threshold.

For example, assuming the injury type is knee joint exercise injury, the rehabilitation action required to be performed by the knee joint exercise injury is leg bending action, the standard index threshold of the leg bending action is 100 times of repetition, and the first 1-2 weeks is the first recovery stage, the first 2-6 weeks is the second recovery stage, wherein, the user inputs rehabilitation starting date of 2021 year 05 month 01 and input injury type of knee joint sport injury, if the current system date is detected to be 2021 year 05 month 07 day, it may be determined that the current rehabilitation stage of the user is the first rehabilitation stage of the athletic injury, and the target criterion index threshold corresponding to the first rehabilitation stage is determined to be 50 times, and if it is detected that the current system date is 2021 year, 05 month and 30 days, the current rehabilitation stage of the user can be determined as the second rehabilitation stage of the athletic injury, and the target standard index threshold corresponding to the second rehabilitation stage is further determined to be 100 times.

In the above exemplary embodiment, the current rehabilitation stage of the user is determined according to the rehabilitation starting date and the injury type input by the user and the current system time, and then the target standard index threshold of the current rehabilitation stage can be determined according to the standard index threshold of the rehabilitation action based on the injury type, so as to correspondingly evaluate the rehabilitation action performed by the user based on the target standard index threshold, and thus the evaluation result is more reasonable.

In an exemplary embodiment, the rehabilitation stage may have a corresponding rehabilitation action, and the method may further include the steps of:

Specifically, for users with different injury types, the user is required to execute rehabilitation motions corresponding to the injury types, and in different rehabilitation stages of the injury types, besides different standard index thresholds of the rehabilitation motions required to be executed by the user, the executed rehabilitation motions are different, so if the rehabilitation motions executed by the user do not belong to the current rehabilitation stage, danger reminding is performed to avoid that the physical injuries are caused by improper rehabilitation motions executed by the user.

For example, suppose a certain sports injury is divided into two rehabilitation stages, a stooping action cannot be performed in the first rehabilitation stage, and a stooping action can be performed in the second rehabilitation stage, and suppose that it is determined that the rehabilitation stage in which the user is currently located is the first rehabilitation stage of the sports injury, if it is recognized that the user performs the stooping action in the first rehabilitation stage in a rehabilitation training video in which the user performs the rehabilitation action, a danger prompt is performed, for example [ the rehabilitation action is not in the first rehabilitation stage ], so that the user can be reminded to stop the action, and the user can perform the action only when entering the second rehabilitation stage, thereby avoiding unnecessary body injury easily caused by continuing the action.

In the above exemplary embodiment, in the process of correcting the rehabilitation action, besides reminding the user whether the rehabilitation action executed by the user meets the standard index threshold of the current rehabilitation stage, it is also determined whether the rehabilitation action executed by the user is the action of the current rehabilitation stage, and if not, danger reminding can be performed to avoid injury of the user and influence on the rehabilitation effect of the user.

In order to enable those skilled in the art to better understand that the embodiment of the present invention implements the rehabilitation motion correction process performed on the user, a complete example is described below. Referring to fig. 2, a complete flow chart of correction of rehabilitation provided by the embodiment of the present invention specifically includes the following steps:

step 1, collecting corresponding related rehabilitation actions aiming at different loss types;

step 2, recording a standard posture skeleton diagram of rehabilitation actions performed by a professional rehabilitation teacher by using a Vision framework;

step 3, determining a standard index threshold value of the rehabilitation action based on the standard posture skeleton diagram;

step 4, recognizing action index data of a posture skeleton diagram of the rehabilitation action from a rehabilitation training video of the user executing the rehabilitation action;

step 5, comparing the action index data of the user with a standard index threshold, specifically, evaluating multiple dimensions such as speed, repetition times, duration and the like respectively;

and 6, correcting the rehabilitation action executed by the user according to the evaluation result.

In the embodiment of the invention, the user can finish the rehabilitation training by shooting the rehabilitation training video for executing the rehabilitation action without wearing any equipment, the user does not need to repeatedly go to and fro the hospital, and the abnormal rehabilitation action in the rehabilitation training can be automatically corrected and reminded at home, so that the rehabilitation quality is improved.

In summary, the embodiment of the present invention is directed to a complete rehabilitation correction system for a user, and evaluates the overall rehabilitation process, including intervention in the current rehabilitation stage, such as insufficient repetition times and insufficient duration of actions, not only action posture correction, but also direct intervention and correction of rehabilitation actions performed by the user in a non-current rehabilitation stage.

It should be noted that, for simplicity of description, the method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present invention is not limited by the illustrated order of acts, as some steps may occur in other orders or concurrently in accordance with the embodiments of the present invention. Further, those skilled in the art will appreciate that the embodiments described in the specification are presently preferred and that no particular act is required to implement the invention.

Referring to fig. 3, a block diagram of a structure of a device for correcting rehabilitation action provided in an embodiment of the present invention is shown, and specifically, the device may include the following modules:

a video obtaining module 302, configured to obtain a rehabilitation training video of a user performing a rehabilitation action;

a data identification module 304, configured to identify motion index data of multiple dimensions from the rehabilitation training video;

a standard obtaining module 306, configured to determine target standard indicator thresholds of multiple dimensions corresponding to the rehabilitation action in a current rehabilitation stage of the user;

a motion correction module 308 to generate correction information for a rehabilitative motion performed by the user based on the motion indicator data when there is the motion indicator data that does not meet the target standard indicator threshold.

In an exemplary embodiment, the apparatus further comprises: the standard index acquisition module is used for acquiring a standard user posture skeleton diagram corresponding to the rehabilitation action; and generating standard index thresholds corresponding to multiple dimensions of the rehabilitation action respectively according to the standard user posture skeleton diagram.

In an exemplary embodiment, the data recognition module 304 is configured to recognize a user gesture skeleton map from video frames of the rehabilitation training video; combining the user gesture skeletal maps into a set of user gesture skeletal maps; and calculating action index data corresponding to the rehabilitation action in a plurality of dimensions according to the user posture skeleton diagram set.

In an exemplary embodiment, the data identification module 304 is configured to obtain joint coordinates of joints in the user gesture skeletal diagrams of the user gesture skeletal diagram set; and calculating action index data respectively corresponding to multiple dimensions according to the joint point coordinates.

In an exemplary embodiment, the criteria obtaining module 306 is configured to obtain a rehabilitation starting date and a damage type; determining the current rehabilitation stage of the user in the injury type according to the rehabilitation starting date and the current system date; and acquiring standard index thresholds corresponding to multiple dimensions of the rehabilitation action of the injury type respectively, and adjusting the standard index thresholds according to the current rehabilitation stage to obtain a target standard index threshold.

In an exemplary embodiment, the rehabilitation phase has a corresponding rehabilitation action, the apparatus further comprising: and the danger reminding module is used for carrying out danger reminding if the rehabilitation action does not belong to the current rehabilitation stage.

In an exemplary embodiment, the motion indicator data comprises at least a speed, an acceleration, a magnitude, a duration and a number of repetitions of a rehabilitative motion.

In summary, in the embodiment of the present invention, a rehabilitation training video for a user to perform a rehabilitation action is obtained, a plurality of dimensions of action index data are identified from the rehabilitation training video, then, a target standard index threshold corresponding to a current rehabilitation stage of the user is determined, and when there is action index data that does not meet the standard index threshold, the rehabilitation action is corrected based on the action identifier. According to the embodiment of the invention, the evaluation of the action index data corresponding to a plurality of dimensions of the rehabilitation action executed by the user can be performed based on the rehabilitation training video of the rehabilitation action executed by the user, and the rehabilitation action executed by the user is corrected according to the evaluation result, so that a doctor does not need to evaluate the rehabilitation action one by one, and the workload and the medical cost of the doctor are reduced.

For the device embodiment, since it is basically similar to the method embodiment, the description is simple, and for the relevant points, refer to the partial description of the method embodiment.

Preferably, an embodiment of the present invention further provides an electronic device, including: the processor, the memory, and the computer program stored in the memory and capable of running on the processor, when being executed by the processor, implement each process of the above-mentioned rehabilitation action correction method embodiment, and can achieve the same technical effect, and are not described herein again to avoid repetition.

The embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when being executed by a processor, the computer program implements each process of the above-mentioned correction method for rehabilitation actions, and can achieve the same technical effect, and in order to avoid repetition, the details are not repeated here. The computer-readable storage medium may be a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk.

Embodiments of the present invention further provide a computer program product, which is stored in a storage medium and executed by at least one processor to implement the processes of the embodiment of the method for correcting rehabilitation actions, and achieve the same technical effects

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element identified by the phrase "comprising an … …" does not exclude the presence of other identical elements in the process, method, article, or apparatus that comprises the element.

Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal (such as a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present invention.

While the present invention has been described with reference to the embodiments shown in the drawings, the present invention is not limited to the embodiments, which are illustrative and not restrictive, and it will be apparent to those skilled in the art that various changes and modifications can be made therein without departing from the spirit and scope of the invention as defined in the appended claims.

Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.

It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.

The functions may be stored in a computer-readable storage medium if they are implemented in the form of software functional units and sold or used as separate products. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: various media capable of storing program codes, such as a U disk, a removable hard disk, a ROM, a RAM, a magnetic disk, or an optical disk.

The above description is only for the specific embodiments of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention, and all the changes or substitutions should be covered within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A method of correcting a rehabilitative action, the method comprising:

acquiring a rehabilitation training video of a user executing a rehabilitation action;

2. The method of claim 1, wherein the identifying the plurality of dimensions of motion metric data from the rehabilitation training video comprises:

recognizing a user gesture skeleton graph from video frames of the rehabilitation training video;

and calculating action index data corresponding to the rehabilitation action in multiple dimensions according to the user posture skeleton graph set.

3. The method of claim 2, wherein the calculating motion index data corresponding to the rehabilitation motion in multiple dimensions according to the user posture skeleton diagram set comprises:

4. The method of claim 1, wherein prior to the identifying the plurality of dimensions of motion metric data from the rehabilitation training video, the method further comprises:

5. The method of claim 1, wherein the determining the current rehabilitation stage of the user, the target standard indicator threshold values of the plurality of dimensions corresponding to the rehabilitation action comprises:

acquiring a rehabilitation starting date and an injury type;

6. The method of claim 1, wherein the rehabilitation stage has a corresponding rehabilitation action, the method further comprising:

7. The method of claim 1, wherein the motion metric data includes at least a speed, acceleration, amplitude, duration, and number of repetitions of a rehabilitative motion.

8. A device for correcting rehabilitation motions, the device comprising:

the video acquisition module is used for acquiring a rehabilitation training video of a user executing rehabilitation actions;

9. An electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

the memory is used for storing a computer program;

the processor, when executing a program stored on the memory, implementing the method of any one of claims 1-7.

10. One or more computer-readable media having instructions stored thereon that, when executed by one or more processors, cause the processors to perform the method of any of claims 1-7.