KR20240043540A - System and method for estimating musculoskeletal disorders based on the estimated pose - Google Patents

Abstract

The method for estimating a musculoskeletal disease according to the present invention includes the steps of receiving an image including a subject, recognizing the position of the subject's skeleton included in the image by applying a first measurement model, based on the position of the recognized skeleton. Thus, recognizing the two-dimensional position coordinates and distances of the subject's joints, and converting the two-dimensional position coordinates into three-dimensional position coordinates based on the two-dimensional position coordinates and distances of the joints, the three-dimensional position coordinates Generating a three-dimensional view including an image of the joint using the step, performing labeling of ground truth data of the positions of the joint included in the three-dimensional view, and attaching ground truth data to the labeled ground truth data. Recognizing the subject's pose based on the subject's pose, analyzing the angle change of the joint included in the recognized pose to extract characteristics of the joint affecting the musculoskeletal disease and the joint affecting the musculoskeletal disease, and the musculoskeletal system It is characterized by comprising the step of determining whether a musculoskeletal disease exists by applying a convolutional neural network to joints that affect the disease and characteristic data of the joints that affect the musculoskeletal disease.

Description

System and method for estimating musculoskeletal disorders based on estimated pose {SYSTEM AND METHOD FOR ESTIMATING MUSCULOSKELETAL DISORDERS BASED ON THE ESTIMATED POSE}

The present invention relates to a system and method for estimating a musculoskeletal disease based on an estimated pose, and more specifically, to a system and method for estimating a musculoskeletal disease based on a pose estimated through human joint data.

When it is difficult for a doctor to directly observe a patient with a musculoskeletal disease due to temporal and spatial limitations, indirect diagnosis can be performed through imaging. However, when diagnosis is performed in this way, accuracy may be relatively low. Accordingly, in order to improve the diagnosis rate, a method of analyzing musculoskeletal movement by performing virtual reality content using a motion recognition camera can be used. If the above method is used as diagnostic auxiliary information, the accuracy can be relatively high, but when analyzing musculoskeletal movements, there is a problem that the length of the skeleton is different for each person, that is, the posture of the user's skeleton data to use the reference skeleton data and content. There is a problem that is not easy to analyze because there is a difference.

The existing skeleton recognition method using Depth Camera is a method of recognizing the skeleton using the SDK provided by the manufacturer, but this has limitations in improving the accuracy of skeleton recognition without continued technical support from the manufacturer. . Therefore, a method is needed to accurately extract the user's posture information, which is the standard for analyzing musculoskeletal exercise, and compare and analyze musculoskeletal-related exercise in real time.

The purpose of the present invention to solve the above problems is to provide a system and method for estimating musculoskeletal disorders based on an estimated pose that can improve the accuracy of skeletal recognition.

In order to achieve the above object, a method of estimating a musculoskeletal disease based on an estimated pose according to an embodiment of the present invention includes receiving an image including a subject, and applying a first measurement model to identify the subject included in the image. Recognizing the position of the skeleton, based on the recognized position of the skeleton, recognizing two-dimensional position coordinates and distances of the subject's joints, and determining the two-dimensional position coordinates and distances of the joints based on the two-dimensional position coordinates and distances of the joints. Converting into three-dimensional position coordinates, generating a three-dimensional view containing an image of the joint using the three-dimensional position coordinates, measuring the positions of the joint included in the three-dimensional view ( Performing labeling of ground truth data and recognizing the subject's pose based on the labeled ground truth data; analyzing angle changes of joints included in the recognized pose to identify joints affecting the musculoskeletal disease and the A step of extracting the characteristics of joints affecting musculoskeletal diseases and applying a convolutional neural network to the joints affecting the musculoskeletal diseases and the characteristic data of the joints affecting the musculoskeletal diseases. It is characterized by including the step of determining whether or not there is a disease.

In addition, the first measurement model is a human pose estimation model, and the distance is a distance calculated based on a depth map projected onto the two-dimensional position coordinates.

In addition, the step of converting the two-dimensional position coordinates into three-dimensional coordinates includes extracting a first parameter containing movement information of the subject's joints using the subject's gait measurement data included in the image, and It is characterized by including the step of extracting a second parameter containing angle information of the subject's joints using the subject's gait measurement data included in the image.

In addition, the first parameter is Step Length, Step Width, Step Time, Step Velocity, Stride Length, and Stride Velocity ), walking speed, distance, cadence, heel strike time, heel strike position, toe off time, It is characterized by including Toe Off Position, Foot Height, Elbow Movement, Knee Movement, and Hip Movement.

In addition, the second parameter is Hip-Knee Angle, Knee Angle, Elbow Angle, Shoulder Arm Angle, and Forward Lean Angle. ), Shoulder Lean Angle, Head Lean Angle, Hip Lean Angle, Foot Swing Angle, Hand Swing Angle and Hip Rotation Angle It is characterized by including (Hip Rotatio Angele).

In addition, the method may further include determining a joint whose reliability value is 0.6 or more in a frame included in the 3D view as an error joint, and removing the error joint.

In addition, the process of recognizing the subject's pose involves dividing direction vectors toward the position of the joint from the depth surface data of the joint position included in the labeled ground truth data using a binary regression tree (Binary regression tree). It is characterized by being performed by learning with a regression tree structure.

Additionally, the position of the joint is optimized using a mean-shift method.

The system and method for estimating musculoskeletal disorders based on the estimated pose according to the present invention can significantly improve the accuracy of skeletal recognition by using a method of analyzing the user's posture information.

Additionally, by estimating musculoskeletal diseases based on data learned from a person's whole body pose data, an objective index for treating musculoskeletal diseases can be calculated.

However, the effects that can be achieved by the system and method for estimating musculoskeletal diseases based on the pose estimated in the embodiments of the present invention are not limited to those mentioned above, and other effects not mentioned can be found in the description below. It will be clearly understood by those skilled in the art to which the present invention pertains.

The accompanying drawings, which are included as part of the detailed description to aid understanding of the present invention, provide embodiments of the present invention, and together with the detailed description, explain the technical idea of the present invention.
1 is a flow chart illustrating a method for estimating a musculoskeletal disease based on an estimated pose according to the present invention.
Figure 2 is a conceptual diagram showing the process of converting 3D coordinates in the method of estimating musculoskeletal disease based on the estimated pose according to the present invention.
Figure 3 is a conceptual diagram showing the structure of the Human Pose Estimation Model used in the process of converting 3D coordinates.
Figure 4 is a conceptual diagram showing joints recognized in the method for estimating musculoskeletal disease based on the estimated pose according to the present invention.
Figure 5 is a photograph showing the results converted into 3D coordinates in the method for estimating musculoskeletal diseases based on the estimated pose according to the present invention.
Figure 6 is a conceptual diagram showing the data processing process of the Q-buffer used as a normalization algorithm in the method of estimating musculoskeletal disease based on the estimated pose according to the present invention.
Figure 7 shows the Min-Max algorithm of the Q-buffer used as a normalization algorithm in the method of estimating musculoskeletal disease based on the estimated pose according to the present invention.
Figure 8 shows the Kalman filter algorithm used as a normalization algorithm in the method for estimating musculoskeletal diseases based on the estimated pose according to the present invention.
Figure 9 shows a real-time three-dimensional view generated in the method for estimating musculoskeletal disease based on the estimated pose according to the present invention.
Figure 10 shows the labeling process of ground truth data of joint positions in the method of estimating musculoskeletal disease based on the estimated pose according to the present invention.
Figures 11 and 12 illustrate the process of estimating and learning the subject's pose in the method of estimating a musculoskeletal disease based on the estimated pose according to the present invention.
Figure 13 shows the process of constructing a binary rare tree for each joint location in the method for estimating musculoskeletal disorders based on the estimated pose according to the present invention.
Figure 14 shows the process of recognizing the subject's whole body pose based on random tree walk in the method of estimating musculoskeletal disease based on the estimated pose according to the present invention.
Figure 15 is a configuration diagram showing the control signal conversion logic in the method for estimating musculoskeletal disease based on the estimated pose according to the present invention.
Figure 16 shows the process of generating joint data in the method of estimating musculoskeletal disease based on the estimated pose according to the present invention.
Figure 17 shows a library generated through a method for estimating musculoskeletal diseases based on the estimated pose according to the present invention.
Figure 18 is a block diagram showing a system for estimating musculoskeletal diseases based on the estimated pose according to the present invention.

Since the present invention can make various changes and have various embodiments, specific embodiments will be illustrated in the drawings and described in detail. However, this is not intended to limit the present invention to specific embodiments, and should be understood to include all changes, equivalents, and substitutes included in the spirit and technical scope of the present invention.

Terms such as first, second, etc. may be used to describe various components, but the components should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, a first component may be named a second component, and similarly, the second component may also be named a first component without departing from the scope of the present invention. The term and/or includes any of a plurality of related stated items or a combination of a plurality of related stated items.

When a component is said to be "connected" or "connected" to another component, it is understood that it may be directly connected to or connected to the other component, but that other components may exist in between. It should be. On the other hand, when it is mentioned that a component is “directly connected” or “directly connected” to another component, it should be understood that there are no other components in between.

The terms used in this application are only used to describe specific embodiments and are not intended to limit the invention. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, terms such as “comprise” or “have” are intended to designate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, but are not intended to indicate the presence of one or more other features. It should be understood that this does not exclude in advance the possibility of the existence or addition of elements, numbers, steps, operations, components, parts, or combinations thereof.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person of ordinary skill in the technical field to which the present invention pertains. Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related technology, and unless explicitly defined in the present application, should not be interpreted in an idealized or excessively formal sense. No.

Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the attached drawings. In order to facilitate overall understanding when describing the present invention, the same reference numerals are used for the same components in the drawings, and duplicate descriptions for the same components are omitted.

1 is a flow chart illustrating a method for estimating a musculoskeletal disease based on an estimated pose according to the present invention.

Referring to Figure 1, the method of estimating a musculoskeletal disease based on an estimated pose according to the present invention receives an image including a subject and applies a first measurement model to determine the location of the subject's skeleton included in the image. Recognizing step (S101), based on the position of the recognized skeleton, recognize the two-dimensional position coordinates and distance of the subject's joint, and recognize the two-dimensional position coordinate and distance based on the two-dimensional position coordinate and distance of the joint. Converting to 3D position coordinates (S102), generating a 3D view including an image of the joint using the 3D position coordinates (S103), the joint included in the 3D view Performing labeling of ground truth data of positions and recognizing the subject's pose based on the labeled ground truth data (S104), analyzing the angle change of the joint included in the recognized pose to determine the musculoskeletal system A step of extracting characteristics of the joint affecting the disease and the joint affecting the musculoskeletal disease (S105) and convolutional data on the characteristic data of the joint affecting the musculoskeletal disease and the joint affecting the musculoskeletal disease It may include a step (S106) of determining whether a musculoskeletal disease exists by applying a neural network (convolutional neural network). Hereinafter, S101 to S106 will be described in detail.

Figure 2 is a conceptual diagram showing the process of converting 3D coordinates in the method of estimating musculoskeletal disease based on the estimated pose according to the present invention, and Figure 3 is a human body posture estimation model used in the process of converting 3D coordinates. (Human Pose Estimation Model) is a conceptual diagram showing the structure, and Figure 4 is a conceptual diagram showing joints recognized in the method for estimating musculoskeletal diseases based on the estimated pose according to the present invention, and Figure 5 is a conceptual diagram showing the structure of the Human Pose Estimation Model according to the present invention. This photo shows the results converted into 3D coordinates from a method for estimating musculoskeletal diseases based on the estimated pose.

Referring to Figures 2 to 5, the method of estimating a musculoskeletal disease based on the estimated pose according to the present invention can obtain three-dimensional coordinates of the subject's joints through S101 and S102.

The method according to the present invention applies a human pose estimation model to recognize the position of the subject's skeleton in a 2D color image, and calculates the two-dimensional position coordinates (x, y) and the distance of the depth map (Depth Map(z)) projected onto the 2D position coordinates can be used to convert to a 3D coordinate system. As an example, the conversion process to 3D coordinates can be performed using OpenCV PoseNetEstimator and TensorFlow PoseNet.

Meanwhile, the method according to the present invention can utilize a total of 18 joints in calculating 3D coordinates by applying the human body pose estimation model, and the 18 joints include the center hip and spine ( Spine) can be added to utilize a total of 20 joints.

At this time, the step of converting the 2D position coordinates into 3D coordinates can be performed using the motion parameters and the angle parameters of the main joints. Specifically, using the subject's gait measurement data included in the image, extracting a first parameter containing movement information of the subject's joints and using the subject's gait measurement data included in the image, This may be performed through the step of extracting a second parameter including angle information.

The first parameter, which is a dynamic parameter, can be calculated as a gait measurement result using spatiotemporal information and can be used as a parameter when analyzing gait using movement information of major joints. Specifically, the first parameter is Step Length, Step Width, Step Time, Step Velocity, Stride Length, and Stride Velocity. , Walking Speed, Distance, Cadence, Heel Strike Time, Heel Strike Position, Toe Off Time, Forefoot Musculoskeletal disease, characterized in that it includes Toe Off Position, Foot Height, Elbow Movement, Knee Movement, and Hip Movement How to estimate .

The second parameter, which is an angle parameter, can be calculated using the angle and rotation values of major joints during walking. Specifically, the second parameter is Hip-Knee Angle, Knee Angle, Elbow Angle, Shoulder Arm Angle, and Forward Lean Angle. , Shoulder Lean Angle, Head Lean Angle, Hip Lean Angle, Foot Swing Angle, Hand Swing Angle and Hip Rotation Angle ( Hip Rotatio Angele) may be included. The results converted into 3D coordinates through the method of estimating musculoskeletal diseases based on the estimated pose according to the present invention are shown in FIG. 5.

Figure 6 is a conceptual diagram showing the data processing process of the Q-buffer used as a normalization algorithm in the method of estimating musculoskeletal diseases based on the estimated pose according to the present invention, and Figure 7 is based on the estimated pose according to the present invention. This shows the Min-Max algorithm of Q-buffer, which is used as a normalization algorithm in the method of estimating musculoskeletal diseases, and Figure 8 is used as a normalization algorithm in the method of estimating musculoskeletal diseases based on the estimated pose according to the present invention. This shows the Kalman filter algorithm.

Referring to Figures 6 to 8, the method of estimating musculoskeletal disease based on the estimated pose according to the present invention can perform normalization by generating a Q-buffer, and normalization can be performed using a Kalman filter. . Both methods can allow normalization of information input in real time.

Referring to FIGS. 6 and 7, the Q-buffer algorithm may be an algorithm that stores continuously input data in the Q-buffer and then averages the Q-buffer. Since the Q-buffer algorithm averages the current data and previous data, it can obtain linear results with reduced noise and jump phenomenon. The Q-buffer averaging algorithm averages the sum from Q(t) to Q(t-n) by the length of Q and can be performed in the following (Equation 1).

(Equation 1)

Here, n is the length of Q, and R(t) is the current data to be processed.

If you only perform averaging for the Q-buffer, error data such as noise or jumps may also be deeply averaged, resulting in incorrect data. To minimize this phenomenon, a method can be used to set minimum and maximum thresholds in the Q-buffer, define data outside the range of the threshold as error data, remove these values, and then average them. The specific algorithm is shown in Figure 7.

Meanwhile, the Kalman filter can be used to find a signal from noise so that a system can appropriately predict changes over time, and can optimally estimate not only the past, present, but also future states. there is.

The Kalman filter can be widely used in satellite navigation, missile trajectory estimation, radar, etc., and recently, with the development of microprocessors, it can be applied to very complex real-time processing systems. The equation used in the Kalman filter algorithm can be implemented as a time update equation and a measurement update equation for prediction, and the Kalman filter algorithm can be implemented as a prediction equation and an observation equation. The specific equation is shown in Figure 8.

Referring to FIG. 8, Time Update is a step of predicting the current state and can deliver the current state estimation result. Measurement Update can update the estimated state values delivered by actual measurement at the corresponding time. In Time Update, x is a state variable, A is a conversion coefficient connecting the state variables, and B and u are additional input values that are unrelated to the system. Q is the difference between the actual value of the state variable x at step k. In Measurement Update, z is the observed value, which is expressed by x and the conversion coefficient H, and R is the error between z and the observed actual value.

The method according to the present invention in time The Measurement Update Algorithm was applied to calculate a new optimal value using the optimal value of and the new input value. In addition, the optimal value and standard deviation for each time are related, and these data can be used to estimate the optimal value in the past, present, and future. hour The predicted value in , standard deviation The algorithm for finding the optimal value may be as shown in Equation 2 below.

(Equation 2)

the first When is 1, the input data becomes the optimal value, and when is 2, the optimal value is obtained by averaging the value of 1 and the new input value, and the standard deviation can be obtained using this optimal value and the previous optimal value. When is 1 or 2, it can be said to be a preparation stage for optimization. From then on, you can obtain the optimal value for the input data by making predictions and observations.

Figure 9 shows a real-time three-dimensional view generated in the method for estimating musculoskeletal disease based on the estimated pose according to the present invention.

The method according to the present invention can provide a three-dimensional view (for example, Skeleton View) using the three-dimensional coordinates of the user's major joints on a real-time skeleton recognition and measurement result analysis screen. 3D View can provide Perspective, Side, and Front View. Meanwhile, the method according to the present invention can determine a joint with a joint reliability value of 0.6 or more in a frame included in a 3D view as an error joint, and perform a process of removing the error joint. The reliability value may have a value of 0 to 1.

Figure 10 shows the labeling process of ground truth data of joint positions in the method of estimating musculoskeletal disease based on the estimated pose according to the present invention.

The method according to the present invention involves the process of recognizing and learning the subject's whole body and labeling ground truth data of the positions of joints included in the whole body part (e.g., a total of 19 joint positions (head, neck, shoulders) Perform labeling of ground truth data corresponding to (left/right), elbow (left/right), wrist (left/right), hip (left/right), knee (left/right), and ankle (left/right)) can do. At this time, using the linkage with depth data obtained from the depth camera, actual data labeling can be performed through a semi-automation method.

Figures 11 and 12 show the process of estimating and learning the subject's pose in the method of estimating musculoskeletal disease based on the estimated pose according to the present invention, and Figure 13 shows the process of estimating and learning the subject's pose based on the estimated pose according to the present invention. In the method for estimating musculoskeletal diseases, the process of constructing a binary rare tree for each joint location is shown. Figure 14 shows the subject's entire body based on a random tree walk in the method for estimating musculoskeletal diseases based on the estimated pose according to the present invention. This shows the process of recognizing a pose.

Referring to Figures 11 to 14, the method according to the present invention can perform a human whole body pose learning process based on self-acquired data and a random tree walk. For example, a human full body pose learning algorithm can be performed using the public database SMMC-10. In addition, the method according to the present invention can perform human full-body pose learning/testing/verification based on random tree walk and geodesic information using self-acquired data (21 people, 16 movements). .

Random tree walk may be a method of optimally estimating the center point of a part by learning the appearance information of each part centered on the estimated human pose position. This can be done by learning the direction vectors that go to the center point of each joint from specific depth surface data in a binary regression tree structure to recognize the human whole body pose.

The method according to the present invention is a human full body recognition method based on full body pose learning and motion feature extraction. For example, learning (depth) data for each of 14 joints can be extracted to learn the subject's full body pose. Afterwards, the pose can be recognized by creating regression trees for each of the 19 joints and hierarchically searching the positions of each joint from the center point of the human depth segment. The final positions of joints can be optimized using mean-shift and deep learning.

The method according to the present invention can classify musculoskeletal diseases based on human whole body pose recognition. In other words, the level of musculoskeletal disease can be classified by applying 16 movement protocols based on depth camera-based marker data. Additionally, by analyzing the angle change trend for each joint, joints and joint characteristics suitable for disease classification can be extracted. Disease classification can be performed using joints and joint characteristic data suitable for disease classification using a convolutional neural network.

Figure 15 is a configuration diagram showing control signal conversion logic in the method of estimating musculoskeletal disease based on the estimated pose according to the present invention.

Referring to FIG. 15, the control module for executing the method according to the present invention may be a semi-automatic editing tool linked to a depth sensor for generating and collecting actual measurement learning data. The control module can generate and collect human body depth map, annotated human body region map, and annotated human body joint information for one human body pose.

In addition, the depth map can be obtained from the current commercial device Microsoft Azure Kinect, and the reference human joint information for editing can be used by Nuitrack2), a commercial pose SDK. The generated and collected dataset may consist of standing, (one/both) arm movements, foot movements, and walking motions, and it may be possible to generate and collect various posture data depending on future application purposes. Meanwhile, the actual human body data generated and collected may consist of 19 joints in 18 areas.

Figure 16 shows the process of generating joint data in the method for estimating musculoskeletal disease based on the estimated pose according to the present invention, and Figure 17 shows the method for estimating musculoskeletal disease based on the estimated pose according to the present invention. The library created through is shown, and data of five motions as shown in FIG. 17 were generated through the human body learning data generation module (learning data generation module for human body posture tracking).

Figure 18 is a block diagram showing a system for estimating musculoskeletal disorders based on the estimated pose according to the present invention.

Referring to FIG. 18, the musculoskeletal disease estimation system 1000 according to the present invention includes a 3D coordinate conversion unit 100, a 3D view creation unit 110, a pose recognition unit 120, and a musculoskeletal disease determination unit 130. ) may include. A 3D coordinate conversion unit 100, a 3D view generator 110, a pose recognition unit 120, and a musculoskeletal disease determination unit included in the system 1000 for estimating a musculoskeletal disease based on an estimated pose according to the present invention. Unit 130 may be a component that performs the processes S101 to S106 described with reference to FIG. 1.

The operation of the method according to an embodiment of the present invention can be implemented as a computer-readable program or code on a computer-readable recording medium. Computer-readable recording media include all types of recording devices that store data that can be read by a computer system. Additionally, computer-readable recording media can be distributed across networked computer systems so that computer-readable programs or codes can be stored and executed in a distributed manner.

Additionally, computer-readable recording media may include hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, etc. Program instructions may include not only machine language code such as that created by a compiler, but also high-level language code that can be executed by a computer using an interpreter, etc.

Although some aspects of the invention have been described in the context of an apparatus, it may also refer to a corresponding method description, where a block or device corresponds to a method step or feature of a method step. Similarly, aspects described in the context of a method may also be represented by corresponding blocks or items or features of a corresponding device. Some or all of the method steps may be performed by (or using) a hardware device, such as a microprocessor, programmable computer, or electronic circuit, for example. In some embodiments, one or more of the most important method steps may be performed by such an apparatus.

In embodiments, a programmable logic device (e.g., a field programmable gate array) may be used to perform some or all of the functionality of the methods described herein. In embodiments, a field programmable gate array may operate in conjunction with a microprocessor to perform one of the methods described herein. In general, the methods are preferably performed by some hardware device.

Although the present invention has been described with reference to preferred embodiments of the present invention, those skilled in the art may make various modifications and changes to the present invention without departing from the spirit and scope of the present invention as set forth in the claims below. You will understand that it is possible.

1000: Musculoskeletal disease estimation system
100: 3D coordinate conversion unit
110: 3D view creation unit
120: Pose recognition unit
130: Musculoskeletal disease determination unit

Claims (8)

As a method of estimating musculoskeletal diseases,
Receiving an image including a subject, and applying a first measurement model to recognize the position of the subject's skeleton included in the image;
Based on the recognized position of the skeleton, recognizing the two-dimensional position coordinates and distances of the subject's joints, and converting the two-dimensional position coordinates into three-dimensional position coordinates based on the two-dimensional position coordinates and distances of the joints step;
generating a 3D view including an image of the joint using the 3D position coordinates;
performing labeling of ground truth data of the positions of the joints included in the 3D view, and recognizing the subject's pose based on the labeled ground truth data;
Analyzing changes in angles of joints included in the recognized pose to extract joints affecting the musculoskeletal disease and characteristics of the joints affecting the musculoskeletal disease; and
Characterized by comprising the step of determining whether a musculoskeletal disease exists by applying a convolutional neural network to the joints affecting the musculoskeletal disease and the characteristic data of the joints affecting the musculoskeletal disease. How to estimate musculoskeletal diseases. In claim 1,
The first measurement model is a Human Pose Estimation Model, and the distance is a distance calculated based on a depth map projected onto the two-dimensional position coordinates. How to. In claim 1,
The step of converting the two-dimensional position coordinates into three-dimensional coordinates is,
extracting a first parameter containing movement information of the subject's joints using the subject's gait measurement data included in the image; and
A method of estimating a musculoskeletal disease, comprising the step of extracting a second parameter containing angle information of the subject's joints using the subject's gait measurement data included in the image. In claim 3,
The first parameter is Step Length, Step Width, Step Time, Step Velocity, Stride Length, Stride Velocity, Walking speed, distance, cadence, heel strike time, heel strike position, toe off time, front heel Musculoskeletal diseases, characterized by including foot off position, foot height, elbow movement, knee movement, and hip movement. How to estimate. In claim 3,
The second parameter is Hip-Knee Angle, Knee Angle, Elbow Angle, Shoulder Arm Angle, Forward Lean Angle, Shoulder Lean Angle, Head Lean Angle, Hip Lean Angle, Foot Swing Angle, Hand Swing Angle and Hip Swing Angle A method for estimating musculoskeletal disorders, characterized in that it includes Rotatio Angele. In claim 1,
A method for estimating a musculoskeletal disease, further comprising determining a joint whose reliability value is 0.6 or more in the frame included in the three-dimensional view as an error joint, and removing the error joint. In claim 1,
The process of recognizing the subject's pose is,
Characterized by learning direction vectors toward the position of the joint from the depth surface data of the joint position included in the labeled ground truth data in a binary regression tree structure. How to estimate musculoskeletal diseases. In claim 7,
A method for estimating musculoskeletal diseases, characterized in that the position of the joint is optimized using a mean-shift method.

Images