WO2021217906A1

WO2021217906A1 - Posture detection method, apparatus and device based on gait features, and storage medium

Info

Publication number: WO2021217906A1
Application number: PCT/CN2020/103198
Authority: WO
Inventors: 田金戈; 徐国强
Original assignee: 深圳壹账通智能科技有限公司
Priority date: 2020-04-28
Filing date: 2020-07-21
Publication date: 2021-11-04
Also published as: CN111639533A

Abstract

A posture detection method, apparatus and device based on gait features, and a storage medium, which relate to the technical field of artificial intelligence. The method comprises: acquiring gait data of a target object, and detecting, in the gait data, human body key points of the target object, so as to obtain a plurality of key point features (101); performing feature classification and positioning regression on the plurality of key point features by using a deep learning network, so as to obtain human body information (102); on the basis of the plurality of key point features, calculating a matching parameter between the human body information and preset standard human body information, wherein the preset standard human body information comprises a preset standard affinity vector (103); and determining a posture condition of the target object according to the matching parameter and a standard threshold value, wherein the standard threshold value is a critical value of an anomalous posture condition (104). Relevant data and information in the method can be stored in a blockchain node. By means of the method, the problems of high cost and low detection efficiency during object posture detection are solved.

Description

Body posture detection method, device, equipment and storage medium based on gait feature

This application claims the priority of the Chinese patent application filed with the Chinese Patent Office on April 28, 2020, the application number is 202010350100.2. The invention title is "Methods, devices, equipment and storage media for posture detection based on gait characteristics", all of which The content is incorporated in the application by reference.

Technical field

This application relates to the field of deep learning technology, in particular to a posture detection method, device, equipment and storage medium based on gait features.

Background technique

People in modern cities often suffer from cervical and lumbar diseases due to work and life and other reasons, such as the loss of cervical spine physiology and disc herniation. These diseases are closely related to long-term fixed posture work and study, and they are not easy to be noticed during the formation and development of the disease. In the early stages of disease development, experienced medical personnel are required to identify and discover. When the disease has progressed to a certain level, it is necessary to go to a medical institution to check and confirm the diagnosis with medical equipment such as CT, MRI, and angiography.

The inventor realizes that when the target object suffers from the initial cervical spine, lumbar spine and other diseases, the gait posture of the target object will be changed due to the disease. When such a posture change is diagnosed with the help of medical tools, the detection cost is high and the need The detection result will be obtained after a certain waiting period, which leads to low detection efficiency of detecting the posture of the target object.

Summary of the invention

The main purpose of this application is to solve the problems of high cost and low detection efficiency when detecting the posture of the target object.

In order to achieve the above objective, the first aspect of the present application provides a posture detection method based on gait features, including: acquiring gait data of a target object, and determining the key points of the target object in the gait data Perform detection to obtain multiple key point features. The human body key points are multiple coordinate points of the human body skeleton; the deep learning network is used to perform feature classification and positioning regression on the multiple key point features to obtain human body information. The information includes the position of the human body part and the affinity vector of the human body part. The human body part affinity vector is used to connect two different key point features; based on the multiple key point features, the difference between the human body information and the preset standard human body information is calculated. The preset standard human body information includes a preset standard affinity vector, and the matching parameter is used to indicate the degree of matching between the body information of the target object and the preset standard body information; according to the matching parameter and The standard threshold value determines the posture condition of the target object, and the standard threshold value is the critical value of the abnormal posture condition.

The second aspect of the present application provides a posture detection device based on gait features, including a memory, a processor, and computer-readable instructions stored on the memory and running on the processor, and the processor executes The computer-readable instruction implements the following steps: acquiring the gait data of the target object, and detecting the key points of the human body of the target object in the gait data to obtain a plurality of key point features, the key to the human body The points are multiple coordinate points of the human skeleton; the deep learning network is used to perform feature classification and positioning regression on the multiple key point features to obtain human body information. The human body information includes the position of the human body part and the affinity vector of the human body part. The human body part affinity vector is used to connect two different key point features; based on the multiple key point features, the matching parameters between the human body information and preset standard human body information are calculated, and the preset standard human body information includes preset A standard affinity vector is set, and the matching parameter is used to indicate the degree of matching between the human body information of the target object and the preset standard human body information; the body condition of the target object is determined according to the matching parameter and the standard threshold, the The standard threshold is the critical value of abnormal posture.

The third aspect of the present application provides a computer-readable storage medium in which computer instructions are stored, and when the computer instructions are run on a computer, the computer executes the following steps: step of obtaining a target object The human body key points of the target object are detected in the gait data, and multiple key point features are obtained. The human body key points are multiple coordinate points of the human skeleton; Performing feature classification and positioning regression on the multiple key point features to obtain human body information, where the human body information includes a human body part position and a human body part affinity vector, and the human body part affinity vector is used to connect two different key point features; The matching parameters between the human body information and preset standard human body information are calculated based on the multiple key point features, the preset standard human body information includes a preset standard affinity vector, and the matching parameter is used to indicate the target object's The degree of matching between the human body information and the preset standard human body information; the posture condition of the target object is determined according to the matching parameter and a standard threshold, and the standard threshold is a critical value for an abnormal posture condition.

The fourth aspect of the present application provides a posture detection device based on gait features, including: a detection module for acquiring gait data of a target object, and determining the key points of the target object’s body in the gait data Perform detection to obtain multiple key point features, the human body key points are multiple coordinate points of the human skeleton; the classification and regression module is used to perform feature classification and positioning regression on the multiple key point features based on the deep learning network, Obtain human body information. The human body information includes the position of the human body part and the affinity vector of the human body part. The human body part affinity vector is used to connect two different key point features; the calculation module is used to based on the multiple key point features Calculate the matching parameters between the body information and preset standard body information, the preset standard body information includes a preset standard affinity vector, and the matching parameters are used to indicate the body information of the target object and the preset standard body information The degree of matching between; a determining module, configured to determine the posture condition of the target object according to the matching parameter and a standard threshold, where the standard threshold is a critical value for an abnormal posture condition.

In the technical solution provided in this application, the gait data of the target object is acquired, and the human body key points of the target object are detected in the gait data to obtain multiple key point features, and the human body key points are the human body Multiple coordinate points of the skeleton; using a deep learning network to perform feature classification and positioning regression on the multiple key point features to obtain human body information. The human body information includes the position of the human body part and the affinity vector of the human body part. The sum vector is used to connect two different key point features; based on the multiple key point features, the matching parameters between the human body information and the preset standard human body information are calculated, and the preset standard human body information includes the preset standard relative information. The sum vector, the matching parameter is used to indicate the degree of matching between the human body information of the target object and the preset standard human body information; the body condition of the target object is determined according to the matching parameter and a standard threshold, and the standard threshold is The critical value of abnormal posture. In the embodiment of this application, the gait data of the target object is processed and analyzed by the convolutional neural network and the deep learning network, and the matching parameters between the human body information obtained after the processing and the preset standard human body information are calculated through the matching parameters Determine the posture of the target object. The gait data of the target object is processed through the convolutional neural network and the deep learning network, which reduces the detection cost and improves the detection efficiency of detecting the body condition of the target object.

Description of the drawings

FIG. 1 is a schematic diagram of an embodiment of a posture detection method based on gait features in an embodiment of the application;

FIG. 2 is a schematic diagram of another embodiment of a posture detection method based on gait features in an embodiment of the application;

FIG. 3 is a schematic diagram of an embodiment of a posture detection device based on gait features in an embodiment of the application;

FIG. 4 is a schematic diagram of another embodiment of a posture detection device based on gait features in an embodiment of the application;

Fig. 5 is a schematic diagram of an embodiment of a posture detection device based on gait features in an embodiment of the application.

Detailed ways

The embodiments of the present application provide a posture detection method, device, device and storage medium based on gait characteristics, which are used to reconstruct a new complex relationship network based on the community characteristics in the traditional complex relationship network as an intermediate variable. The community weighted graph is used to associate the originally unrelated community groups with similarity, and the label propagation algorithm is used to identify the risk of the community weighted graph to obtain the spread risk value of the unrelated communities, and realize the risk spread between unrelated communities Analyzing the situation has enhanced the ability to identify and control the risks of community groups.

In order to enable those skilled in the art to better understand the solution of the present application, the embodiments of the present application will be described below in conjunction with the accompanying drawings in the embodiments of the present application.

The terms "first", "second", "third", "fourth", etc. (if any) in the description and claims of this application and the above-mentioned drawings are used to distinguish similar objects, without having to use To describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments described herein can be implemented in a sequence other than those illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those clearly listed. Steps or units, but may include other steps or units that are not clearly listed or are inherent to these processes, methods, products, or equipment.

For ease of understanding, the following describes the specific process of the embodiment of the present application. Please refer to FIG. 1. An embodiment of the posture detection method based on gait features in the embodiment of the present application includes:

In one embodiment, the method for detecting posture based on gait features includes:

101. Obtain the gait data of the target object, and detect the key points of the human body of the target object in the gait data to obtain multiple key point features, and the human body key points are multiple coordinate points of the human skeleton;

It is understandable that the execution subject of this application may be a posture detection device based on gait characteristics, or may also be a terminal or a server, which is not specifically limited here. The embodiment of the present application takes the server as the execution subject as an example for description.

The server obtains the gait data of the target object, and after processing the gait data, it detects the key points of the target object's human body in the processed gait data, and then obtains multiple key point features, where the human body key points are the target Multiple coordinate points in the human skeleton of the object, these coordinate points are indispensable representative points in the human skeleton.

It should be noted that the gait data of the target object acquired by the server refers to the gait video data of the target object's continuous walking. The body information of the target object in the video data is analyzed to determine the body state of the target object. The server uses convolutional neural network (convolutional neural network, CNN) to detect the human body key points of the target object in the gait data, and obtains multiple key feature points. CNN is a type of feedforward neural network that includes convolutional calculations and has a deep structure. The network is one of the representative algorithms of deep learning. CNN has the ability to characterize learning, and can classify the input information according to its hierarchical structure. According to this ability of CNN, it can realize the processing of gait data, and then obtain the gait feature map, so that the server can perform the gait feature The key points of the human body are detected in the figure.

102. Use a deep learning network to perform feature classification and positioning regression on multiple key point features to obtain human body information. The human body information includes the position of the human body part and the affinity vector of the human body part. The affinity vector of the human body part is used to connect two different key points feature;

The server uses a deep learning network to perform feature classification and positioning regression on multiple key point features, determine the key point feature category, and obtain the human body information of the target object. The human body information includes the position of the human body and the affinity vector of the human body. The position of the human body is Multiple, human body part affinity vectors are used to connect two different key point features.

The server uses a bottom-up human body key point detection algorithm, which mainly includes two parts, key point classification and key point positioning regression. The key point detection requires the server to calculate the difference between all key point features and preset human position features. Confidence degree, to identify the category of key point features and determine the position of the corresponding body part represented by the key point feature, and then the server calculates the affinity between different key point features, and connects different key point features according to the degree of affinity Together to determine the human body part affinity vector. The human body part affinity vector is the basis of the human skeleton. Multiple human body part affinity vectors can be connected together to form a human skeleton. The human skeleton here depicts the general motion of the human body. In form, the server obtains the detected human body information from the human body part position and the human body part affinity vector.

After the server has determined the classification of all the key point features, it needs to perform positioning regression on the key point features, that is, connect the different key point features with high affinity to obtain the human body part affinity vector. The affinity here refers to the degree of association between two different key point features. When the degree of association between two different key point features is greater than the second threshold, it indicates the correlation between the two different key point features The greater the degree, the two different key point features are connected to form a human body part affinity vector, and multiple human body part affinity vectors are connected to form a human body skeleton. A key point feature can be connected to a different number of other key point features. For example, the hand key point feature is only connected to the elbow key point feature, and the shoulder key point feature is connected to the elbow key point feature and the torso key point feature respectively.

103. Calculate the matching parameters between the human body information and the preset standard human body information based on multiple key point features. The preset standard human body information includes a preset standard affinity vector, and the matching parameters are used to indicate the body information of the target object and the preset standard The degree of matching between human body information;

The server calculates the matching parameters between the human body information and the preset standard human body information according to the relationship between multiple key point features. The preset standard human body information includes the preset standard affinity vector, which is a large amount of data It is input into the model and calculated, and the matching parameter is used to indicate the degree of matching between the human body information of the target object and the preset standard human body information.

After the server calculates the human body part affinity vectors of multiple key point features, it connects the multiple human body part affinity vectors to obtain a human body posture skeleton. The human body posture skeleton is compared with the standard posture skeleton. Analysis, the server can get the posture of the target object. The posture skeleton of the human body is compared with the standard posture skeleton. In essence, it is a comparison between the affinity vector of multiple human body parts at the same position and the preset standard affinity vector. The affinity vector of the human body part constitutes the posture skeleton of the human body. The preset standard affinity vector is the basis of the standard posture skeleton. The standard affinity vector is preset here, which is obtained through continuous calculation and training of a large number of human posture samples. Human postures of different heights and weights have different The standard posture skeleton and preset standard affinity vector.

104. Determine the posture of the target object according to the matching parameters and the standard threshold, and the standard threshold is the critical value of the abnormal posture.

The server is to calculate the confidence between the human body part affinity vector and the preset standard affinity vector, and after obtaining the matching parameters, compare the matching parameters with the standard thresholds of different posture conditions to determine the posture status of the target object.

The standard thresholds of posture conditions here are used to illustrate the critical values of different posture conditions, and the standard thresholds are calculated through a large amount of data. For example, the standard threshold for cervical spine abnormality is 0.8, which means that when the matching parameter is greater than the standard threshold 0.8, the posture of the target object is cervical spine abnormality. In addition, there should be at least one standard threshold. Each standard threshold represents the critical value of different posture conditions. The standard threshold and the matching parameter correspond to each other, that is to say, the body part affinity vector for calculating the matching parameter must be the posture represented by the standard threshold. The situation is relevant. For example, the standard threshold of cervical spine abnormality is 0.8, and the key point features of the human body part affinity vector that form the matching parameters should be related to the neck, which can be the head key point feature, the shoulder key point feature, and the trunk key point feature. Use different human body parts affinity vectors to calculate multiple matching parameters to obtain a more accurate body condition of the target object.

It is understandable that the posture of the target object obtained by the server here can be various, such as: the standard threshold for minor cervical spine abnormality is 0.65, that is to say, when the matching parameter is greater than the standard threshold 0.65, the posture of the target object is mild cervical spine. Abnormal; such as: the standard threshold of severe cervical spine abnormality is 0.89, that is to say, when the matching parameter is greater than the standard threshold 0.89, the posture of the target object is severe cervical spine abnormality. Different human body part affinity vectors can be obtained through different key point features. The server can then compare the different human body part affinity vectors with the preset standard affinity vector to get the target object compared with the standard health condition. Abnormal posture.

In the embodiment of the application, the artificial convolutional neural network and the deep learning network are used to process and analyze the gait data of the target object, and to calculate the matching parameters between the human body information obtained after the processing and the preset standard human body information, Determine the posture of the target object through matching parameters. At the same time, this application can also be applied in the field of smart medical care to promote the construction of smart cities. This application uses convolutional neural networks and deep learning networks to process the gait data of the target object, reducing the cost of detection and improving the physical condition of the detected target object The detection efficiency.

Please refer to FIG. 2, another embodiment of the posture detection method based on gait features in the embodiment of the present application includes:

201. Calculate preset standard human body information, which includes preset standard affinity vectors;

The server uses a deep learning network and a convolutional neural network to calculate preset standard human body information, where the preset standard human body information includes a preset standard affinity vector.

It is understandable that the preset standard human body information here includes body information under standard health conditions and human body information suffering from different degrees of cervical and lumbar spine diseases. The preset standard human body information is the server collecting a large amount of human body information data, and a large number of The human body information data is obtained by deep learning training, and the feature extraction layer of different abstract features of the gait data is extracted through the deep learning network, and the feature classification and positioning regression of the key points of the human body are performed, and the relevant loss function is used to perform the training results. Evaluation, and finally update the network parameters through backpropagation, repeat this process until the network converges, and get the preset standard human body information.

It should be noted that the method for the server to calculate the preset standard human body information is the same as that of calculating the human body information. Both use the convolutional neural network to process the gait data, and then use the deep learning network to classify the key points of the human body. Positioning regression, so as to obtain human body information, the data results obtained in this way are more accurate and representative. The specific calculation steps are described in the following steps, and will not be repeated here.

202. Obtain the gait data of the target object, and detect the key points of the human body of the target object in the gait data to obtain multiple key point features, where the key points of the human body are multiple coordinate points of the human skeleton;

The server obtains the gait data of the target object, and after processing the gait data, it detects the key points of the target object's human body in the processed gait data, and then obtains multiple key point features, where the human body key points are the target Multiple coordinate points in the human skeleton of the object, these coordinate points are indispensable representative points in the human skeleton. specific:

The server normalizes the gait data of the target object to obtain the basic processing data; in the basic processing data, the server uses the convolutional neural network to detect the human body key points of the target object, and obtains multiple key point features. The human body key points are Multiple coordinate points of the human skeleton. Specifically: First, the server uses a convolutional neural network to calculate the convolution of the basic processing data to obtain the first processed data; second, the server performs down-sampling processing on the first processed data, and extracts multiple sampling vectors in the first processed data to obtain The second processing data; then the server performs non-linear mapping of the second processing data to obtain the gait feature map; finally the server detects the human body key points of the target object in the gait feature map, and obtains multiple key point features, the human body key points are Multiple coordinate points of the human skeleton.

The gait data of the target object acquired by the server refers to the gait video data of the target object's continuous walking. The body information of the target object in the video data is analyzed to determine the body state of the target object. It is understandable that before the server detects the key points of the target object's human body through the gait data, the server needs to normalize the gait data of the target object. The server normalizes the video data, which is beneficial to the next step. One step. The server obtains the pixel value on the gait data, and the server normalizes the pixel value to obtain the basic processing data. The normalization does not change the contrast of the image, and at the same time, it ensures that the pixel values of all pictures after normalization are in the range of [0, 1]. The formula used is as follows:

In the formula: a'is the pixel value of the basic processing data, and a is the original pixel value of the gait data. It is understandable that by normalizing the gait data of the target object, mapping the data to the range of 0 to 1, and then inputting the obtained basic processing data into the network model, the server calculation is more convenient and faster.

After obtaining the basic processing data, the server uses the convolutional neural network CNN to detect the human body key points of the target object in the gait data, and obtains multiple key feature points. CNN is a type of feedforward that includes convolution calculations and has a deep structure. Neural network is one of the representative algorithms of deep learning. CNN has the ability to characterize learning, and can classify the input information according to its hierarchical structure. According to this ability of CNN, it can realize the processing of gait data, and then obtain the gait feature map, so that the server can perform the gait feature The key points of the human body are detected in the figure. CNN contains a feature extractor composed of a convolutional layer and a sub-sampling layer. The feature extractor is used to manipulate different parameters to repeatedly process the basic processing data to extract features at different levels.

In the convolutional layer of the convolutional neural network, the server calculates the convolution of the basic processing data. In a convolutional layer of CNN, it usually contains several feature planes. Each feature plane is composed of some rectangularly arranged neurons. The neurons in the feature plane share weights, and the shared weights here are the convolution kernels. The convolution kernel is generally initialized in the form of a random decimal matrix. During the training of the network, the convolution kernel will learn to obtain reasonable weights, that is, the server calculates the convolution of the basic processing data, and then obtains the first processing data; the server is getting After the first processed data, down-sampling is performed on the first processed data. Down-sampling is also called pooling. It can be regarded as a special convolution process. The purpose is to reduce the size of the feature map and perform down-sampling. After that, multiple sampling vectors are extracted to obtain the second processed data; then the server uses the excitation function to perform non-linear mapping on the second processed data, thereby obtaining the gait feature map, which can display different shapes in the gait feature map Feature points: Finally, the server screens out the key points of the target object in the gait feature map, and obtains multiple key point features. The key points of the human body here are the coordinate points that specifically represent the combination of the human skeleton. You can pass multiple key feature points. Obtain the human body information of the target object, and better determine the body condition of the target object.

203. Use the deep learning network to perform feature classification and positioning regression on multiple key point features to obtain human body information. The human body information includes the position of the human body part and the affinity vector of the human body part. The affinity vector of the human body part is used to connect two different key points feature;

The server uses a deep learning network to perform feature classification and positioning regression on multiple key point features, determine the key point feature category, and obtain the human body information of the target object. The human body information includes the position of the human body and the affinity vector of the human body. The position of the human body is Multiple, human body part affinity vectors are used to connect two different key point features. specific:

The server first uses a deep learning network to calculate the confidence between multiple key point features and preset human body position features, thereby obtaining multiple key point feature categories. The preset human body position features are used to indicate the positions of different human body parts, and Each preset human body position feature corresponds to a human body part position; secondly, the server compares the confidence level with the first threshold, and when the confidence level is greater than the first threshold, determines the human body part position corresponding to the key point feature for calculating the confidence level, the key point The position of the human body part corresponding to the feature is the position of the human body part corresponding to the preset human body position feature; then the server calculates the affinity between the two different key point features, and performs positioning regression on the two different key point features; the server then compares the relative The degree of sum is compared with the second threshold. When the degree of affinity is greater than the second threshold, two different key point features are connected. The corresponding two different key point features are two different key point features for calculating the affinity. Generate the human body part affinity vector, where the human body part affinity vector is used to connect two different key point features; finally the server connects multiple human body part affinity vectors to obtain human body information, which includes the position of the human body part and the human body part Affinity vector.

It should be noted that the preset human body position features here refer to the key points of the standard human skeleton, and there are many preset human body position features, such as: standard head (5), shoulders (2), There are 18 torso (1), elbows (2), hands (2), hips (2), knees (2), and feet (2). The server presets the position of the human body through calculation. The confidence level between the feature and the identified key point feature is used to determine the classification of the key point feature. When the confidence level between the key point feature and the preset human position feature is greater than the first threshold, it indicates that the key point feature and the The preset human body position feature fits, and the server sets the classification of the key point feature as a classification corresponding to the preset human body position feature. In addition, the value of the first threshold here can be set differently according to actual conditions, which is not limited in this application.

For example, the server sets the first threshold to 0.6, the server calculates the confidence level between the key point feature and the preset elbow feature to be 0.3, and then calculates the confidence level between the key point feature and the preset knee feature to be 0.8 , And the confidence level between the key point feature and other preset human body position features does not exceed 0.8, indicating that the key point feature fits with the preset knee feature, and the server determines that the key point feature is the preset knee feature.

For example, the server sets the second threshold value to 0.55, the server calculates the affinity between the torso key point feature and the shoulder key point feature as 0.88, and the affinity between the torso key point feature and the foot key point feature is 0.15 , The affinity between the torso key point feature and the head key point feature is 0.75. When the calculated affinity is greater than the second preset 0.55, the server will connect the two key point features of the calculated affinity together , That is, the torso key point feature is connected with the shoulder key point feature and the head key point feature respectively to obtain the human body part affinity vector.

204. Calculate the matching parameters between the human body information and the preset standard human body information based on multiple key point features. The preset standard human body information includes a preset standard affinity vector, and the matching parameters are used to indicate the body information of the target object and the preset standard The degree of matching between human body information;

Specifically: the server first obtains multiple human body part affinity vectors of the target object; then the server obtains preset standard human body information that matches the target object, and the preset standard human body information includes multiple different preset standard affinity vectors; and finally The server uses the similarity algorithm to calculate the confidence between the human body part affinity vector and the corresponding preset standard affinity vector to obtain the matching parameters. The two key point features in the human body part affinity vector are in the preset standard affinity vector The categories of the preset body position features are the same, and the matching parameters are used to indicate the degree of matching between the body information of the target object and the preset standard body information.

After the server calculates the human body part affinity vectors of multiple key point features, it connects the multiple human body part affinity vectors to obtain a human body posture skeleton. The human body posture skeleton is compared with the standard posture skeleton. Analysis, the server can get the posture of the target object. The posture skeleton of the human body is compared with the standard posture skeleton. In essence, it is a comparison between the affinity vector of multiple human body parts at the same position and the preset standard affinity vector. The affinity vector of the human body part constitutes the posture skeleton of the human body. The preset standard affinity vector is the basis of the standard posture skeleton. The standard affinity vector is preset here, which is obtained through continuous calculation and training of a large number of human posture samples. Human postures of different heights and weights have different The standard posture skeleton and the preset standard affinity vector, so when the matching parameter is compared with the standard threshold, the preset standard human body information for calculating the matching parameter must be the data that the target object wants to match.

It should be noted that when using the similarity algorithm to calculate the confidence between the human body part affinity vector and the preset standard affinity vector, the two key point features in the human body part affinity vector are compared with the preset standard affinity vector. The categories of the preset human body position features are the same, that is, the two key feature points constituting the human body part affinity vector are the same as the two preset human body position characteristics constituting the preset standard affinity vector.

For example, take the comparison of the human body part affinity vector between the head key point feature and the torso key point of the target object as an example. First, the server obtains the human body part affinity vector between the head key point feature and the torso key point feature , Hereinafter referred to as the first affinity vector, and secondly, the server will filter out the preset standard human body information that matches the target object. The preset standard human body information that matches the target object here refers to the standard health Based on the human body posture skeleton information, the preset standard affinity vector between the standard head key point feature and the standard torso key point feature is obtained from the preset standard human body information, which is hereinafter referred to as the second affinity vector, and finally , The server will calculate the confidence level between the first affinity vector and the second affinity vector, that is, calculate the degree of matching between the first affinity vector and the second affinity vector, and obtain a matching parameter.

205. Determine the posture of the target object according to the matching parameters and the standard threshold, where the standard threshold is the critical value of the abnormal posture.

The server is to calculate the confidence between the human body part affinity vector and the preset standard affinity vector, and after obtaining the matching parameters, compare the matching parameters with the standard thresholds of different posture conditions to determine the posture status of the target object. Specifically: the server determines whether the configuration parameter is greater than the standard threshold, and the standard threshold is the critical value of the abnormal posture; if the configuration parameter is greater than the standard threshold, the server determines that the posture of the target object is abnormal.

Here, the standard threshold of posture conditions is used to illustrate the critical value of different posture conditions, and the standard threshold is calculated through a large amount of data. For example, the standard threshold for cervical spine abnormality is 0.8, which means that when the matching parameter is greater than the standard threshold 0.8, the posture of the target object is cervical spine abnormality. In addition, there should be at least one standard threshold. Each standard threshold represents the critical value of different posture conditions. The standard threshold and the matching parameter correspond to each other, that is to say, the body part affinity vector for calculating the matching parameter must be the posture represented by the standard threshold. The situation is relevant. For example, the standard threshold of cervical spine abnormality is 0.8, and the key point features of the human body part affinity vector that form the matching parameters should be related to the neck, which can be the head key point feature, the shoulder key point feature, and the trunk key point feature. Use different human body parts affinity vectors to calculate multiple matching parameters to obtain a more accurate body condition of the target object.

In the embodiment of this application, the gait data of the target object is processed and analyzed by the convolutional neural network and the deep learning network, and the matching parameters between the human body information obtained after the processing and the preset standard human body information are calculated through the matching parameters Determine the posture of the target object. The gait data of the target object is processed through the convolutional neural network and the deep learning network, which reduces the detection cost and improves the detection efficiency of detecting the body condition of the target object.

The posture detection method based on gait features in the embodiments of the present application is described above, and the posture detection device based on gait features in the embodiments of the present application is described below. Please refer to FIG. 3. An embodiment of the posture detection device includes:

The detection module 301 is used to obtain the gait data of the target object, and detect the key points of the human body of the target object in the gait data to obtain multiple key point features, and the human body key points are multiple coordinate points of the human skeleton;

The classification and regression module 302 is used for feature classification and positioning regression of multiple key point features based on the deep learning network to obtain human body information. The human body information includes the position of the body part and the affinity vector of the body part. The affinity vector of the body part is used for connection Two different key point features;

The first calculation module 303 is configured to calculate matching parameters between the human body information and preset standard human body information based on multiple key point features. The preset standard human body information includes a preset standard affinity vector, and the matching parameters are used to indicate the target object's The degree of matching between the body information and the preset standard body information;

The determining module 304 is configured to determine the posture condition of the target object according to the matching parameters and the standard threshold value, and the standard threshold value is the critical value of the abnormal posture condition.

Please refer to FIG. 4, another embodiment of a body posture detection device based on gait features in the embodiment of the present application includes:

Optionally, the classification and regression module 302 can also be specifically used for:

The deep learning network is used to calculate the confidence between multiple key point features and preset human body position features, and the multiple key point features are classified. The preset human body position features are used to indicate the position of different body parts, and each preset The human body position feature corresponds to the position of a human body part;

When the confidence is greater than the first threshold, determine the position of the human body part corresponding to the key point feature for calculating the confidence, and the position of the human body part corresponding to the key point feature is the position of the human body part corresponding to the preset human position feature;

Calculate the affinity between two different key point features, and perform positioning regression on two different key point features;

When the affinity is greater than the second threshold, connect two different key point features to generate a human body part affinity vector, and the human body part affinity vector is used to connect two different key point features;

Connect multiple human body part affinity vectors to obtain human body information. The human body information includes the position of the human body part and the human body part affinity vector.

Optionally, the first calculation module 303 may also be specifically used for:

Acquire affinity vectors of multiple human body parts of the target object;

Obtain preset standard human body information that matches the target object, and the preset standard human body information includes a preset standard affinity vector;

The similarity algorithm is used to calculate the confidence between the human body part affinity vector and the corresponding preset standard affinity vector, and the matching parameters are obtained. The two key point features in the human body part affinity vector are compared with those in the preset standard affinity vector. The categories of the preset body position features are the same, and the matching parameters are used to indicate the degree of matching between the body information of the target object and the preset standard body information.

Optionally, the determining module 304 may also be specifically used for:

Judge whether the configuration parameter is greater than the standard threshold, the standard threshold is the critical value of abnormal posture;

If the configuration parameter is greater than the standard threshold, it is determined that the posture of the target object is abnormal.

Optionally, the detection module 301 includes:

The processing unit 3011 is used to obtain the gait data of the target object, and to normalize the gait data to obtain basic processing data;

The detection unit 3012 is used for detecting the key points of the human body of the target object by using the convolutional neural network in the basic processing data to obtain multiple key point features, and the key points of the human body are multiple coordinate points of the human skeleton.

Optionally, the detection unit 3012 may also be specifically configured to:

Use convolutional neural network to calculate the convolution of the basic processing data to obtain the first processing data;

Perform down-sampling processing on the first processed data, and extract multiple sampling vectors in the first processed data to obtain second processed data;

Perform nonlinear mapping on the second processed data to obtain a gait feature map;

The human body key points of the target object are detected in the gait feature map, and multiple key point features are obtained. The human body key points are multiple coordinate points of the human skeleton.

Optionally, the posture detection device based on gait features further includes:

The second calculation module 305 is used to calculate preset standard human body information, and the preset standard human body information includes a preset standard affinity vector.

Figures 3 and 4 above describe in detail the posture detection device based on gait features in the embodiment of the present application from the perspective of modular functional entities. The following describes the posture detection device based on gait features in the embodiment of the present application from the perspective of hardware processing. Give a detailed description.

FIG. 5 is a schematic structural diagram of a posture detection device based on gait features provided by an embodiment of the present application. The posture detection device 500 based on gait features may have relatively large differences due to different configurations or performances, and may include one or One or more central processing units (CPU) 510 (for example, one or more processors) and memory 520, one or more storage media 530 for storing application programs 533 or data 532 (for example, one or one storage device with a large amount of storage ). Among them, the memory 520 and the storage medium 530 may be short-term storage or persistent storage. The program stored in the storage medium 530 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the body posture detection device 500 based on gait characteristics. Further, the processor 510 may be configured to communicate with the storage medium 530, and execute a series of instruction operations in the storage medium 530 on the posture detection device 500 based on the gait characteristics.

The gait feature-based posture detection device 500 may also include one or more power sources 540, one or more wired or wireless network interfaces 550, one or more input and output interfaces 560, and/or, one or more operating systems 531 , Such as Windows Serve, Mac OS X, Unix, Linux, FreeBSD and so on. Those skilled in the art can understand that the structure of the posture detection device based on gait features shown in FIG. 5 does not constitute a limitation on the posture detection device based on gait features, and may include more or less components than shown in the figure, or Combining certain components, or different component arrangements.

This application also provides a computer-readable storage medium. The computer-readable storage medium may be a non-volatile computer-readable storage medium. The computer-readable storage medium may also be a volatile computer-readable storage medium. Instructions are stored in the read storage medium, and when the instructions are run on the computer, the computer executes the steps of the posture detection method based on gait characteristics.

Further, the computer-readable storage medium may mainly include a storage program area and a storage data area, where the storage program area may store an operating system, an application program required by at least one function, etc.; the storage data area may store Data created by the use of nodes, etc.

The present application also provides a posture detection device based on gait features, including: a memory and at least one processor, the memory stores instructions, and the memory and the at least one processor are interconnected by wires; the at least one processor A processor calls the instructions in the memory, so that the intelligent path planning device executes the steps in the above-mentioned posture detection method based on gait characteristics.

The present application also provides a computer-readable storage medium. The computer-readable storage medium may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. The computer-readable storage medium stores computer instructions, and when the computer instructions are executed on the computer, the computer executes the following steps:

Acquiring the gait data of the target object, and detecting the human body key points of the target object in the gait data to obtain multiple key point features, and the human body key points are multiple coordinate points of the human skeleton;

The deep learning network is used to perform feature classification and positioning regression on the multiple key point features to obtain human body information. The human body information includes the position of the human body part and the affinity vector of the human body part, and the human body part affinity vector is used to connect two Different key point characteristics;

The matching parameters between the human body information and preset standard human body information are calculated based on the multiple key point features, the preset standard human body information includes a preset standard affinity vector, and the matching parameter is used to indicate the target object’s The degree of matching between the body information and the preset standard body information;

The posture condition of the target object is determined according to the matching parameter and a standard threshold value, where the standard threshold value is a critical value for an abnormal posture condition.

Those skilled in the art can clearly understand that, for the convenience and conciseness of the description, the specific working process of the system, device and unit described above can refer to the corresponding process in the foregoing method embodiment, which will not be repeated here.

If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solution of the present application essentially or the part that contributes to the existing technology or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , Including several instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods in the various embodiments of the present application. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (read-only memory, ROM), random access memory (random access memory, RAM), magnetic disks or optical disks and other media that can store program codes. .

The blockchain referred to in this application is a new application mode of computer technology such as distributed data storage, point-to-point transmission, consensus mechanism, and encryption algorithm. Blockchain, essentially a decentralized database, is a series of data blocks associated with cryptographic methods. Each data block contains a batch of network transaction information for verification. The validity of the information (anti-counterfeiting) and the generation of the next block. The blockchain can include the underlying platform of the blockchain, the platform product service layer, and the application service layer.

Above, the above embodiments are only used to illustrate the technical solutions of the present application, not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, a person of ordinary skill in the art should understand that it can still implement the foregoing various implementations. The technical solutions recorded in the examples are modified, or some of the technical features are equivalently replaced; these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

A posture detection method based on gait features, which includes:

Acquiring the gait data of the target object, and detecting the human body key points of the target object in the gait data to obtain multiple key point features, and the human body key points are multiple coordinate points of the human skeleton;

The deep learning network is used to perform feature classification and positioning regression on the multiple key point features to obtain human body information. The human body information includes the position of the human body part and the affinity vector of the human body part, and the human body part affinity vector is used to connect two Different key point characteristics;

The matching parameters between the human body information and preset standard human body information are calculated based on the multiple key point features, the preset standard human body information includes a preset standard affinity vector, and the matching parameter is used to indicate the target object's The degree of matching between the body information and the preset standard body information;

The posture condition of the target object is determined according to the matching parameter and a standard threshold value, where the standard threshold value is a critical value for an abnormal posture condition.
The method for body posture detection based on gait features according to claim 1, wherein the deep learning network is used to perform feature classification and positioning regression on the multiple key point features to obtain human body information, and the human body information includes body parts Position and human body part affinity vector, where the human body part affinity vector is used to connect two different key point features including:

A deep learning network is used to calculate the confidence between multiple key point features and preset human body position features, and the multiple key point features are classified. The preset human body position features are used to indicate the positions of different human body parts, and Each preset human body position feature corresponds to a human body part position;

When the confidence level is greater than the first threshold, determining the human body part position corresponding to the key point feature for calculating the confidence level, and the human body part position corresponding to the key point feature is the human body part position corresponding to the preset human body position feature;

Calculate the affinity between two different key point features, and perform positioning regression on the two different key point features;

When the affinity is greater than the second threshold, connecting the two different key point features to generate a body part affinity vector, where the body part affinity vector is used to connect the two different key point features;

A plurality of the human body part affinity vectors are connected to obtain human body information, and the human body information includes the position of the human body part and the human body part affinity vector.
The method for body posture detection based on gait features according to claim 2, wherein the calculation of the matching parameters between the body information and preset standard body information based on the plurality of key point features, the preset standard The human body information includes a preset standard affinity vector, and the matching parameter is used to indicate the degree of matching between the body information of the target object and the preset standard human body information including:

Obtain the affinity vector of the human body part of the target object;

Acquiring preset standard human body information matching the target object, where the preset standard human body information includes a plurality of preset standard affinity vectors;

The similarity algorithm is used to calculate the confidence between the human body part affinity vector and the corresponding preset standard affinity vector to obtain matching parameters. The two key point features in the human body part affinity vector are compared with the preset The categories of the preset body position features in the standard affinity vector are the same, and the matching parameter is used to indicate the degree of matching between the body information of the target object and the preset standard body information.
The gait feature-based posture detection method according to claim 1, wherein the determining the posture condition of the target object according to the matching parameter and a standard threshold value, the standard threshold value being the critical value of the abnormal posture condition comprises :

Judging whether the configuration parameter is greater than a standard threshold, the standard threshold being the critical value of the abnormal posture;

If the configuration parameter is greater than the standard threshold, it is determined that the posture of the target object is abnormal.
The gait feature-based posture detection method according to any one of claims 1 to 4, wherein the gait data of the target object is acquired, and the human body of the target object is critical in the gait data. Points are detected to obtain multiple key point features. The human body key points are multiple coordinate points of the human body skeleton including:

Obtain gait data of the target object, and normalize the gait data to obtain basic processing data;

In the basic processing data, a convolutional neural network is used to detect the human body key points of the target object to obtain multiple key point features, and the human body key points are multiple coordinate points of the human body skeleton.
The method for body posture detection based on gait features according to claim 5, wherein, in the basic processing data, a convolutional neural network is used to detect the human body key points of the target object to obtain a plurality of key point features, The key points of the human body are multiple coordinate points of the skeleton of the human body including:

Using a convolutional neural network to calculate the convolution of the basic processing data to obtain the first processing data;

Performing down-sampling processing on the first processed data, and extracting multiple sampling vectors in the first processed data to obtain second processed data;

Performing nonlinear mapping on the second processed data to obtain a gait feature map;

The human body key points of the target object are detected in the gait feature map to obtain multiple key point features, and the human body key points are multiple coordinate points of the human body skeleton.
The gait feature-based posture detection method according to any one of claims 1-6, wherein the gait data of the target object is acquired, and the human body of the target object is measured in the gait data. Key points are detected to obtain multiple key point features, where the key points of the human body are before multiple coordinate points of the human skeleton, and further include:

Calculate preset standard human body information, where the preset standard human body information includes a preset standard affinity vector.
A posture detection device based on gait features, including a memory, a processor, and computer-readable instructions stored on the memory and capable of running on the processor. When the processor executes the computer-readable instructions To achieve the following steps:

Acquiring the gait data of the target object, and detecting the human body key points of the target object in the gait data to obtain multiple key point features, and the human body key points are multiple coordinate points of the human skeleton;

The deep learning network is used to perform feature classification and positioning regression on the multiple key point features to obtain human body information. The human body information includes the position of the human body part and the affinity vector of the human body part, and the human body part affinity vector is used to connect two Different key point characteristics;

The matching parameters between the human body information and preset standard human body information are calculated based on the multiple key point features, the preset standard human body information includes a preset standard affinity vector, and the matching parameter is used to indicate the target object's The degree of matching between the body information and the preset standard body information;

The posture condition of the target object is determined according to the matching parameter and a standard threshold value, where the standard threshold value is a critical value for an abnormal posture condition.
According to the gait feature-based posture detection device according to claim 8, the processor further implements the following steps when executing the computer program:

A deep learning network is used to calculate the confidence between multiple key point features and preset human body position features, and the multiple key point features are classified. The preset human body position features are used to indicate the positions of different human body parts, and Each preset human body position feature corresponds to a human body part position;

When the confidence level is greater than the first threshold, determining the human body part position corresponding to the key point feature for calculating the confidence level, and the human body part position corresponding to the key point feature is the human body part position corresponding to the preset human body position feature;

Calculate the affinity between two different key point features, and perform positioning regression on the two different key point features;

When the affinity is greater than the second threshold, connecting the two different key point features to generate a body part affinity vector, where the body part affinity vector is used to connect the two different key point features;

A plurality of the human body part affinity vectors are connected to obtain human body information, and the human body information includes the position of the human body part and the human body part affinity vector.
According to the gait feature-based posture detection device according to claim 9, the processor further implements the following steps when executing the computer program:

Obtain the affinity vector of the human body part of the target object;

Acquiring preset standard human body information matching the target object, where the preset standard human body information includes a plurality of preset standard affinity vectors;

The similarity algorithm is used to calculate the confidence between the human body part affinity vector and the corresponding preset standard affinity vector to obtain matching parameters. The two key point features in the human body part affinity vector are compared with the preset The categories of the preset body position features in the standard affinity vector are the same, and the matching parameter is used to indicate the degree of matching between the body information of the target object and the preset standard body information.
According to the gait feature-based posture detection device according to claim 8, the processor further implements the following steps when executing the computer program:

Judging whether the configuration parameter is greater than a standard threshold, the standard threshold being the critical value of the abnormal posture;

If the configuration parameter is greater than the standard threshold, it is determined that the posture of the target object is abnormal.
According to the gait feature-based posture detection device according to claim 8, the processor further implements the following steps when executing the computer program:

Obtain gait data of the target object, and normalize the gait data to obtain basic processing data;

In the basic processing data, a convolutional neural network is used to detect the human body key points of the target object to obtain multiple key point features, and the human body key points are multiple coordinate points of the human body skeleton.
According to the gait feature-based posture detection device according to claim 12, the processor further implements the following steps when executing the computer program:

Using a convolutional neural network to calculate the convolution of the basic processing data to obtain the first processing data;

Performing down-sampling processing on the first processed data, and extracting multiple sampling vectors in the first processed data to obtain second processed data;

Performing nonlinear mapping on the second processed data to obtain a gait feature map;

The human body key points of the target object are detected in the gait feature map to obtain multiple key point features, and the human body key points are multiple coordinate points of the human body skeleton.
According to the gait feature-based posture detection device according to any one of claims 8-13, the processor further implements the following steps when executing the computer program:

Calculate preset standard human body information, where the preset standard human body information includes a preset standard affinity vector.
A computer-readable storage medium in which computer instructions are stored, and when the computer instructions are executed on a computer, the computer executes the following steps:

Acquiring the gait data of the target object, and detecting the human body key points of the target object in the gait data to obtain multiple key point features, and the human body key points are multiple coordinate points of the human skeleton;

The deep learning network is used to perform feature classification and positioning regression on the multiple key point features to obtain human body information. The human body information includes the position of the human body part and the affinity vector of the human body part, and the human body part affinity vector is used to connect two Different key point characteristics;

The matching parameters between the human body information and preset standard human body information are calculated based on the multiple key point features, the preset standard human body information includes a preset standard affinity vector, and the matching parameter is used to indicate the target object's The degree of matching between the body information and the preset standard body information;

The posture condition of the target object is determined according to the matching parameter and a standard threshold value, where the standard threshold value is a critical value for an abnormal posture condition.
The computer-readable storage medium according to claim 15, when the computer instructions are executed on the computer, the computer is caused to further perform the following steps:

A deep learning network is used to calculate the confidence between multiple key point features and preset human body position features, and the multiple key point features are classified. The preset human body position features are used to indicate the positions of different human body parts, and Each preset human body position feature corresponds to a human body part position;

When the confidence level is greater than the first threshold, determining the human body part position corresponding to the key point feature for calculating the confidence level, and the human body part position corresponding to the key point feature is the human body part position corresponding to the preset human body position feature;

Calculate the affinity between two different key point features, and perform positioning regression on the two different key point features;

When the affinity is greater than the second threshold, connecting the two different key point features to generate a body part affinity vector, where the body part affinity vector is used to connect the two different key point features;

A plurality of the human body part affinity vectors are connected to obtain human body information, and the human body information includes the position of the human body part and the human body part affinity vector.
The computer-readable storage medium according to claim 16, when the computer instructions are executed on the computer, the computer is caused to further perform the following steps:

Obtain the affinity vector of the human body part of the target object;

Acquiring preset standard human body information matching the target object, where the preset standard human body information includes a plurality of preset standard affinity vectors;

The similarity algorithm is used to calculate the confidence between the human body part affinity vector and the corresponding preset standard affinity vector to obtain matching parameters. The two key point features in the human body part affinity vector are compared with the preset The categories of the preset body position features in the standard affinity vector are the same, and the matching parameter is used to indicate the degree of matching between the body information of the target object and the preset standard body information.
The computer-readable storage medium according to claim 15, when the computer instructions are executed on the computer, the computer is caused to further perform the following steps:

Judging whether the configuration parameter is greater than a standard threshold, the standard threshold being the critical value of the abnormal posture;

If the configuration parameter is greater than the standard threshold, it is determined that the posture of the target object is abnormal.
The computer-readable storage medium according to claim 15, when the computer instructions are executed on the computer, the computer is caused to further perform the following steps:

Obtain gait data of the target object, and normalize the gait data to obtain basic processing data;

In the basic processing data, a convolutional neural network is used to detect the human body key points of the target object to obtain multiple key point features, and the human body key points are multiple coordinate points of the human body skeleton.
A posture detection device based on gait features, wherein the posture detection based on gait features includes:

The detection module is used to obtain the gait data of the target object, and detect the key points of the human body of the target object in the gait data to obtain multiple key point features. The key points of the human body are the multiple of the human skeleton. Coordinate points;

The classification and regression module is used for feature classification and positioning regression of the multiple key point features based on the deep learning network to obtain human body information. The human body information includes the position of the human body part and the affinity vector of the human body part. The sum vector is used to connect two different key point features;

The calculation module is configured to calculate matching parameters between the human body information and preset standard human body information based on the multiple key point features, where the preset standard human body information includes a preset standard affinity vector, and the matching parameters are used To indicate the degree of matching between the human body information of the target object and the preset standard human body information;

The determining module is configured to determine the posture condition of the target object according to the matching parameter and a standard threshold value, where the standard threshold value is a critical value for an abnormal posture condition.