CN114601454A

CN114601454A - Method for monitoring bedridden posture of patient

Info

Publication number: CN114601454A
Application number: CN202210243496.XA
Authority: CN
Inventors: 孙卫国; 张若愚; 何晓锋
Original assignee: Shanghai Taiyi Health Technology Co ltd
Current assignee: Shanghai Taiyi Health Technology Co ltd
Priority date: 2022-03-11
Filing date: 2022-03-11
Publication date: 2022-06-10

Abstract

The invention discloses a method for monitoring the bedridden posture of a patient, which comprises the following steps: s1) acquiring image information of human body posture behaviors through a depth camera; s2) extracting human skeleton information from the collected image information to obtain human action skeleton map information; s3) the patient lying in bed is identified according to the human body action skeleton diagram. The step S3 classifies the posture of the relevant image by using a classification feature algorithm, and compares the classified posture type result with the national standard bed-lying posture. The monitoring method for the bedridden posture of the patient can accurately identify whether the posture of the patient is correct or not, so that the behavior of the patient after the operation or when the patient with serious diseases is in bed rest can be automatically corrected in a standard way.

Description

Method for monitoring bedridden posture of patient

Technical Field

The invention relates to a method for recognizing human body postures, in particular to a method for monitoring the lying posture of a patient.

Background

The patients who lie in bed for a long time and cannot take care of themselves in life due to various factors show an increasing trend. With respect to data statistics, there are many patients who only need the rehabilitation and nursing apparatus domestically. For paralyzed patients, who spend most of their time in bed, they need to turn over frequently to prevent bed clothes from dragging. At present, the nursing beds capable of turning over are too simple in function, require assistance of medical staff, are difficult to meet the requirement of turning over, or are high in price and difficult to popularize. In addition, the patient with mobility disability lies in bed for a long time, and certain psychological disorders can be caused inevitably, which are specifically represented by sleep disorder, mental hypoactivity, excessive thinking, depression, pessimism, self-responsibility, inappetence, physical decline, delusions and the like, and have great influence on the development and recovery of the disease.

The human body posture recognition and analysis field mainly analyzes and understands the posture behaviors of individuals from videos or images so as to judge the correctness and the effectiveness of the postures of the individuals. The human behavior recognition technology is widely applied to various fields, such as security monitoring systems, human-computer interaction systems and the like, and is not commonly applied in the medical field, but this does not represent the defect that the posture behavior technology has applicability in the aspect of medical care. Instead, with the rapid development of information digitization, the human behavior recognition technology has a wide development prospect in the medical field.

Currently, the improvement of the nursing method in the medical field mainly reflects in the improvement scheme and the voice control of medical equipment, and few methods for recognizing, monitoring and analyzing the behavior and posture of a patient during the nursing period are produced. The improvements are mainly embodied in that the use experience of a patient is increased, secondary damage caused in the use process is avoided, data such as patient behaviors and the like cannot be effectively utilized after the use, a certain data waste phenomenon is caused, the improvement at the current stage cannot be accurately evaluated, and the existing system and functions are difficult to improve so as to provide an effective theoretical basis. The patient's bed position data and behavior data are susceptible to influences of unrelated factors, such as clothing, lighting, gauze dressing, and the like. Without noise reduction preprocessing of a large amount of image data, it is difficult to accurately identify the correctness of the posture of the patient and to specify the standard behaviors of the patient such as sitting, lying, leaning and the like.

Patients and family members hospitalized in a ward can always hear doctors and nurses to emphasize that the patients with disturbance of consciousness or paralysis need to turn over and put in proper positions regularly. However, since the family members are not aware of the importance of the disease, a considerable number of patients still have complications such as pressure sores or pneumonia caused by falling down. These sequelae not only cause the patients themselves to lose their ability to take care of themselves, but also increase the burden on the family and society.

Disclosure of Invention

The technical problem to be solved by the invention is to provide a method for monitoring the lying posture of a patient, which can accurately identify whether the posture of the patient is correct or not, so that the standard correction can be automatically carried out on the behavior of the patient after the operation or when the patient with serious diseases lies in bed for rest.

The technical scheme adopted by the invention for solving the technical problems is to provide a method for monitoring the lying posture of a patient, which comprises the following steps: s1) acquiring image information of human posture behaviors through a depth camera; s2) extracting human skeleton information from the collected image information to obtain human action skeleton map information; s3) the patient lying in bed is identified according to the human body action skeleton diagram.

Compared with the prior art, the invention has the following beneficial effects: the monitoring method for the bedridden posture of the patient can accurately identify whether the posture of the patient is correct or not, so that the behavior of the patient after the operation or when the patient with serious diseases is in bed rest can be automatically corrected in a standard way.

Drawings

FIG. 1 is a schematic view of the monitoring process of the present invention for the patient's bed-ridden posture;

FIG. 2 is a diagram illustrating the format of the vectorized standard labels file processed according to the present invention.

Detailed Description

The invention is further described below with reference to the figures and examples.

The invention provides a method for monitoring the lying posture of a patient, which comprises the steps of extracting image information of human posture behaviors and key point depth information of a human skeleton map, extracting the extracted image into an original database through a posture recognition algorithm to perform behavior comparison fitting, fitting the extracted data into a human skeleton map cloud end, performing feature extraction on the image data of the skeleton map, further constructing a 3D (three-dimensional) coordinate unit structure map, performing related image posture classification by utilizing a classification feature algorithm, and performing matching rate calculation, skeleton joint deviation three-dimensional angle calculation and joint space vector accurate numerical calculation on a classified posture type result and a posture relative position coordinate of an original national standard unit.

Referring to fig. 1, the method for monitoring the lying posture of a patient provided by the invention comprises the following specific processing steps:

1. data monitoring

Data acquisition chooses for use the degree of depth camera to carry out multi-angle acquisition video and image data, because the patient head of sick bed is at the head of a bed, because the ward barrier shelters from and indoor light influences, consequently, settles the degree of depth camera and surveillance camera head in ward ceiling top, the place ahead, the left side, the right, wait four directions, acquire the depth information of four directions respectively. The camera can capture multi-azimuth image information of a patient, and no barrier for blocking the sight line exists in the range.

2. Data processing

In the process of identifying and analyzing the lying posture of the patient, the human skeleton information graph can be focused, the alpha phase optimization model can obviously highlight the internal human information such as the skeleton and the like of the patient in the lying process and is not influenced by the indoor environment, such as the dressing of the patient, the decoration of a ward, medical equipment, light and the like, the interference of identification is reduced, and the identification precision is improved.

The video information of human body behaviors and the depth information of key points of the human body skeleton map can be collected through the depth camera. And extracting human skeleton information from the collected video data. Human skeleton information can be extracted through an alpha phase, the alpha phase detects human skeleton joint points in a bottom-up mode, firstly, confidence degree of human body part detection is predicted through a network, positions of human key skeleton points are detected, after the positions of the key skeleton points are obtained, affinity between the parts which are associated with the parts is predicted through the network, finally, the confidence degree and the affinity are analyzed through a greedy algorithm, the key points are connected, and therefore human action skeleton diagram information can be obtained.

And extracting the acquired image into a bed skeleton map of the patient by using an Alphapose posture recognition algorithm. The skeleton diagram is composed of 25 joint points and 24 skeleton edges, and is a natural structure presentation of human joints and bones. The original skeleton data in each frame is always provided as a sequence of vectors, each vector representing the two or three dimensional coordinates of a respective human joint.

Taking the center position of the sickbed as an origin O coordinate, calculating the three-position vector offset and the offset angle of the human skeleton for measuring the standard posture accuracy, and assuming that the standard posture offset of the patient is (x)₁，x₂，x₃) The actual patient skeleton offset is (y)₁，y₂，y₃) Then the actual error offset vector is (x)₁-y₁，x₂-y₂，x₃-y₃)

3. Gesture recognition

The human body target detection of Alphapose adopts yolov5 algorithm and aims at a network training module. The system comprises a data acquisition sub-module, a data standardization sub-module and a model training sub-module. The main flow is to obtain the image file of the standard example through a data interface, and the data can be updated and shared in real time. The network training module for the network includes the preparation and preprocessing of data sets, the training of processed data labels, and the like.

1) Wherein the data acquisition submodule

The method is mainly used for training image and video data of a gesture recognition model, and particularly, the main source is obtained in three aspects, one is that the method comprises a national standard example image file stored in a standard library, and the example gesture image data can be automatically classified. The second is mainly real-time storage data for testing and comparison when the hospital detects the patient behaviors in real time.

2) Data standardization submodule

The data standardization sub-module performs vectorization processing on the national standard example image files stored in the standard library, trains the network by using a standard txt label file, converts an original image data set file used for training into a txt file data form, and the processed vectorization standard labels file is shown in fig. 2: in the labels file there is one line per object, each line being in class, x _ center, y _ center, width, height format. The box coordinates must be in a normalized xyz format (from 0 to 1). Since only the type of the object such as the lying posture of the patient is considered in the present invention, the labels file corresponding to each image has only one row, and the type is set to 0. Coordinate points at the upper left and the lower right of the patient image are extracted from the picture name, and a labels file in txt format is output after a series of operations.

The calculation formula is as follows:

wherein the coordinates of the center point of the normalized rectangular frame of the detected patient image are represented as (x _ center, y _ center), and the width and height of the rectangular frame of the detected image are set as width and height. w, h are respectively expressed as the width and length of the rectangular box of the normalized detected patient image.

3) Model training submodule

Pictures are randomly imported from a national standard patient posture standard data set to serve as the data set of the invention, and 70% of the data set serves as a training set and 30% of the data set serves as a verification set. For larger training sets, where the training data is similar to the pre-training data, the pre-trained weight parameters may be used to initialize the network and then begin training. In order to obtain a better model, the invention trains 300 epochs and stores the optimal model. And selecting an Adam optimizer, wherein the learning rate is reduced along with the increase of the training times, network parameters are continuously adjusted, and the main change of the hyper-parameters mainly comprises weight attenuation, learning rate, momentum and the like. Wherein the learning rate is an indispensable hyper-parameter. The learning rate is adjusted according to the training performance, and it should be noted that the convergence rate of the function is reduced when the value is too low, and the global optimum value of the objective function is difficult to obtain when the value is too high. In order to prevent the occurrence of the model overfitting phenomenon, the weight attenuation is included, and meanwhile, the specification item is included in the cost function. The learning rate decay can prevent the objective function from converging at a local optimum while maintaining the convergence rate.

4) Gesture recognition submodule

And (3) performing feature extraction on the extracted skeleton diagram by using a diagram convolution neural network, and given a 2D or 3D form coordinate sequence of the body joint, constructing a naturally-connected space-time diagram which takes the joint as a node and the human body structure and time as edges. Thus, the input to the model is the joint coordinate vectors on the graph nodes. This can be seen as an image-based simulation CNN, where the input is formed by pixel vectors located on a 2D image grid. And (4) performing multilayer space-time graph convolution operation on input data to generate a feature graph with a higher level on the graph. It will then be classified by the standard SoftMax classifier into the corresponding action class.

The features of the human skeleton include spatial relative features and motion features, the spatial features being extracted from the joints and the skeleton. The joints are represented as 3D coordinates and the bones as the difference between the two joint coordinates. The motion characteristics are generally represented as motion information, which includes displacement, joint direction, motion velocity, acceleration, and the like.

For features including spatial relative features, including the calculation of the relative vector size and error size of the calculated joint angle, assume the spatial vector size L of a specification file (referring to a training file, i.e., a file in the national standards library)_M＝(x_m，y_m，z_m) Wherein M is the definition name of each joint, the label is M belonging to {1, 2, 3.. M }, and the space vector of the joint detected in real time is L_N＝(x_n，y_n，z_n) Wherein N is the definition name of each joint detected in real time, the label is N epsilon {1, 2, 3.. N }, the space vector of the joint of the patient detected in real time is the error p, and the calculation formula is as follows

Wherein the larger the value of p is, the lower the image model matching rate is, the larger the degree of representing the posture of the patient from the standard posture is, and the posture feedback needs to be corrected in time; otherwise, the matching degree is higher, and the correction is not needed. If the value setting threshold value of p is larger than 0.5, the situation that the posture difference of the patient is too large is indicated, and alarm processing is needed. Otherwise, the data can be automatically stored in the data storage submodule to be used as training data for model training to be called, and therefore a closed-loop detection storage system is formed.

The motion features include computing motion information of the joints as coordinate differences along a time dimension. Similarly, the deformation of a bone is represented as the difference between vectors of the same bone in successive frames. Formally, the motion of the joint v in time t is calculated as

Deformation of the patient's bone defines the joint as similarly defined:

the acceleration of the patient's joint l is defined in a calculation:

wherein v is_t1Refers to the instantaneous speed of the patient captured by the camera at time t1, where v_t2Refers to the instantaneous speed of the patient captured by the camera at time t 2; wherein the average movement trend direction of the joint of the patient is

The patient motion trend direction is the direction of the AB vector, and the coordinate of the starting point of the patient starting motion is A ═ x₁,y₁,z₁) The final joint coordinate position is B ═ x₂,y₂,z₂)。

And finally, the obtained skeleton map information and skeleton information of the human body can be fed to a posture recognition module for recognition and analysis in the first several stages, and recognition, analysis and alarm are mainly carried out on standard recovery resting posture behaviors of postoperative patients or critically ill patients when the patients are in bed rest, such as (a patient lying on the back position and a patient lying on the side position).

The invention realizes the closed-loop management of the system and the secondary utilization of data, compared with the traditional openposition model, the adopted Alphaposition optimization model firstly detects the human body and then obtains key points and a skeleton, so the accuracy and the AP value (the AP is measured to be the quality of the learned model on each category) of the Alphaposition optimization model are obviously higher than the openposition model.

The target detection algorithm of the invention: YOLOv5 is implemented in PyTorch, benefits from a mature PyTorch ecosystem, is simpler to support, is easier to deploy, and has the following advantages over YOLOv4, YOLOv 5:

(1) the speed is faster. On YOLOv5 Colab notebook, running TeslaP100, it can be seen that the inference time per image only needs 0.007 seconds, which means 140 Frames Per Second (FPS), at a speed more than 2 times that of YOLOv 4.

(2) The precision is higher. In the Roboflow test on the blood cell count and test (BCCD) dataset, 100 epochs alone were trained to achieve an average accuracy (mAP) of approximately 0.895. Admittedly EfficientDet and YOLOv4 perform comparably, but it is very rare to have such a comprehensive performance boost without any loss in accuracy.

(3) The volume is smaller. The weight file for YOLOv5 is 27 megabytes. The weight file for YOLOv4 (in the Darknet architecture) was 244 megabits. YOLOv5 is approximately 90% smaller than YOLOv 4! This means that YOLOv5 can be more easily deployed onto embedded devices.

Although the present invention has been described with respect to one or more embodiments thereof, it will be understood by those skilled in the art that the foregoing and various other changes, omissions and deviations in the form and detail thereof may be made without departing from the scope of this invention.

Claims

1. A method for monitoring the bed position of a patient is characterized by comprising the following steps:

s1) acquiring image information of human posture behaviors through a depth camera;

s2) extracting human skeleton information from the collected image information to obtain human action skeleton map information;

s3) the patient lying in bed is identified according to the human body action skeleton diagram.

2. The method for monitoring the posture of a patient in bed as claimed in claim 1, wherein the depth camera of step S1 is connected to a plurality of cameras, which are disposed above, in front of, on the left of and on the right of the ceiling of the ward to acquire the depth information in four directions, respectively.

3. The method for monitoring the lying posture of the patient as claimed in claim 1, wherein the step S2 of detecting the human skeletal joint points by the alphaphase model in a bottom-up manner comprises:

firstly, predicting the confidence coefficient of human body part detection through a network, and detecting the position of a key skeleton point of a human body; after the positions of the key skeletal points are obtained, the affinity between the related parts is predicted through a network; and finally, analyzing the confidence and the affinity by a greedy algorithm, and connecting the key points to obtain a human body action skeleton diagram of the patient.

4. The method for monitoring the posture of a patient in bed as claimed in claim 3, wherein the human body motion skeleton map is composed of 25 joint points and 24 skeleton edges, the original skeleton data in each frame is always provided as a vector sequence, each vector represents the two-dimensional or three-dimensional coordinates of the corresponding human body joint; the step S2 further includes calculating the three-position vector offset and the offset angle of the human body skeleton with the center position of the patient bed as the origin O coordinate.

5. The method for monitoring the lying posture of the patient as claimed in claim 1, wherein the step S3 of classifying the relevant image posture by using a classification feature algorithm and comparing the classified posture type result with the national standard lying posture specifically comprises:

s31) acquiring the national standard example image files stored in the standard library and carrying out vectorization processing to generate a patient posture standard data set;

s32) randomly importing pictures from the patient posture standard data set as a target data set of the neural network model, taking 70% of the target data set as a training set and taking 30% of the target data set as a verification set;

s33) carrying out feature extraction on the obtained human body action skeleton diagram by using a diagram convolution neural network, giving a 2D or 3D form coordinate sequence of a body joint, and constructing a naturally communicated space-time diagram which takes the joint as a node and the human body structure and time as sides; and generating a higher-level feature map on input data by adopting multilayer space-time diagram convolution operation, and then classifying the feature map into a corresponding action category through a standard SoftMax classifier.

6. The method for monitoring the ambulatory position of a patient as set forth in claim 5, wherein said vectorized standard labels file processed in step S31 has a row of objects, each row including a category, x _ center, y _ center, width and height format, and wherein (x _ center, y _ center) is the coordinates of the center point of the normalized rectangular frame of the image of the patient.

7. The method for monitoring the posture of a patient in bed as set forth in claim 5, wherein the features of the human skeleton include spatially relative features extracted from joints and bones, and the step S33 includes:

calculating the relative vector size and error size of joint angle, and assuming the space vector size L of the specification file_M＝(x_m，y_m，z_m) Wherein M is the definition name of each joint, and the label is M belonging to {1, 2, 3.. M };

real-time detection of joint space vector L_N＝(x_n，y_n，z_n) Wherein N is the definition name of each joint detected in real time, and the label is N belonging to {1, 2, 3.. N };

the space vector error p of the joint of the patient is detected in real time,

wherein a larger value of p indicates a lower image model matching rate and a larger degree of representing the posture of the patient from the standard posture; if the value of p is larger than the set threshold value, judging that the posture difference of the patient is too large, feeding back the posture to be corrected in real time and carrying out alarm processing; otherwise, the data is automatically stored in the data storage submodule and is used as training data of the neural network model to be called, and therefore a closed loop detection storage system is formed.

8. The method for monitoring the posture of a patient lying in bed as claimed in claim 5, wherein the characteristics of the human skeleton include motion characteristics, and the step S33 includes:

calculating the motion information of the joint as a coordinate difference along a time dimension, and simultaneously setting the deformation of the skeleton as a vector difference of the same skeleton in the continuous frames; the average movement trend direction of the patient's joints is

The patient motion trend direction is the direction of AB vector, and the coordinate of the patient starting motion starting point is A ═ Cx₁,y₁,z₁) The final joint coordinate position is B ═ x₂,y₂,z₂)。