CN112022096A - Sleep state monitoring method and device - Google Patents
Sleep state monitoring method and device Download PDFInfo
- Publication number
- CN112022096A CN112022096A CN202010924581.3A CN202010924581A CN112022096A CN 112022096 A CN112022096 A CN 112022096A CN 202010924581 A CN202010924581 A CN 202010924581A CN 112022096 A CN112022096 A CN 112022096A
- Authority
- CN
- China
- Prior art keywords
- average
- sleep state
- target
- key points
- human body
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/48—Other medical applications
- A61B5/4806—Sleep evaluation
- A61B5/4812—Detecting sleep stages or cycles
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/02—Detecting, measuring or recording pulse, heart rate, blood pressure or blood flow; Combined pulse/heart-rate/blood pressure determination; Evaluating a cardiovascular condition not otherwise provided for, e.g. using combinations of techniques provided for in this group with electrocardiography or electroauscultation; Heart catheters for measuring blood pressure
- A61B5/0205—Simultaneously evaluating both cardiovascular conditions and different types of body conditions, e.g. heart and respiratory condition
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
- A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
- A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
- A61B5/1121—Determining geometric values, e.g. centre of rotation or angular range of movement
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
- A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
- A61B5/1126—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb using a particular sensing technique
- A61B5/1128—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb using a particular sensing technique using image analysis
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/48—Other medical applications
- A61B5/4806—Sleep evaluation
- A61B5/4809—Sleep detection, i.e. determining whether a subject is asleep or not
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/48—Other medical applications
- A61B5/4806—Sleep evaluation
- A61B5/4815—Sleep quality
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Veterinary Medicine (AREA)
- Public Health (AREA)
- Animal Behavior & Ethology (AREA)
- Surgery (AREA)
- Molecular Biology (AREA)
- Medical Informatics (AREA)
- Heart & Thoracic Surgery (AREA)
- Biomedical Technology (AREA)
- Pathology (AREA)
- Biophysics (AREA)
- Physiology (AREA)
- Theoretical Computer Science (AREA)
- Dentistry (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- Cardiology (AREA)
- Bioinformatics & Computational Biology (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Multimedia (AREA)
- Human Computer Interaction (AREA)
- Geometry (AREA)
- Evolutionary Computation (AREA)
- Pulmonology (AREA)
- Anesthesiology (AREA)
- Psychiatry (AREA)
- Signal Processing (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
Abstract
The invention relates to a sleep state monitoring method and a device, which are characterized in that feature extraction is carried out on an acquired sleep state image of a target through a constructed human posture recognition network model to obtain human key points, the two-dimensional length of the key points of left and right shoulders, the deviation angle of the connecting line of the central node of the left and right shoulders and a root node, and the included angle of a shoulder elbow connecting line and an elbow connecting line are calculated according to the obtained human key points to carry out primary judgment on the posture of the target human body, and then the final judgment on the sleep state is realized by combining the average angular speed and the average heartbeat of the target. The method can accurately obtain the state of the target in the sleep within a certain period of time, is convenient for executing a corresponding awakening strategy on the sleep of the target subsequently, and improves the sleep quality of the target.
Description
Technical Field
The invention belongs to the technical field of medical detection, and particularly relates to a sleep state monitoring method and device.
Background
In the medical field, sleep monitoring and awakening of a patient are of great importance, and a traditional sleep monitoring system comprises a respiration monitoring device, a blood oxygen saturation detection device, brain wave detection and the like, and is judged according to the principles that the human body emits different brain wave frequencies in different sleep stages, heartbeats, respiratory frequencies and the like. However, it should be noted that the conventional device is not only expensive, but also the sleeping experience of the patient is reduced due to the foreign body sensation caused by the patch electrode or the blood oxygen clip.
At present, the method for detecting the thoracic cavity fluctuation and the quilt coverage range of a person in sleep in real time through a camera is widely used for sleep monitoring of a patient, and when respiratory frequency is monitored, the requirement on the camera is high, the camera is required to always keep high frame rate shooting, and the service life of the camera is shortened; when a plurality of patients exist in the ward at the same time, the monitoring accuracy is questioned, and the monitoring method is still simple for capturing the action details in sleep; or the heartbeat and the respiratory frequency of the patient are detected through special wearable equipment (such as a smart bracelet), but the sleep state of the patient cannot be accurately tested because the detection precision is possibly not high and the problem of inaccurate detection still exists.
Disclosure of Invention
The invention aims to provide a sleep state monitoring method and a sleep state monitoring device, which are used for solving the problem that the sleep state monitoring method in the prior art cannot monitor an accurate sleep state.
In order to achieve the above object, the technical solution of the sleep state monitoring method provided by the present invention includes the following steps:
1) acquiring a sleep state image set of a target in a target area, wherein the sleep state image set comprises a training data set and a test data set;
2) constructing a human body posture recognition network model, and training the constructed human body posture recognition network model by using the training data set to obtain a trained human body posture recognition network model;
3) inputting a data set to be tested into a trained human body posture recognition network model to obtain human body key points of the data set to be tested, wherein the human body key points comprise head center key points, left and right shoulder key points, left and right elbow key points, left and right hand key points and left and right hip key points;
4) respectively calculating the two-dimensional length of key points of the left shoulder and the right shoulder, the connecting line offset angle of a central node of the left shoulder and the right shoulder and a root node, and the included angle between a shoulder elbow connecting line and an elbow connecting line according to the acquired key points of the human body; judging the two-dimensional length of key points of the left shoulder and the right shoulder and the size of a first set threshold, the connecting line offset angle of a central node of the left shoulder and the right shoulder and a root node and the size of a second set threshold, and the included angle of a shoulder elbow connecting line and an elbow connecting line and the size of a third set threshold, and if the three are all larger than the corresponding set thresholds, judging the posture of the human body to be a large-amplitude motion; if at least one of the three is not greater than the corresponding set threshold, judging the human body posture to be a small-amplitude action; wherein the root node is the center of a key point of the left and right hip bones;
5) acquiring the angular velocity and the heartbeat of the target motion through wearable equipment, and respectively calculating the average angular velocity and the average heartbeat;
6) according to the average angular velocity and the average heartbeat in the step 5), the sizes of the average angular velocity and the fourth set threshold value and the sizes of the average heartbeat and the average threshold value are respectively judged, and the sleep state of the patient is obtained by combining the judgment result in the step 4):
when the judgment result is that the large-amplitude movement is performed, the average angular speed is greater than a fourth set threshold value, and the average heartbeat is greater than an average threshold value, the target is in a shallow sleep or rapid eye movement period;
when the judgment result is that the large-amplitude movement is performed, the average angular speed is smaller than a fourth set threshold value, and the average heartbeat is larger than the average threshold value, the target is in a shallow sleep or awakening state;
when the judgment result is that the small-amplitude motion is performed, the average angular velocity is greater than a fourth set threshold value, and the average heartbeat is greater than an average threshold value, the target is in a light sleep state;
and when the judgment result is that the small-amplitude motion is performed, the average angular speed is greater than the fourth set threshold value, and the average heartbeat is less than the average threshold value, the target is in the deep sleep range.
The invention has the beneficial effects that:
according to the invention, the sleep state images of the target collected in continuous time periods are subjected to feature extraction through the constructed human posture recognition network model to obtain human key points, the two-dimensional length of the left and right shoulder key points, the deviation angle of the connecting line of the left and right shoulder center nodes and the root node, and the included angle of the shoulder connecting line and the elbow connecting line are calculated according to the obtained human key points, the primary judgment of the posture of the target human body is carried out, then the final judgment of the sleep state is realized by combining the average angular speed and the average heartbeat of the target, the state of the target in sleep in a certain time period can be accurately obtained, the corresponding wake-up strategy is conveniently carried out on the sleep of the target in the follow-up process, and the sleep quality of.
Further, the method also comprises a step of executing a corresponding wake-up strategy according to the sleep state of the target in the step 6).
Further, in step 1), the method further includes performing normalization processing on the training data set and the test data set.
Further, the human body posture recognition network model constructed in the step 2) is one of openpos, Densepose and HRNet.
Further, when training the human body posture recognition network model in the step 2), a mean square error loss function is adopted for verification.
The invention also provides a sleep state monitoring device, which comprises a processor and a memory, wherein the processor is used for executing the technical scheme of the sleep state monitoring method stored in the memory.
Drawings
FIG. 1 is a flow chart of an embodiment of a sleep state monitoring method of the present invention;
fig. 2 is a block diagram of a sleep state monitoring method according to an embodiment of the present invention.
Detailed Description
The embodiments of the present invention will be explained below with reference to the drawings and specific examples.
Sleep state monitoring method embodiment
The sleep state monitoring method provided by the invention mainly adopts an AIoT (artificial intelligence Internet of things) technology to carry out two-dimensional human body posture estimation and motion state analysis on the acquired sleep image of the target in the target area, so that the action of the person in the target area in the sleep in a certain period of time can be obtained, and the sleep stage of the person can be judged.
Taking a ward area of a certain hospital as an example, the sleep state monitoring method of the invention is specifically introduced; as shown in fig. 1, the monitoring method includes the following steps:
1) acquiring a sleep state image set of a target area patient in continuous time periods, wherein the sleep state image set comprises a training data set and a data set to be tested;
in the embodiment, an IR camera is adopted to shoot the sleep state of a patient in a target area, the shooting content can be pictures or videos, and the purpose is to obtain sleep state images in continuous time periods of the target area and divide the obtained sleep state images into a training data set and a test data set; the training data set is manually calibrated Heatmap label data.
In this embodiment, a normalization method is used to preprocess the acquired sleep state image set.
2) And constructing a human body posture recognition network model, and performing end-to-end training on the constructed human body posture recognition network model by using a training data set to obtain the trained human body posture recognition network model.
The human body posture recognition network model in the embodiment comprises a human body key point Encoder (Encoder1) and a human body key point Decoder (Decode 1), wherein the human body key point Encoder is used for extracting the characteristics of an input image; the human body key point decoder is used for carrying out up-sampling and outputting the human body key point Heatmap with the same size as the input image. Wherein the output human body key point Heatmap conforms to the heat spot with Gaussian distribution, and the value range of the output human body key point is [ 0-1 ] because the input image is normalized in advance.
It should be noted that the human body key point Decoder (Decoder1) in the present invention sets corresponding feature output channels according to the number of selected human body key points, for example, when the human body key points Heatmap selects 9 types, which are respectively head center key points (1 type), left and right shoulder key points (2 types), left and right elbow key points (2 types), left and right hand key points (2 types), and left and right hip key points (2 types), 10 channels are set for feature output, and one of the channels is a background type.
When the human body posture recognition network model is trained, the selected training data set is marked by generating hot spots centering on key points of left and right hip bones by using Gaussian kernel convolution, and the marked training data set is input into the human body posture recognition network model and trained by combining a mean square error loss function. The mean square error loss function is prior art and will not be described herein too much.
The human body posture recognition network model constructed in the embodiment can be one of openposition (human body posture recognition), densipose (human body posture estimation) and HRNet, and the human body posture recognition is the prior art, so that the invention is not restricted.
3) Inputting a data set to be tested into a trained human body posture recognition network model, and obtaining human body key points of the data set to be tested, wherein the human body key points comprise head center key points, left and right shoulder key points, left and right elbow key points, left and right hand key points and left and right hip key points.
4) Calculating two-dimensional lengths of key points of left and right shoulders, an offset angle of a connecting line of a central node of the left and right shoulders and a root node, and an included angle of a shoulder elbow connecting line and an elbow connecting line according to the acquired key points of the human body in the step 3); and comparing the two-dimensional length of the key points of the left shoulder and the right shoulder with the size of a first set threshold, the connection line offset angle of the central node of the left shoulder and the right shoulder with the root node with the size of a second set threshold, and the included angle of the shoulder elbow connecting line and the elbow connecting line with the size of a third set threshold, if the three values are all larger than the corresponding set thresholds, judging the human body posture to be large-amplitude motion, and if at least one of the three values is not larger than the corresponding set threshold, judging the human body posture to be small-amplitude motion.
In the embodiment, according to the acquired key points of the human body, the two-dimensional length of key points of the left and right shoulders, the connecting line offset angle between the central node of the left and right shoulders and the root node, and the included angle between the shoulder elbow connecting line and the elbow connecting line are calculated, and the three parameters are integrated to perform preliminary judgment on large-amplitude actions and small-amplitude actions of the sleep state of the patient; obtaining an inclined posture according to the transformation of the connection line of the centers of the key points of the left shoulder and the right shoulder in the continuous multi-frame images and a root node (the root node is the center of the key point of the left crotch and the right crotch); the amplitude and direction of the arm movement of the patient can be judged according to the included angle between the shoulder-elbow connecting line and the elbow connecting line.
In the embodiment, three parameters of two-dimensional length of key points of left and right shoulders, deviation angle of connecting lines of a central node of the left and right shoulders and a root node, and included angle of a shoulder elbow connecting line and an elbow connecting line are calculated according to key points of a human body; certainly, as other implementation manners, multiple parameters can be calculated by using the key points of the human body according to actual conditions, so that enough action posture types in sleep can be divided, and further, the sleep state of the patient can be analyzed more finely.
In this embodiment, the value range of the first set threshold is set to 20cm to 30cm, the value range of the second set threshold is set to 10 ° to 30 °, and the value range of the third set threshold is set to 10 ° to 30 °, which can be determined according to actual conditions.
5) Acquiring the angular velocity and the heartbeat of the self movement of the patient by using wearable equipment, and respectively calculating the average angular velocity and the average heartbeat;
specifically, through set up wearable equipment on one's body at the patient, like bracelet gyroscope, bracelet heartbeat sensor, gather the angular velocity and the heartbeat of patient's self motion, calculate average angular velocity and patient's average heartbeat respectively.
Wherein the hand ring gyroscope is used for detecting the angular velocity of the movement of the patient according to the three-axis angular velocity phi detected by the hand ring gyroscope(x,y,z)And obtaining the average angular velocity by taking the root mean square of the angular velocities of the three.
The bracelet heartbeat sensor is used for detecting the heartbeat of a patient, calculating the average heartbeat (bpm) of the patient in the sleep time period according to the detected heartbeat in the set time. For example, normal men wake up with an average heart beat of 60-80bpm, slightly lower in light sleep and 50-70bpm or less in deep sleep.
6) Judging the size of the average angular velocity and the fourth set threshold value and the size of the average heartbeat and the average threshold value according to the average angular velocity and the average heartbeat in the step 5), and combining the judgment result in the step 4) to obtain the sleep state of the patient:
when the judgment result is that the patient is in a shallow sleep or rapid eye movement period (REM), the average angular velocity is greater than a fourth set threshold and the average heartbeat is greater than an average threshold;
when the judgment result is that the large-amplitude movement is performed, the average angular speed is smaller than a fourth set threshold value, and the average heartbeat is larger than the average threshold value, the patient is in a shallow sleep or wakefulness state;
when the judgment result is that the small-amplitude motion, the average angular velocity and the average heartbeat are respectively greater than the fourth set threshold and the average threshold, the transient muscle jump symptom or the small-amplitude motion may be generated, and the patient is in a light sleep state;
and when the judgment result is that the small-amplitude motion is performed, the average angular velocity is greater than the fourth set threshold value, and the average heartbeat is less than the average threshold value, the patient is in the deep sleep range.
In the above embodiment, when the patient is determined to be in the shallow sleep or the Rapid Eye Movement (REM), the number of beats of the heart of the patient is detected to perform further determination, and if the number of beats of the heart does not change significantly, the patient is in the Rapid Eye Movement (REM) state.
The embodiment further comprises the following steps: when the judgment result is that the large-amplitude movement is performed, the average angular speed is greater than the fourth set threshold, and the average heartbeat is less than the average threshold, the subsequent state judgment is needed, namely whether the heartbeat is accelerated or not is observed, if the heartbeat is accelerated, the deep sleep state is probably changed into the light sleep state, and if the heartbeat is not obviously changed, the deep sleep state is still maintained, and the operation belongs to the unconscious movement.
Based on the embodiment, the sleep state of the patient can be accurately judged according to the acquired human body key points and the combination of the information such as the gyroscope, the heartbeat sensor and the like in the bracelet worn by the patient.
7) Executing a corresponding awakening strategy according to the sleep state of the patient in the step 6).
Specifically, according to the sleep state of the patient, the wake-up strategy of the embodiment is divided into three stages, and when the wake-up period starts, and the patient is in deep sleep, the slow wake-up is performed when the sleep cycle of the patient is adjusted to shallow sleep; when the patient is in an awakening state, the patient can be directly awakened; if the patient does not switch from deep sleep to light sleep at the end of the awakening period, the patient is awakened slowly, and the discomfort of the patient is reduced. The wake-up period refers to the wake-up time.
The awakening operation in the embodiment is completed by peripheral auxiliary equipment, such as measures of intelligent pillow vibration, intelligent curtain opening, bedside lamp opening and the like, and the cooperation of multiple intelligent awakening equipment is a mature technology, so that the awakening operation is not restricted.
The invention can also carry out corresponding analysis according to the sleep quality of the patient all night, obtain the ratio of the total time length occupied by deep sleep, and adjust the awakening treatment decision according to the ratio; if the sleep quality is good, the sleep can be awakened when the light sleep starts; if the sleep quality is poor, the light sleep time can be prolonged.
In order to implement the sleep state monitoring method, the invention provides a sleep state monitoring system, and specifically, as shown in fig. 2, the sleep state monitoring system comprises an infrared thermal imaging camera and an artificial intelligence internet of things (AIoT), wherein the infrared thermal imaging camera sends acquired data to the artificial intelligence internet of things, and the artificial intelligence internet of things stores, analyzes and judges the received data.
The artificial intelligence Internet of things (AIoT) comprises a human body key point perception module and a processing decision module. The human body key point sensing module is used for detecting human body key points of an image shot by the infrared thermal imaging camera and extracting a characteristic image; and the processing decision module performs data analysis according to the extracted features, judges the sleep state of the user and executes a corresponding awakening strategy according to the corresponding sleep state.
The infrared thermal imaging camera (or the IR camera) in the embodiment is arranged at the upper corner capable of comprehensively covering the range of a patient, and the angle posture of each rotation is fixed, so that the accuracy of the comparison of the front frame and the rear frame is ensured. If the sleeping state of a plurality of people in a certain space is monitored, for example, all patients in a ward, the cradle head tour of the cameras can be set according to the distribution of the sickbed position areas, and the cycle of the circulating path can be set to be longer; meanwhile, in order to reduce power consumption, a monitoring wake-up module may be provided, configured to control the camera to wake up within a set monitoring time and to be in a sleep state within a non-monitoring time period (a sampling frame rate is 1 fps).
It should be noted that the camera or the IR camera used in this embodiment takes a thermal imaging image, that is, the thermal imaging is used for analysis, so that the privacy of the user is better protected. As another embodiment, the original RGB camera may be further improved for performing thermal imaging shooting.
Sleep state monitoring device embodiment
The sleep state monitoring device provided by the invention is actually a device such as a computer with data processing capability, and the device comprises a processor and a memory, wherein the processor is used for executing instructions to realize the sleep state monitoring method of the invention.
Claims (6)
1. A sleep state monitoring method is characterized by comprising the following steps:
1) acquiring a sleep state image set of a target in a target area, wherein the sleep state image set comprises a training data set and a test data set;
2) constructing a human body posture recognition network model, and training the constructed human body posture recognition network model by using the training data set to obtain a trained human body posture recognition network model;
3) inputting a data set to be tested into a trained human body posture recognition network model to obtain human body key points of the data set to be tested, wherein the human body key points comprise head center key points, left and right shoulder key points, left and right elbow key points, left and right hand key points and left and right hip key points;
4) respectively calculating the two-dimensional length of key points of the left shoulder and the right shoulder, the connecting line offset angle of a central node of the left shoulder and the right shoulder and a root node, and the included angle between a shoulder elbow connecting line and an elbow connecting line according to the acquired key points of the human body; judging the two-dimensional length of key points of the left shoulder and the right shoulder and the size of a first set threshold, the connecting line offset angle of a central node of the left shoulder and the right shoulder and a root node and the size of a second set threshold, and the included angle of a shoulder elbow connecting line and an elbow connecting line and the size of a third set threshold, and if the three are all larger than the corresponding set thresholds, judging the posture of the human body to be a large-amplitude motion; if at least one of the three is not greater than the corresponding set threshold, judging the human body posture to be a small-amplitude action; wherein the root node is the center of a key point of the left and right hip bones;
5) acquiring the angular velocity and the heartbeat of the target motion through wearable equipment, and respectively calculating the average angular velocity and the average heartbeat;
6) according to the average angular velocity and the average heartbeat in the step 5), the size of the average angular velocity and the fourth set threshold value and the size of the average heartbeat and the average threshold value are respectively judged, and the judgment result in the step 4) is combined to obtain the sleep state of the target:
when the judgment result is that the large-amplitude movement is performed, the average angular speed is greater than a fourth set threshold value, and the average heartbeat is greater than an average threshold value, the target is in a shallow sleep or rapid eye movement period;
when the judgment result is that the large-amplitude movement is performed, the average angular speed is smaller than a fourth set threshold value, and the average heartbeat is larger than the average threshold value, the target is in a shallow sleep or awakening state;
when the judgment result is that the small-amplitude motion is performed, the average angular velocity is greater than a fourth set threshold value, and the average heartbeat is greater than an average threshold value, the target is in a light sleep state;
and when the judgment result is that the small-amplitude motion is performed, the average angular velocity is greater than the fourth set threshold value, and the average heartbeat is less than the average threshold value, the target is in a deep sleep state.
2. The sleep state monitoring method according to claim 1, further comprising a step of executing a corresponding wake-up policy according to the sleep state of the target in step 6).
3. The sleep state monitoring method according to claim 1, further comprising normalizing the training data set and the test data set in step 1).
4. The sleep state monitoring method according to claim 1, wherein the human posture recognition network model constructed in the step 2) is one of openpos, Densepose and HRNet.
5. The sleep state monitoring method according to claim 4, wherein in the step 2), a mean square error loss function is adopted for verification when training the human posture recognition network model.
6. A sleep state monitoring apparatus comprising a processor and a memory, characterized in that the processor executes a computer program stored in the memory implementing the sleep state monitoring method according to any one of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010924581.3A CN112022096A (en) | 2020-09-05 | 2020-09-05 | Sleep state monitoring method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010924581.3A CN112022096A (en) | 2020-09-05 | 2020-09-05 | Sleep state monitoring method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112022096A true CN112022096A (en) | 2020-12-04 |
Family
ID=73591600
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010924581.3A Withdrawn CN112022096A (en) | 2020-09-05 | 2020-09-05 | Sleep state monitoring method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112022096A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112580522A (en) * | 2020-12-22 | 2021-03-30 | 北京每日优鲜电子商务有限公司 | Method, device and equipment for detecting sleeper and storage medium |
| CN112617819A (en) * | 2020-12-21 | 2021-04-09 | 西南交通大学 | Method and system for recognizing lower limb posture of infant |
| CN112926541A (en) * | 2021-04-09 | 2021-06-08 | 济南博观智能科技有限公司 | Sleeping post detection method and device and related equipment |
| CN113465155A (en) * | 2021-06-23 | 2021-10-01 | 深圳市海清视讯科技有限公司 | Method, system, host and storage medium for reducing quilt kicking behavior of infants |
-
2020
- 2020-09-05 CN CN202010924581.3A patent/CN112022096A/en not_active Withdrawn
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112617819A (en) * | 2020-12-21 | 2021-04-09 | 西南交通大学 | Method and system for recognizing lower limb posture of infant |
| CN112580522A (en) * | 2020-12-22 | 2021-03-30 | 北京每日优鲜电子商务有限公司 | Method, device and equipment for detecting sleeper and storage medium |
| CN112926541A (en) * | 2021-04-09 | 2021-06-08 | 济南博观智能科技有限公司 | Sleeping post detection method and device and related equipment |
| CN112926541B (en) * | 2021-04-09 | 2022-11-08 | 济南博观智能科技有限公司 | Sleeping post detection method and device and related equipment |
| CN113465155A (en) * | 2021-06-23 | 2021-10-01 | 深圳市海清视讯科技有限公司 | Method, system, host and storage medium for reducing quilt kicking behavior of infants |
| CN113465155B (en) * | 2021-06-23 | 2022-07-26 | 深圳市海清视讯科技有限公司 | Method, system, host and storage medium for reducing quilt kicking behavior of infants |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112022096A (en) | Sleep state monitoring method and device | |
| US20180122073A1 (en) | Method and device for determining vital parameters | |
| US10004410B2 (en) | System and methods for measuring physiological parameters | |
| US9436984B2 (en) | Compensating for motion induced artifacts in a physiological signal extracted from a single video | |
| Villarroel et al. | Non-contact vital sign monitoring in the clinic | |
| CN108135487A (en) | For obtaining the equipment, system and method for the vital sign information of object | |
| US20140376789A1 (en) | Compensating for motion induced artifacts in a physiological signal extracted from multiple videos | |
| CN106413533B (en) | Apparatus, system and method for detecting apnea of a subject | |
| Li et al. | A non-contact vision-based system for respiratory rate estimation | |
| WO2019140155A1 (en) | Systems, devices, and methods for tracking and/or analyzing subject images and/or videos | |
| McDuff et al. | Fusing partial camera signals for noncontact pulse rate variability measurement | |
| WO2019173237A1 (en) | Systems, devices, and methods for tracking and analyzing subject motion during a medical imaging scan and/or therapeutic procedure | |
| Chen et al. | Respiratory rate estimation from face videos | |
| CA2787170A1 (en) | Multivariate residual-based health index for human health monitoring | |
| Chen et al. | Non-contact heart rate monitoring in neonatal intensive care unit using RGB camera | |
| Przybyło | A deep learning approach for remote heart rate estimation | |
| Sugita et al. | Noise reduction technique for single-color video plethysmography using singular spectrum analysis | |
| Mo et al. | Collaborative three-tier architecture noncontact respiratory rate monitoring using target tracking and false peaks eliminating algorithms | |
| Chan et al. | Improving heart rate and heart rate variability estimation from video through a hr-rr-tuned filter | |
| US20210225489A1 (en) | Determining the likelihood of patient self-extubation | |
| Gupta et al. | Remote photoplethysmography‐based human vital sign prediction using cyclical algorithm | |
| CN112037916A (en) | Shared multifunctional sudden death prevention physiological information detection system and method thereof | |
| Liu et al. | Adaptive-weight network for imaging photoplethysmography signal extraction and heart rate estimation | |
| Mahajan et al. | Arduino based portable ECG and PPG signal acquisition system | |
| TW202145254A (en) | Information processing system and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20201204 |