CN116687394B - Tumble detection method, device, equipment and storage medium based on millimeter wave radar - Google Patents
Tumble detection method, device, equipment and storage medium based on millimeter wave radar Download PDFInfo
- Publication number
- CN116687394B CN116687394B CN202310975071.2A CN202310975071A CN116687394B CN 116687394 B CN116687394 B CN 116687394B CN 202310975071 A CN202310975071 A CN 202310975071A CN 116687394 B CN116687394 B CN 116687394B
- Authority
- CN
- China
- Prior art keywords
- point cloud
- falling
- action
- cloud image
- state
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 42
- 230000009471 action Effects 0.000 claims abstract description 117
- 201000010099 disease Diseases 0.000 claims abstract description 55
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 claims abstract description 55
- 238000000034 method Methods 0.000 claims abstract description 45
- 230000007704 transition Effects 0.000 claims abstract description 23
- 238000006243 chemical reaction Methods 0.000 claims abstract description 20
- 230000008859 change Effects 0.000 claims abstract description 16
- 238000012549 training Methods 0.000 claims description 18
- 230000002159 abnormal effect Effects 0.000 claims description 17
- 230000036772 blood pressure Effects 0.000 claims description 17
- 238000012544 monitoring process Methods 0.000 claims description 16
- 230000008569 process Effects 0.000 claims description 16
- 230000036760 body temperature Effects 0.000 claims description 15
- 238000000605 extraction Methods 0.000 claims description 13
- 230000003938 response to stress Effects 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 9
- 238000012545 processing Methods 0.000 claims description 9
- 230000002792 vascular Effects 0.000 claims description 9
- 230000009467 reduction Effects 0.000 claims description 7
- 238000011478 gradient descent method Methods 0.000 claims description 6
- 239000004973 liquid crystal related substance Substances 0.000 claims description 6
- 238000001914 filtration Methods 0.000 claims description 4
- 201000003152 motion sickness Diseases 0.000 claims description 4
- 230000002747 voluntary effect Effects 0.000 claims description 4
- 238000013473 artificial intelligence Methods 0.000 abstract description 7
- 238000005516 engineering process Methods 0.000 abstract description 6
- 230000006870 function Effects 0.000 description 10
- 238000004891 communication Methods 0.000 description 8
- 238000007726 management method Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 3
- 230000006399 behavior Effects 0.000 description 3
- 230000010247 heart contraction Effects 0.000 description 3
- 208000002173 dizziness Diseases 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 208000024335 physical disease Diseases 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- UCTWMZQNUQWSLP-VIFPVBQESA-N (R)-adrenaline Chemical compound CNC[C@H](O)C1=CC=C(O)C(O)=C1 UCTWMZQNUQWSLP-VIFPVBQESA-N 0.000 description 1
- 229930182837 (R)-adrenaline Natural products 0.000 description 1
- 206010061245 Internal injury Diseases 0.000 description 1
- 208000007502 anemia Diseases 0.000 description 1
- 206010003119 arrhythmia Diseases 0.000 description 1
- 230000006793 arrhythmia Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 229960005139 epinephrine Drugs 0.000 description 1
- 238000000802 evaporation-induced self-assembly Methods 0.000 description 1
- 230000001815 facial effect Effects 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 208000019622 heart disease Diseases 0.000 description 1
- 230000006698 induction Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012847 principal component analysis method Methods 0.000 description 1
- 230000000284 resting effect Effects 0.000 description 1
- 230000000630 rising effect Effects 0.000 description 1
- 238000001228 spectrum Methods 0.000 description 1
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
- A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
- A61B5/1116—Determining posture transitions
- A61B5/1117—Fall detection
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/02—Detecting, measuring or recording pulse, heart rate, blood pressure or blood flow; Combined pulse/heart-rate/blood pressure determination; Evaluating a cardiovascular condition not otherwise provided for, e.g. using combinations of techniques provided for in this group with electrocardiography or electroauscultation; Heart catheters for measuring blood pressure
- A61B5/0205—Simultaneously evaluating both cardiovascular conditions and different types of body conditions, e.g. heart and respiratory condition
- A61B5/02055—Simultaneously evaluating both cardiovascular condition and temperature
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
- A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
- A61B5/1126—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb using a particular sensing technique
- A61B5/1128—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb using a particular sensing technique using image analysis
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
- A61B5/7267—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems involving training the classification device
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/766—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using regression, e.g. by projecting features on hyperplanes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Abstract
The invention relates to an artificial intelligence technology and discloses a tumbling detection method, a tumbling detection device, tumbling detection equipment and a storage medium based on millimeter wave radar. The method comprises the following steps: when the target user is monitored to be converted from a vertical state to a lying state according to the user point cloud image stream captured by the millimeter wave radar, intercepting the corresponding state conversion point cloud image stream, and recording a time period node corresponding to the state conversion point cloud image stream; identifying the body value change of the target user in the time period node to obtain a body value curve set; and carrying out physical characteristic recognition based on stress reaction and diseases on the body numerical curve set to obtain a physical characteristic set, carrying out dumping action characteristic recognition on the state transition point cloud image stream to obtain a dumping characteristic set, carrying out full-connection classification judgment on the dumping characteristic set and the physical characteristic set, and analyzing the falling state of the target user. The invention can improve the accuracy of tumble detection.
Description
Technical Field
The invention relates to the technical field of artificial intelligence, in particular to a fall detection method, a fall detection device, fall detection equipment and a computer readable storage medium based on millimeter wave radar.
Background
With the development of medical technology and the rising of artificial intelligence technology, intelligent monitoring gradually replaces manual supervision to supervise and manage users, home monitoring equipment is gradually derived, and information such as physical health data, exercise data and sleep data of the users is monitored. In addition, human body characteristics can be captured through a camera shooting technology, wearing equipment and the like, and the falling behavior of a person is alarmed and fallen.
However, today's fall detection is mainly performed through gesture recognition, but gesture monitoring has confusing properties, for example, a user's own lying gesture and a fall gesture, a person's dizziness causes a stun gesture to be confused with a slip or stumbled gesture, and the fall type of fall detection is not clear, the accuracy is not high, and misjudgment often occurs.
Disclosure of Invention
The invention provides a fall detection method, a device, equipment and a storage medium based on a millimeter wave radar, and mainly aims to detect fall in two aspects of gesture and body value of a user through the millimeter wave radar and improve the fall detection accuracy.
In order to achieve the above object, the present invention provides a fall detection method based on millimeter wave radar, comprising:
Capturing a user point cloud image in a target area by utilizing a millimeter wave radar, and storing the user point cloud image acquired at each time point into a preset storage space according to a time sequence to obtain a point cloud image stream;
when the target user is monitored to be converted from the vertical state to the lying state according to the point cloud image flow, intercepting a corresponding state conversion point cloud image flow from the point cloud image flow, and recording a time period node corresponding to the state conversion point cloud image flow;
identifying the body value change of the target user in the time period node by using a preset body value acquisition service to obtain a body value curve set;
performing dumping action feature recognition on the state transition point cloud image stream by using a pre-trained dumping action recognition model to obtain a dumping feature set, and performing physical feature recognition on the body numerical curve set based on stress response and diseases to obtain a body feature set;
and carrying out full-connection classification judgment on the dumping characteristic set and the physical characteristic set, and analyzing the tumbling state of the target user.
Optionally, the performing dumping action feature recognition on the state transition point cloud image stream by using a pre-trained dumping action recognition model to obtain a dumping feature set includes:
Carrying out characteristic engineering operation based on unbalance, falling and impact on a preset action characteristic sample to obtain a falling characteristic type;
extracting whole body action characteristics of the state transition point cloud image flow by utilizing an action recognition network in a pre-trained tumbling action recognition model to obtain an action characteristic sequence set;
and according to the type of the falling feature, extracting the feature in the action feature sequence set to obtain a falling feature set.
Optionally, the step of performing physical feature recognition on the body numerical curve set based on stress response and disease to obtain a physical feature set includes:
performing anomaly monitoring on the body numerical curve set to obtain each anomaly curve in the body numerical curve set and sign information corresponding to each anomaly curve;
performing feature extraction operation on each abnormal curve and the physical sign information corresponding to each abnormal curve by utilizing a disease identification network in the falling action identification model to obtain an abnormal physical sign feature sequence;
and carrying out disease prediction and identification on the abnormal sign feature sequence by utilizing a multi-classification judgment network in the tumbling action identification model, and judging whether the target user has stress reaction or sudden disease or not to obtain a body feature set.
Optionally, before the training of the tumble motion recognition model, the method further includes:
acquiring a tumbling action recognition model comprising an action recognition network, a disease recognition network and a multi-classification judgment network, acquiring a pre-constructed tumbling sample set, and configuring the action recognition network and the disease recognition network as auxiliary tasks;
sequentially extracting a target sample from the falling sample set, and carrying out network forward prediction on the target sample by utilizing the falling action recognition model to respectively obtain an action prediction result, a disease prediction result and a falling prediction result, wherein the falling prediction result comprises a disease sudden motion sickness type, a voluntary lying leaning dumping type and an accidental falling type;
calculating a first auxiliary loss of the action predicted result, a second auxiliary loss of the disease predicted result and a model loss of the fall predicted result according to the real label corresponding to the target sample by using a cross entropy loss algorithm;
according to the first auxiliary loss, the second auxiliary loss and the model loss, the network parameters of the tumbling action recognition model are reversely updated by utilizing a gradient descent method, so that an updated tumbling action recognition model is obtained;
Identifying a convergence of the model loss;
returning to the step of sequentially extracting one target sample from the falling sample set when the model loss is not converged, and iteratively updating the updated falling action recognition model;
and stopping the training process when the model loss converges, and obtaining the trained tumble motion recognition model.
Optionally, the identifying, by using a preset body value acquisition service, a body value change of the target user in the time period node, to obtain a body value curve set includes:
identifying heartbeat signal acquisition and vascular pulse waveforms in the user point cloud image by using a preset body value acquisition service, and carrying out blood pressure model prediction according to the heartbeat signal acquisition and vascular pulse waveforms to obtain a blood pressure identification result;
acquiring a heart rate identification result according to the heartbeat signal, and identifying the body temperature in the user point cloud image to obtain a body temperature identification result;
and recording the blood pressure recognition result, the heart rate recognition result and the body temperature recognition result according to the time period node to obtain a body numerical curve set.
Optionally, before the target user is monitored to be changed from the vertical state to the lying state according to the point cloud image stream, the method further comprises:
Performing trunk regression frame selection on the user point cloud images of each body posture of the target user, and marking the axis of the framed trunk;
identifying an included angle between the axis and a horizontal line;
when the included angle between the axis and the horizontal line is in a first preset interval, judging that the work and rest state of the target user is a vertical state;
when the included angle between the axis and the horizontal line is in a second preset interval, judging that the work and rest state of the target user is a leaning state in the lying state;
and when the included angle between the axis and the horizontal line is in a third preset interval, judging that the work and rest state of the target user is a lying state in the lying state.
Optionally, the capturing the user point cloud image in the target area by using the millimeter wave radar includes:
acquiring an original point cloud image scanned by a millimeter wave radar, and identifying and removing artifacts in the original point cloud image by using a signal processing algorithm to obtain a real image;
noise filtering is carried out on the real image, and a noise reduction image is obtained;
and performing deblurring and contrast enhancement operations on the noise reduction image according to a preset data enhancement strategy to obtain a user point cloud image.
In order to solve the above problems, the present invention also provides a fall detection device based on millimeter wave radar, the device comprising:
The state monitoring module is used for capturing the user point cloud images in the target area by utilizing the millimeter wave radar, and storing the user point cloud images acquired at each time point into a preset storage space according to time sequence to obtain a point cloud image stream;
the data acquisition module is used for intercepting a corresponding state transition point cloud image stream from the point cloud image stream when the target user is monitored to be changed from a vertical state to a lying state according to the point cloud image stream, recording a time period node corresponding to the state transition point cloud image stream, and identifying the body value change of the target user in the time period node by utilizing a preset body value acquisition service to obtain a body value curve set;
the feature extraction module is used for carrying out dumping action feature recognition on the state transition point cloud image flow by utilizing a pre-trained dumping action recognition model to obtain a dumping feature set, and carrying out physical feature recognition on the body numerical curve set based on stress response and diseases to obtain a body feature set;
and the falling judgment module is used for carrying out full-connection classification judgment on the pouring feature set and the physical feature set and analyzing the falling state of the target user.
In order to solve the above-mentioned problems, the present invention also provides an electronic apparatus including:
at least one processor; the method comprises the steps of,
a memory communicatively coupled to the at least one processor; wherein, the liquid crystal display device comprises a liquid crystal display device,
the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the above-described millimeter wave radar-based fall detection method.
In order to solve the above-described problems, the present invention also provides a computer-readable storage medium having stored therein at least one computer program that is executed by a processor in an electronic device to implement the above-described fall detection method based on millimeter wave radar.
The embodiment of the invention firstly monitors the body posture of the user, and starts the falling detection process when the work and rest state of the user is changed into the lying state; the invention mainly detects the falling action and the physical value of the user, wherein the falling action can detect the characteristics of unbalance, falling, impact and the like, and the physical value can detect whether the user generates stress reaction due to careless falling or whether the physical value of the user is in motion due to sudden illness, so that the falling of the user is accurately distinguished, and the actions such as autonomous lying down of the user can be effectively distinguished, thereby improving the accuracy of the falling detection. Therefore, the method, the device, the equipment and the storage medium for detecting the tumbling based on the millimeter wave radar provided by the embodiment of the invention can be used for detecting the tumbling in two aspects of the gesture and the body value of the user through the millimeter wave radar, so that the tumbling detection accuracy is improved.
Drawings
Fig. 1 is a schematic flow chart of a fall detection method based on millimeter wave radar according to an embodiment of the present application;
FIG. 2 is a detailed flowchart of a step in a method for detecting a fall based on millimeter wave radar according to an embodiment of the present application;
FIG. 3 is a detailed flowchart of a step in a method for detecting a fall based on millimeter wave radar according to an embodiment of the present application;
FIG. 4 is a functional block diagram of a fall detection device based on millimeter wave radar according to an embodiment of the present application;
fig. 5 is a schematic structural diagram of an electronic device for implementing the fall detection method based on millimeter wave radar according to an embodiment of the present application.
The achievement of the objects, functional features and advantages of the present application will be further described with reference to the accompanying drawings, in conjunction with the embodiments.
Detailed Description
It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application.
The embodiment of the application provides a tumbling detection method based on millimeter wave radar. In the embodiment of the present application, the execution body of the fall detection method based on the millimeter wave radar includes, but is not limited to, at least one of a server, a terminal, and the like, which can be configured to execute the method provided in the embodiment of the present application. In other words, the fall detection method based on the millimeter wave radar may be performed by software or hardware installed in a terminal device or a server device, and the software may be a blockchain platform. The service end includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, and the like. The server may be an independent server, or may be a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (ContentDelivery Network, CDN), and basic cloud computing services such as big data and artificial intelligence platforms.
Referring to fig. 1, a schematic flow chart of a fall detection method based on millimeter wave radar according to an embodiment of the present invention is shown. In this embodiment, the method for detecting a fall based on millimeter wave radar includes:
s1, capturing a user point cloud image in a target area by utilizing a millimeter wave radar, and storing the user point cloud image acquired at each time point into a preset storage space according to a time sequence to obtain a point cloud image stream.
In the embodiment of the invention, the millimeter wave generated by the millimeter wave radar is positioned in the wavelength range where the microwave and the far infrared wave are intersected, so that the device has the characteristics of two wave spectrums, and is more suitable for the family fall monitoring. The millimeter wave generates a human body point cloud image, the human body point cloud image is composed of a series of three-dimensional coordinate points, each point represents a position in space, and the color of each point can represent the reflection intensity or other attributes of the human body point cloud image. By processing and analyzing the point cloud, not only the human body posture image is detected, but also the body parameters in the human body can be monitored.
In detail, in the embodiment of the present invention, the capturing, by using the millimeter wave radar, the user point cloud image in the target area includes:
Acquiring an original point cloud image scanned by a millimeter wave radar, and identifying and removing artifacts in the original point cloud image by using a signal processing algorithm to obtain a real image;
noise filtering is carried out on the real image, and a noise reduction image is obtained;
and performing deblurring and contrast enhancement operations on the noise reduction image according to a preset data enhancement strategy to obtain a user point cloud image.
According to the embodiment of the invention, false information in the image is reduced through artifact removal operation, then noise filtering is carried out through a filter to reduce the influence on the image quality, and finally the visual effect of the image is enhanced through data enhancement. The data enhancement strategy comprises a deblurring strategy and a contrast enhancement strategy, wherein the deblurring strategy is used for carrying out deblurring treatment on an image so as to improve the definition and detail expression capability of the image; the contrast enhancement strategy is to enhance the outline and detail of the target object in the image by adjusting the gray level and brightness contrast of the image, and finally obtain the available user point cloud image.
In addition, the millimeter wave radar in the embodiment of the invention is used for monitoring in real time, and can monitor video information data within one week, so that the invention constructs a storage space for storing user point cloud image data locally or in a cloud end, and automatically clears overtime data.
S2, when the target user is monitored to be converted from the vertical state to the lying state according to the point cloud image flow, a corresponding state conversion point cloud image flow is intercepted from the point cloud image flow, and a time period node corresponding to the state conversion point cloud image flow is recorded.
The vertical state in the embodiment of the invention can comprise upright, normal walking, sitting and the like, and the lying state can comprise leaning on a wall, sitting on a chair back, lying and the like.
In detail, before S2, the method further includes monitoring whether the target user is in a vertical state or a lying state according to the point cloud image stream by adopting the following method:
performing trunk regression frame selection on the user point cloud images of each body posture of the target user, and marking the axis of the framed trunk;
identifying an included angle between the axis and a horizontal line;
when the included angle between the axis and the horizontal line is in a first preset interval, judging that the work and rest state of the target user is a vertical state;
when the included angle between the axis and the horizontal line is in a second preset interval, judging that the work and rest state of the target user is a leaning state in the lying state;
and when the included angle between the axis and the horizontal line is in a third preset interval, judging that the work and rest state of the target user is a lying state in the lying state.
In the embodiment of the invention, trunk regression frame selection can be performed, then the axis of a person is determined, and the included angle with the horizontal line is calculated. The human vertical state may be determined when the first preset interval having the included angle of 90 ° to 70 °, the resting state may be determined when the second preset interval having the included angle of 70 ° to 30 °, and the lying state may be determined when the third preset interval having the included angle of 30 ° to 0 °. The specific numerical value of each preset interval can be specifically adjusted according to furniture types, personal habits and other conditions in users.
According to the interval where the included angle is located, the embodiment of the invention can judge the work and rest states of the user, wherein the leaning state and the lying state belong to the lying state.
When monitoring that the work and rest states of the user are changed from the vertical state to the lying state in the point cloud images of the adjacent user in the preset time period, intercepting the point cloud images of the adjacent user corresponding to the change of the work and rest states of the user from the vertical state to the lying state from the point cloud image stream, obtaining a state change point cloud image stream, and recording a time period node corresponding to the state change point cloud image stream.
In the embodiment of the invention, the time for lying down or falling down of a person is estimated to be less than 2 seconds, and the time for falling down caused by sudden physical diseases can be 3 to 4 seconds. For example, the user 11 am: 24:20 falls, 11 a.m. can be extracted from the storage space: 24: 15-11 am: 24: the user point cloud image over a 20-time period is analyzed.
S3, recognizing the body value change of the target user in the time period node by using a preset body value acquisition service, and obtaining a body value curve set.
In the embodiment of the invention, the body value acquisition service is a service for detecting body information of a user according to millimeter wave images, and the principle is that heart beating signals and vascular pulse waveform signals of a human body are obtained through millimeter wave radars; extracting characteristic parameters of heart beating and pulse waveforms, such as peak values, waveform forms and the like, through a signal processing algorithm; and combining machine learning or a model algorithm, establishing a relation model between blood pressure and heart beating and pulse waveform characteristics, and finally identifying various body information in the millimeter wave image through the model.
In detail, in the embodiment of the present invention, the identifying, by using a preset body value acquisition service, a body value change of the target user in the time period node, to obtain a body value curve set includes:
identifying heartbeat signal acquisition and vascular pulse waveforms in the user point cloud image by using a preset body value acquisition service, and carrying out blood pressure model prediction according to the heartbeat signal acquisition and vascular pulse waveforms to obtain a blood pressure identification result;
Acquiring a heart rate identification result according to the heartbeat signal, and identifying the body temperature in the user point cloud image to obtain a body temperature identification result;
and recording the blood pressure recognition result, the heart rate recognition result and the body temperature recognition result according to the time period node to obtain a body numerical curve set.
The embodiment of the invention can collect the body temperature, the blood pressure and the heart rate of the target user in the body value collection service, for example, when the user 11 a.m.: 24:20 falls, 11 am can be extracted: 24: 15-11 am: 24: and constructing a body numerical curve set according to the blood pressure identification result, the heart rate identification result and the body temperature identification result in the 20 time period.
S4, utilizing a pre-trained falling action recognition model to perform falling action feature recognition on the state transition point cloud image stream to obtain a falling feature set, and performing physical feature recognition on the body numerical curve set based on stress response and diseases to obtain a body feature set.
In the embodiment of the invention, the tumbling action recognition model is a Transformer-based image recognition model and is used for detecting tumbling from both the behavior action and the body value of the user.
In detail, referring to fig. 2, in an embodiment of the present invention, the performing a pouring action feature recognition on the state transition point cloud image stream to obtain a pouring feature set includes:
s401, carrying out characteristic engineering operation based on unbalance, falling and impact on a preset action characteristic sample to obtain a falling characteristic type;
s402, performing whole body motion feature extraction on the state transition point cloud image flow by utilizing a motion recognition network in a pre-trained tumbling motion recognition model to obtain a motion feature sequence set;
s403, according to the tumbling feature types, extracting features in the action feature sequence set to obtain a dumping feature set.
In the embodiment of the invention, the feature engineering can be performed by a principal component analysis method, and the actions with large influence on the recognition results of unbalance, falling and impact of the action features in the action feature sample are judged, so that the falling feature type is obtained, and then the whole body action feature extraction is performed, so that all the action features of the user, such as facial action features, hand action features, trunk features and the like, are obtained. And further, through relation clustering with the tumbling feature types, a more important dumping feature set is obtained through extraction.
Further, referring to fig. 3, in an embodiment of the present invention, the performing body characteristic recognition on the body numerical curve set based on stress response and disease to obtain a body characteristic set includes:
s411, performing anomaly monitoring on the body numerical curve set to obtain each anomaly curve in the body numerical curve set and sign information corresponding to each anomaly curve;
s412, performing feature extraction operation on each abnormal curve and the sign information corresponding to each abnormal curve by using a disease identification network in the falling action identification model to obtain an abnormal sign feature sequence;
s413, carrying out disease prediction and identification on the abnormal sign feature sequence, and judging whether the target user has stress reaction or sudden disease or not to obtain a body feature set.
It is known that normal people fall down to have stress reaction, epinephrine surge, blood pressure rise, heart rate rise and body temperature rise. Some diseases such as anemia and heart disease can cause conditions such as blood pressure reduction and arrhythmia, so that people are dizziness and faint, and if all the physical indexes do not respond, the user can voluntarily lie down to rest and other behaviors.
Therefore, the embodiment of the invention can detect the body value through the operations of S61-S63 to obtain the body characteristic set, and has an important effect on the subsequent judgment of whether the user falls normally.
S5, carrying out full-connection classification judgment on the dumping feature set and the physical feature set, and analyzing the tumbling state of the target user.
In the embodiment of the invention, after the dumping feature set and the physical feature set are obtained, the features can be sent to the full-connection layer in the dumping action recognition model to carry out multi-classification judgment, so that the user can be judged to lie down or fall down carelessly or fall down caused by disease induction.
In detail, in an embodiment of the present invention, before the training is used to identify the model by using the pre-trained tumbling action, the method further includes:
acquiring a tumbling action recognition model comprising an action recognition network, a disease recognition network and a multi-classification judgment network, acquiring a pre-constructed tumbling sample set, and configuring the action recognition network and the disease recognition network as auxiliary tasks;
sequentially extracting a target sample from the falling sample set, and carrying out network forward prediction on the target sample by utilizing the falling action recognition model to respectively obtain an action prediction result, a disease prediction result and a falling prediction result, wherein the falling prediction result comprises a disease sudden motion sickness type, a voluntary lying leaning dumping type and an accidental falling type;
Calculating a first auxiliary loss of the action predicted result, a second auxiliary loss of the disease predicted result and a model loss of the fall predicted result according to the real label corresponding to the target sample by using a cross entropy loss algorithm;
according to the first auxiliary loss, the second auxiliary loss and the model loss, the network parameters of the tumbling action recognition model are reversely updated by utilizing a gradient descent method, so that an updated tumbling action recognition model is obtained;
identifying a convergence of the model loss;
returning to the step of sequentially extracting one target sample from the falling sample set when the model loss is not converged, and iteratively updating the updated falling action recognition model;
and stopping the training process when the model loss converges, and obtaining the trained tumble motion recognition model.
In the embodiment of the invention, the falling action recognition model comprises an action recognition network, a disease recognition network and a multi-classification judgment network, wherein the action recognition network and the disease recognition network are responsible for feature extraction, and the extraction result is a model intermediate link, so the invention configures the action recognition network and the disease recognition network as auxiliary tasks and carries out auxiliary training on the output results of the action recognition network and the disease recognition network.
According to the auxiliary task, the method uses an auxiliary loss method, the recognition results of the action recognition network and the disease recognition network as intermediate links, the loss is calculated independently, and the loss of the multi-classification judgment network is used as model loss. Therefore, the motion recognition and disease analysis process of the falling motion recognition model is only carried out in the computer training process, the falling motion recognition and disease analysis process does not occur in the training progress display process, and only the output result of the multi-classification judgment network of the falling motion recognition model is displayed in the training progress display process. The increased auxiliary loss in the training process can accelerate convergence, improve model training efficiency, enhance supervision and enhance counter-propagation of gradients, so that the falling sample set can perform machine learning of two actions more efficiently.
The cross entropy loss algorithm and the gradient descent method are adopted in the training process. The cross entropy loss algorithm is used for monitoring the real action label of the target sample and the first auxiliary loss of the action prediction result, the real physical disease analysis label of the target sample and the second auxiliary loss of the disease prediction result and the model loss between the real falling result label of the target sample and the falling prediction result, and the gradient descent method is a method for calculating a function minimum value and is used for updating model network parameters through loss values.
Then, the embodiment of the invention controls the model training progress in a mode of controlling model loss convergence, when the model loss is not converged, the model training effect is obvious, the model training can be continued, when the model loss is converged, the model training progress effect is not obvious, and in order to avoid the model overfitting phenomenon, the training process can be stopped, so that the training-completed falling action recognition model is obtained.
In addition, in another embodiment of the present invention, after the target user falls, the body numerical curve set within a preset time period after falling, for example, 5 minutes, may be analyzed, so as to avoid internal injury that the user cannot feel, and when the body numerical curve set is abnormal, the alarm may be timely given, so as to improve the user security.
The embodiment of the invention firstly monitors the body posture of the user, and starts the falling detection process when the work and rest state of the user is changed into the lying state; the invention mainly detects the falling action and the physical value of the user, wherein the falling action can detect the characteristics of unbalance, falling, impact and the like, and the physical value can detect whether the user generates stress reaction due to careless falling or whether the physical value of the user is in motion due to sudden illness, so that the falling of the user is accurately distinguished, and the actions such as autonomous lying down of the user can be effectively distinguished, thereby improving the accuracy of the falling detection. Therefore, the method for detecting the tumbling of the millimeter wave radar provided by the embodiment of the invention can be used for detecting the tumbling of the gesture and the body value of the user through the millimeter wave radar, and the tumbling detection accuracy is improved.
Fig. 4 is a functional block diagram of a fall detection device based on millimeter wave radar according to an embodiment of the present invention.
The fall detection device 100 based on millimeter wave radar of the present invention may be installed in an electronic apparatus. According to the implemented functions, the fall detection device 100 based on millimeter wave radar may include a state monitoring module 101, a data acquisition module 102, a feature extraction module 103, and a fall judgment module 104. The module of the invention, which may also be referred to as a unit, refers to a series of computer program segments, which are stored in the memory of the electronic device, capable of being executed by the processor of the electronic device and of performing a fixed function.
In the present embodiment, the functions concerning the respective modules/units are as follows:
the state monitoring module 101 is configured to capture a user point cloud image in a target area by using a millimeter wave radar, and store the user point cloud image acquired at each time point into a preset storage space according to a time sequence to obtain a point cloud image stream;
the data acquisition module 102 is configured to, when it is monitored according to the point cloud image stream that the target user is changed from a vertical state to a lying state, intercept a corresponding state transition point cloud image stream from the point cloud image stream, record a time period node corresponding to the state transition point cloud image stream, and identify a body value change of the target user in the time period node by using a preset body value acquisition service, so as to obtain a body value curve set;
The feature extraction module 103 is configured to perform dumping action feature recognition on the state transition point cloud image stream by using a pre-trained dumping action recognition model to obtain a dumping feature set, and perform physical feature recognition on the body numerical curve set based on stress response and disease to obtain a body feature set;
the fall judgment module 104 is configured to perform full-connection classification judgment on the dumping feature set and the physical feature set, and analyze a fall state of the target user.
In detail, each module in the fall detection device 100 of the millimeter wave radar according to the embodiment of the present application adopts the same technical means as the fall detection method of the millimeter wave radar described in fig. 1 to 3, and can produce the same technical effects, which are not described herein.
Fig. 5 is a schematic structural diagram of an electronic device 1 for implementing a fall detection method based on millimeter wave radar according to an embodiment of the present application.
The electronic device 1 may comprise a processor 10, a memory 11, a communication bus 12 and a communication interface 13, and may further comprise a computer program stored in the memory 11 and executable on the processor 10, such as a fall detection program based on a millimeter wave radar.
The processor 10 may be formed by an integrated circuit in some embodiments, for example, a single packaged integrated circuit, or may be formed by a plurality of integrated circuits packaged with the same function or different functions, including one or more central processing units (Central Processing Unit, CPU), a microprocessor, a digital processing chip, a graphics processor, a combination of various control chips, and so on. The processor 10 is a Control Unit (Control Unit) of the electronic apparatus 1, connects respective parts of the entire electronic apparatus using various interfaces and lines, executes various functions of the electronic apparatus and processes data by running or executing programs or modules stored in the memory 11 (for example, executing a fall detection program based on millimeter wave radar, etc.), and recalls data stored in the memory 11.
The memory 11 includes at least one type of readable storage medium including flash memory, a removable hard disk, a multimedia card, a card type memory (e.g., SD or DX memory, etc.), a magnetic memory, a magnetic disk, an optical disk, etc. The memory 11 may in some embodiments be an internal storage unit of the electronic device, such as a mobile hard disk of the electronic device. The memory 11 may in other embodiments also be an external storage device of the electronic device, such as a plug-in mobile hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card) or the like, which are provided on the electronic device. Further, the memory 11 may also include both an internal storage unit and an external storage device of the electronic device. The memory 11 may be used not only for storing application software installed in an electronic device and various types of data such as codes of fall detection programs of millimeter wave radars, but also for temporarily storing data that has been output or is to be output.
The communication bus 12 may be a peripheral component interconnect standard (Peripheral Component Interconnect, PCI) bus, or an extended industry standard architecture (Extended Industry Standard Architecture, EISA) bus, among others. The bus may be classified as an address bus, a data bus, a control bus, etc. The bus is arranged to enable a connection communication between the memory 11 and at least one processor 10 etc.
The communication interface 13 is used for communication between the electronic device 1 and other devices, including a network interface and a user interface. Optionally, the network interface may include a wired interface and/or a wireless interface (e.g., WI-FI interface, bluetooth interface, etc.), typically used to establish a communication connection between the electronic device and other electronic devices. The user interface may be a Display (Display), an input unit such as a Keyboard (Keyboard), or alternatively a standard wired interface, a wireless interface. Alternatively, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch, or the like. The display may also be referred to as a display screen or display unit, as appropriate, for displaying information processed in the electronic device and for displaying a visual user interface.
Fig. 5 shows only an electronic device with components, it being understood by a person skilled in the art that the structure shown in fig. 5 does not constitute a limitation of the electronic device 1, and may comprise fewer or more components than shown, or may combine certain components, or may be arranged in different components.
For example, although not shown, the electronic device 1 may further include a power source (such as a battery) for supplying power to each component, and preferably, the power source may be logically connected to the at least one processor 10 through a power management device, so that functions of charge management, discharge management, power consumption management, and the like are implemented through the power management device. The power supply may also include one or more of any of a direct current or alternating current power supply, recharging device, power failure detection circuit, power converter or inverter, power status indicator, etc. The electronic device 1 may further include various sensors, bluetooth modules, wi-Fi modules, etc., which will not be described herein.
It should be understood that the embodiments described are for illustrative purposes only and are not limited to this configuration in the scope of the patent application.
The fall detection program based on millimeter wave radar stored in the memory 11 in the electronic device 1 is a combination of a plurality of instructions, which when executed in the processor 10, can realize:
Capturing a user point cloud image in a target area by utilizing a millimeter wave radar, and storing the user point cloud image acquired at each time point into a preset storage space according to a time sequence to obtain a point cloud image stream;
when the target user is monitored to be converted from the vertical state to the lying state according to the point cloud image flow, intercepting a corresponding state conversion point cloud image flow from the point cloud image flow, and recording a time period node corresponding to the state conversion point cloud image flow;
identifying the body value change of the target user in the time period node by using a preset body value acquisition service to obtain a body value curve set;
performing dumping action feature recognition on the state transition point cloud image stream by using a pre-trained dumping action recognition model to obtain a dumping feature set, and performing physical feature recognition on the body numerical curve set based on stress response and diseases to obtain a body feature set;
and carrying out full-connection classification judgment on the dumping characteristic set and the physical characteristic set, and analyzing the tumbling state of the target user.
In particular, the specific implementation method of the above instructions by the processor 10 may refer to the description of the relevant steps in the corresponding embodiment of the drawings, which is not repeated herein.
Further, the modules/units integrated in the electronic device 1 may be stored in a computer readable storage medium if implemented in the form of software functional units and sold or used as separate products. The computer readable storage medium may be volatile or nonvolatile. For example, the computer readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a U disk, a removable hard disk, a magnetic disk, an optical disk, a computer Memory, a Read-Only Memory (ROM).
The present invention also provides a computer readable storage medium storing a computer program which, when executed by a processor of an electronic device, can implement:
capturing a user point cloud image in a target area by utilizing a millimeter wave radar, and storing the user point cloud image acquired at each time point into a preset storage space according to a time sequence to obtain a point cloud image stream;
when the target user is monitored to be converted from the vertical state to the lying state according to the point cloud image flow, intercepting a corresponding state conversion point cloud image flow from the point cloud image flow, and recording a time period node corresponding to the state conversion point cloud image flow;
Identifying the body value change of the target user in the time period node by using a preset body value acquisition service to obtain a body value curve set;
performing dumping action feature recognition on the state transition point cloud image stream by using a pre-trained dumping action recognition model to obtain a dumping feature set, and performing physical feature recognition on the body numerical curve set based on stress response and diseases to obtain a body feature set;
and carrying out full-connection classification judgment on the dumping characteristic set and the physical characteristic set, and analyzing the tumbling state of the target user.
In the several embodiments provided in the present invention, it should be understood that the disclosed apparatus, device and method may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the modules is merely a logical function division, and there may be other manners of division when actually implemented.
The modules described as separate components may or may not be physically separate, and components shown as modules may or may not be physical units, may be located in one place, or may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional module in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units can be realized in a form of hardware or a form of hardware and a form of software functional modules.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof.
The present embodiments are, therefore, to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference signs in the claims shall not be construed as limiting the claim concerned.
The blockchain is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, consensus mechanism, encryption algorithm and the like. The Blockchain (Blockchain), which is essentially a decentralised database, is a string of data blocks that are generated by cryptographic means in association, each data block containing a batch of information of network transactions for verifying the validity of the information (anti-counterfeiting) and generating the next block. The blockchain may include a blockchain underlying platform, a platform product services layer, an application services layer, and the like.
The embodiment of the application can acquire and process the related data based on the artificial intelligence technology. Among these, artificial intelligence (Artificial Intelligence, AI) is the theory, method, technique and application system that uses a digital computer or a digital computer-controlled machine to simulate, extend and extend human intelligence, sense the environment, acquire knowledge and use knowledge to obtain optimal results.
Furthermore, it is evident that the word "comprising" does not exclude other elements or steps, and that the singular does not exclude a plurality. A plurality of units or means recited in the system claims can also be implemented by means of software or hardware by means of one unit or means. The terms first, second, etc. are used to denote a name, but not any particular order.
Finally, it should be noted that the above-mentioned embodiments are merely for illustrating the technical solution of the present application and not for limiting the same, and although the present application has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications and equivalents may be made to the technical solution of the present application without departing from the spirit and scope of the technical solution of the present application.
Claims (5)
1. The tumbling detection method based on the millimeter wave radar is characterized by comprising the following steps of:
Capturing a user point cloud image in a target area by utilizing a millimeter wave radar, and storing the user point cloud image acquired at each time point into a preset storage space according to a time sequence to obtain a point cloud image stream;
when the target user is monitored to be converted from the vertical state to the lying state according to the point cloud image flow, intercepting a corresponding state conversion point cloud image flow from the point cloud image flow, and recording a time period node corresponding to the state conversion point cloud image flow;
identifying the body value change of the target user in the time period node by using a preset body value acquisition service to obtain a body value curve set;
performing tumbling action feature recognition on the state transition point cloud image flow by using a pre-trained tumbling action recognition model to obtain a tumbling feature set, and performing physical feature recognition on the body numerical curve set based on stress response and diseases to obtain a body feature set;
performing full-connection classification judgment on the tumbling feature set and the physical feature set, and analyzing the tumbling state of the target user; the method for identifying the tumbling action features of the cloud image flow of the state transition points by utilizing the pre-trained tumbling action identification model comprises the following steps of:
Carrying out characteristic engineering operation based on unbalance, falling and impact on a preset action characteristic sample to obtain a falling characteristic type;
extracting whole body action characteristics of the state transition point cloud image flow by utilizing an action recognition network in a pre-trained tumbling action recognition model to obtain an action characteristic sequence set;
according to the type of the falling feature, extracting the action feature in the action feature sequence set to obtain a falling feature set; the body characteristic set is obtained by carrying out body characteristic identification based on stress response and diseases on the body numerical curve set, and the method comprises the following steps:
performing anomaly monitoring on the body numerical curve set to obtain each anomaly curve in the body numerical curve set and sign information corresponding to each anomaly curve;
performing feature extraction operation on each abnormal curve and the physical sign information corresponding to each abnormal curve by utilizing a disease identification network in the falling action identification model to obtain an abnormal physical sign feature sequence;
performing disease prediction recognition on the abnormal sign feature sequence by utilizing a multi-classification judgment network in the tumbling action recognition model, and judging whether the target user has stress reaction or sudden disease or not to obtain a body feature set; wherein, before the tumble motion recognition model is trained, the method further comprises:
Acquiring a tumbling action recognition model comprising an action recognition network, a disease recognition network and a multi-classification judgment network, acquiring a pre-constructed tumbling sample set, and configuring the action recognition network and the disease recognition network as auxiliary tasks;
sequentially extracting a target sample from the falling sample set, and carrying out network forward prediction on the target sample by utilizing the falling action recognition model to respectively obtain an action prediction result, a disease prediction result and a falling prediction result, wherein the falling prediction result comprises a disease sudden motion sickness type, a voluntary lying leaning falling type and an accidental falling type;
calculating a first auxiliary loss of the action predicted result, a second auxiliary loss of the disease predicted result and a model loss of the fall predicted result according to the real label corresponding to the target sample by using a cross entropy loss algorithm;
according to the first auxiliary loss, the second auxiliary loss and the model loss, the network parameters of the tumbling action recognition model are reversely updated by utilizing a gradient descent method, so that an updated tumbling action recognition model is obtained;
identifying a convergence of the model loss;
returning to the step of sequentially extracting one target sample from the falling sample set when the model loss is not converged, and iteratively updating the updated falling action recognition model;
Stopping the training process when the model loss converges to obtain a training-completed falling action recognition model; the step of identifying the body value change of the target user in the time period node by using a preset body value acquisition service to obtain a body value curve set comprises the following steps:
identifying heartbeat signals and vascular pulse waveforms in the user point cloud image by using a preset body value acquisition service, and predicting a blood pressure model according to the heartbeat signals and the vascular pulse waveforms to obtain a blood pressure identification result;
obtaining a heart rate identification result according to the heartbeat signal, and identifying the body temperature in the user point cloud image to obtain a body temperature identification result;
recording the blood pressure recognition result, the heart rate recognition result and the body temperature recognition result according to the time period node to obtain a body numerical curve set; wherein, before the target user is monitored to be changed from the vertical state to the lying state according to the point cloud image stream, the method further comprises:
performing trunk regression frame selection on the user point cloud images of each body posture of the target user, and marking the axis of the framed trunk;
Identifying an included angle between the axis and a horizontal line;
when the included angle between the axis and the horizontal line is in a first preset interval, judging that the work and rest state of the target user is a vertical state;
when the included angle between the axis and the horizontal line is in a second preset interval, judging that the work and rest state of the target user is a leaning state in the lying state;
and when the included angle between the axis and the horizontal line is in a third preset interval, judging that the work and rest state of the target user is a lying state in the lying state.
2. The method for detecting a fall based on a millimeter wave radar according to claim 1, wherein the capturing of the user point cloud image in the target area using the millimeter wave radar comprises:
acquiring an original point cloud image scanned by a millimeter wave radar, and identifying and removing artifacts in the original point cloud image by using a signal processing algorithm to obtain a real image;
noise filtering is carried out on the real image, and a noise reduction image is obtained;
and performing deblurring and contrast enhancement operations on the noise reduction image according to a preset data enhancement strategy to obtain a user point cloud image.
3. A fall detection device based on millimeter wave radar, the device comprising:
The state monitoring module is used for capturing the user point cloud images in the target area by utilizing the millimeter wave radar, and storing the user point cloud images acquired at each time point into a preset storage space according to time sequence to obtain a point cloud image stream;
the data acquisition module is used for intercepting a corresponding state transition point cloud image stream from the point cloud image stream when the target user is monitored to be changed from a vertical state to a lying state according to the point cloud image stream, recording a time period node corresponding to the state transition point cloud image stream, and identifying the body value change of the target user in the time period node by utilizing a preset body value acquisition service to obtain a body value curve set;
the characteristic extraction module is used for carrying out tumble action characteristic recognition on the state transition point cloud image flow by utilizing a pre-trained tumble action recognition model to obtain a tumble characteristic set, and carrying out physical characteristic recognition on the body numerical curve set based on stress response and diseases to obtain a physical characteristic set;
the falling judgment module is used for carrying out full-connection classification judgment on the falling feature set and the physical feature set and analyzing the falling state of the target user;
The method for identifying the tumbling action features of the cloud image flow of the state transition points by utilizing the pre-trained tumbling action identification model comprises the following steps of:
carrying out characteristic engineering operation based on unbalance, falling and impact on a preset action characteristic sample to obtain a falling characteristic type;
extracting whole body action characteristics of the state transition point cloud image flow by utilizing an action recognition network in a pre-trained tumbling action recognition model to obtain an action characteristic sequence set;
according to the type of the falling feature, extracting the action feature in the action feature sequence set to obtain a falling feature set;
the body characteristic set is obtained by carrying out body characteristic identification based on stress response and diseases on the body numerical curve set, and the method comprises the following steps:
performing anomaly monitoring on the body numerical curve set to obtain each anomaly curve in the body numerical curve set and sign information corresponding to each anomaly curve;
performing feature extraction operation on each abnormal curve and the physical sign information corresponding to each abnormal curve by utilizing a disease identification network in the falling action identification model to obtain an abnormal physical sign feature sequence;
Performing disease prediction recognition on the abnormal sign feature sequence by utilizing a multi-classification judgment network in the tumbling action recognition model, and judging whether the target user has stress reaction or sudden disease or not to obtain a body feature set;
wherein, before utilizing the pre-trained tumble motion recognition model, further comprises:
acquiring a tumbling action recognition model comprising an action recognition network, a disease recognition network and a multi-classification judgment network, acquiring a pre-constructed tumbling sample set, and configuring the action recognition network and the disease recognition network as auxiliary tasks;
sequentially extracting a target sample from the falling sample set, and carrying out network forward prediction on the target sample by utilizing the falling action recognition model to respectively obtain an action prediction result, a disease prediction result and a falling prediction result, wherein the falling prediction result comprises a disease sudden motion sickness type, a voluntary lying leaning falling type and an accidental falling type;
calculating a first auxiliary loss of the action predicted result, a second auxiliary loss of the disease predicted result and a model loss of the fall predicted result according to the real label corresponding to the target sample by using a cross entropy loss algorithm;
According to the first auxiliary loss, the second auxiliary loss and the model loss, the network parameters of the tumbling action recognition model are reversely updated by utilizing a gradient descent method, so that an updated tumbling action recognition model is obtained;
identifying a convergence of the model loss;
returning to the step of sequentially extracting one target sample from the falling sample set when the model loss is not converged, and iteratively updating the updated falling action recognition model;
stopping the training process when the model loss converges to obtain a training-completed falling action recognition model;
the step of identifying the body value change of the target user in the time period node by using a preset body value acquisition service to obtain a body value curve set comprises the following steps:
identifying heartbeat signals and vascular pulse waveforms in the user point cloud image by using a preset body value acquisition service, and predicting a blood pressure model according to the heartbeat signals and the vascular pulse waveforms to obtain a blood pressure identification result;
obtaining a heart rate identification result according to the heartbeat signal, and identifying the body temperature in the user point cloud image to obtain a body temperature identification result;
Recording the blood pressure recognition result, the heart rate recognition result and the body temperature recognition result according to the time period node to obtain a body numerical curve set;
before the target user is monitored to be changed from the vertical state to the lying state according to the point cloud image stream, the method further comprises the following steps:
performing trunk regression frame selection on the user point cloud images of each body posture of the target user, and marking the axis of the framed trunk;
identifying an included angle between the axis and a horizontal line;
when the included angle between the axis and the horizontal line is in a first preset interval, judging that the work and rest state of the target user is a vertical state;
when the included angle between the axis and the horizontal line is in a second preset interval, judging that the work and rest state of the target user is a leaning state in the lying state;
and when the included angle between the axis and the horizontal line is in a third preset interval, judging that the work and rest state of the target user is a lying state in the lying state.
4. An electronic device, the electronic device comprising:
at least one processor; the method comprises the steps of,
a memory communicatively coupled to the at least one processor; wherein, the liquid crystal display device comprises a liquid crystal display device,
the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the millimeter wave radar-based fall detection method according to any one of claims 1 to 2.
5. A computer-readable storage medium storing a computer program, wherein the computer program when executed by a processor implements the fall detection method based on millimeter wave radar according to any one of claims 1 to 2.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310975071.2A CN116687394B (en) | 2023-08-04 | 2023-08-04 | Tumble detection method, device, equipment and storage medium based on millimeter wave radar |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310975071.2A CN116687394B (en) | 2023-08-04 | 2023-08-04 | Tumble detection method, device, equipment and storage medium based on millimeter wave radar |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116687394A CN116687394A (en) | 2023-09-05 |
| CN116687394B true CN116687394B (en) | 2023-10-20 |
Family
ID=87843652
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310975071.2A Active CN116687394B (en) | 2023-08-04 | 2023-08-04 | Tumble detection method, device, equipment and storage medium based on millimeter wave radar |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116687394B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117589128B (en) * | 2024-01-17 | 2024-04-12 | 山东交通学院 | Method, equipment and medium for identifying road cone attitude change |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111166342A (en) * | 2020-01-07 | 2020-05-19 | 四川宇然智荟科技有限公司 | Millimeter wave radar and camera fused fall detection device and detection method thereof |
| CN112215185A (en) * | 2020-10-21 | 2021-01-12 | 成都信息工程大学 | System and method for detecting falling behavior from monitoring video |
| CN113589276A (en) * | 2021-07-05 | 2021-11-02 | 镇江同润智能科技有限公司 | Fall detection method based on millimeter wave radar |
| CN113827216A (en) * | 2021-09-18 | 2021-12-24 | 特斯联科技集团有限公司 | Method and system for sensorless heart rate monitoring based on micro-motion algorithm |
| CN114283494A (en) * | 2021-12-14 | 2022-04-05 | 联仁健康医疗大数据科技股份有限公司 | Early warning method, device, equipment and storage medium for user falling |
| CN115089135A (en) * | 2022-04-25 | 2022-09-23 | 无锡博奥玛雅医学科技有限公司 | Millimeter wave radar-based elderly health state detection method and system |
| CN116106855A (en) * | 2023-04-13 | 2023-05-12 | 中国科学技术大学 | Tumble detection method and tumble detection device |
| CN116236173A (en) * | 2023-03-02 | 2023-06-09 | 康力元(天津)医疗科技有限公司 | Intelligent care monitoring management system based on millimeter wave radar |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP6356616B2 (en) * | 2015-02-17 | 2018-07-11 | 日本電信電話株式会社 | Sequential posture identification device, autonomic nerve function information acquisition device, method and program |
| US10262423B2 (en) * | 2016-03-29 | 2019-04-16 | Verily Life Sciences Llc | Disease and fall risk assessment using depth mapping systems |
| WO2019103620A2 (en) * | 2017-11-21 | 2019-05-31 | Omniscient Medical As | System, sensor and method for monitoring health related aspects of a patient |
| US11568262B2 (en) * | 2020-03-25 | 2023-01-31 | Ventech Solutions, Inc. | Neural network based radiowave monitoring of fall characteristics in injury diagnosis |
-
2023
- 2023-08-04 CN CN202310975071.2A patent/CN116687394B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111166342A (en) * | 2020-01-07 | 2020-05-19 | 四川宇然智荟科技有限公司 | Millimeter wave radar and camera fused fall detection device and detection method thereof |
| CN112215185A (en) * | 2020-10-21 | 2021-01-12 | 成都信息工程大学 | System and method for detecting falling behavior from monitoring video |
| CN113589276A (en) * | 2021-07-05 | 2021-11-02 | 镇江同润智能科技有限公司 | Fall detection method based on millimeter wave radar |
| CN113827216A (en) * | 2021-09-18 | 2021-12-24 | 特斯联科技集团有限公司 | Method and system for sensorless heart rate monitoring based on micro-motion algorithm |
| CN114283494A (en) * | 2021-12-14 | 2022-04-05 | 联仁健康医疗大数据科技股份有限公司 | Early warning method, device, equipment and storage medium for user falling |
| CN115089135A (en) * | 2022-04-25 | 2022-09-23 | 无锡博奥玛雅医学科技有限公司 | Millimeter wave radar-based elderly health state detection method and system |
| CN116236173A (en) * | 2023-03-02 | 2023-06-09 | 康力元(天津)医疗科技有限公司 | Intelligent care monitoring management system based on millimeter wave radar |
| CN116106855A (en) * | 2023-04-13 | 2023-05-12 | 中国科学技术大学 | Tumble detection method and tumble detection device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116687394A (en) | 2023-09-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN116687394B (en) | Tumble detection method, device, equipment and storage medium based on millimeter wave radar | |
| CN109920208A (en) | Tumble prediction technique, device, electronic equipment and system | |
| CN116311539B (en) | Sleep motion capturing method, device, equipment and storage medium based on millimeter waves | |
| US20210319585A1 (en) | Method and system for gaze estimation | |
| CN109919179A (en) | Aneurysms automatic testing method, device and computer readable storage medium | |
| CN111797756A (en) | Video analysis method, device and medium based on artificial intelligence | |
| CN109674474B (en) | Sleep apnea recognition method, device and computer readable medium | |
| CN114203300A (en) | Health state evaluation method and system, server and storage medium | |
| CN116942149B (en) | Lumbar vertebra monitoring method, device, equipment and storage medium based on millimeter wave radar | |
| CN116313164B (en) | Anti-interference sleep monitoring method, device, equipment and storage medium | |
| CN113435353A (en) | Multi-mode-based in-vivo detection method and device, electronic equipment and storage medium | |
| CN114187561A (en) | Abnormal behavior identification method and device, terminal equipment and storage medium | |
| CN112580543B (en) | Behavior recognition method, system and device | |
| CN114864094A (en) | Intensive care unit intelligent control method, equipment and medium | |
| CN112487980A (en) | Micro-expression-based treatment method, device, system and computer-readable storage medium | |
| CN115268531A (en) | Water flow temperature regulation control method, device, equipment and storage medium for intelligent bathtub | |
| CN115721283A (en) | Heart rate health early warning method and system, computer equipment and medium | |
| CN114816055A (en) | Eyeball motion track capturing and analyzing method, device and medium based on VR equipment | |
| CN117253096B (en) | Finger-adaptive health index monitoring method, device, equipment and storage medium | |
| CN113499037A (en) | Method and device for determining sleep quality of user, electronic equipment and storage medium | |
| CN114612098A (en) | Transaction information processing method and device and computer readable storage medium | |
| CN117594227B (en) | Health state monitoring method, device, medium and equipment based on wearable equipment | |
| CN114373549B (en) | Self-adaptive exercise prescription health intervention method and system for old people | |
| KR102605576B1 (en) | Psychological Evaluation System based on Multidimensional Virtual Reality Phenotype | |
| CN117598711B (en) | QRS complex detection method, device, equipment and medium for electrocardiosignal |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |