CN116942142A - Personnel intelligent monitoring method based on millimeter wave radar and related equipment - Google Patents
Personnel intelligent monitoring method based on millimeter wave radar and related equipment Download PDFInfo
- Publication number
- CN116942142A CN116942142A CN202310673735.XA CN202310673735A CN116942142A CN 116942142 A CN116942142 A CN 116942142A CN 202310673735 A CN202310673735 A CN 202310673735A CN 116942142 A CN116942142 A CN 116942142A
- Authority
- CN
- China
- Prior art keywords
- information
- personnel
- millimeter wave
- monitoring
- distance
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 74
- 238000000034 method Methods 0.000 title claims abstract description 62
- 230000033001 locomotion Effects 0.000 claims abstract description 20
- 238000012545 processing Methods 0.000 claims abstract description 13
- 238000007781 pre-processing Methods 0.000 claims abstract description 11
- 238000013135 deep learning Methods 0.000 claims abstract description 7
- 239000011159 matrix material Substances 0.000 claims description 25
- 230000006870 function Effects 0.000 claims description 11
- 238000013136 deep learning model Methods 0.000 claims description 10
- 238000000605 extraction Methods 0.000 claims description 10
- 238000004891 communication Methods 0.000 claims description 9
- 230000004927 fusion Effects 0.000 claims description 9
- 238000011176 pooling Methods 0.000 claims description 8
- 239000012535 impurity Substances 0.000 claims description 6
- 238000000354 decomposition reaction Methods 0.000 claims description 4
- 238000001514 detection method Methods 0.000 claims description 4
- 238000001914 filtration Methods 0.000 claims description 4
- 230000029058 respiratory gaseous exchange Effects 0.000 claims description 4
- 238000001228 spectrum Methods 0.000 claims description 4
- 230000003068 static effect Effects 0.000 claims description 4
- 238000004458 analytical method Methods 0.000 claims description 3
- 238000012935 Averaging Methods 0.000 claims description 2
- 230000008859 change Effects 0.000 claims description 2
- 238000006073 displacement reaction Methods 0.000 claims description 2
- 230000000241 respiratory effect Effects 0.000 claims description 2
- 230000036387 respiratory rate Effects 0.000 claims description 2
- 238000000926 separation method Methods 0.000 claims description 2
- 230000009466 transformation Effects 0.000 claims description 2
- 230000000474 nursing effect Effects 0.000 abstract description 2
- 230000008569 process Effects 0.000 description 9
- 238000004590 computer program Methods 0.000 description 3
- 238000005259 measurement Methods 0.000 description 3
- 230000032683 aging Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000005286 illumination Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 208000035473 Communicable disease Diseases 0.000 description 1
- 230000005856 abnormality Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000002592 echocardiography Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
- G08B21/02—Alarms for ensuring the safety of persons
- G08B21/04—Alarms for ensuring the safety of persons responsive to non-activity, e.g. of elderly persons
- G08B21/0407—Alarms for ensuring the safety of persons responsive to non-activity, e.g. of elderly persons based on behaviour analysis
- G08B21/043—Alarms for ensuring the safety of persons responsive to non-activity, e.g. of elderly persons based on behaviour analysis detecting an emergency event, e.g. a fall
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/0002—Remote monitoring of patients using telemetry, e.g. transmission of vital signals via a communication network
- A61B5/0015—Remote monitoring of patients using telemetry, e.g. transmission of vital signals via a communication network characterised by features of the telemetry system
- A61B5/002—Monitoring the patient using a local or closed circuit, e.g. in a room or building
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
- A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
- A61B5/1113—Local tracking of patients, e.g. in a hospital or private home
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
- A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
- A61B5/1116—Determining posture transitions
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
- A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
- A61B5/1116—Determining posture transitions
- A61B5/1117—Fall detection
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
- G01S13/88—Radar or analogous systems specially adapted for specific applications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
- G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Medical Informatics (AREA)
- Molecular Biology (AREA)
- Evolutionary Computation (AREA)
- Animal Behavior & Ethology (AREA)
- Veterinary Medicine (AREA)
- Heart & Thoracic Surgery (AREA)
- Pathology (AREA)
- Software Systems (AREA)
- Public Health (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Surgery (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Remote Sensing (AREA)
- Radar, Positioning & Navigation (AREA)
- Physiology (AREA)
- Dentistry (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Multimedia (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Electromagnetism (AREA)
- Emergency Management (AREA)
- Business, Economics & Management (AREA)
- Gerontology & Geriatric Medicine (AREA)
- Social Psychology (AREA)
- Psychology (AREA)
Abstract
The application discloses a personnel intelligent monitoring method and related equipment based on millimeter wave radar, and relates to the technical field of nursing monitoring, wherein the method comprises the following steps: transmitting millimeter wave signals and receiving radar original echo data; preprocessing according to the radar original echo data to obtain time domain feature information of a target human body; obtaining monitoring information of a target human body by processing according to the time domain characteristic information and adopting a deep learning method, wherein the monitoring information at least comprises the number, the position, the movement track, vital sign information and gesture recognition information of people; and generating corresponding alarm information and control instructions according to the monitoring information, wherein the alarm information is sent to a set crowd, and the control instructions are sent to set household appliances. The application can acquire various monitoring data of personnel in real time, is non-contact, is simple and convenient, and can realize intelligent monitoring of personnel and intelligent control of household appliances.
Description
Technical Field
The application relates to the technical field of nursing monitoring, in particular to a personnel intelligent monitoring method based on millimeter wave radar and related equipment.
Background
The current aging trend of society is aggravated, the number of aging population is continuously increased, and the influence of infectious diseases causes shortage of medical resources and increases the pressure of personnel monitoring, so that monitoring equipment is required to realize real-time monitoring of personnel in daily life.
Traditional solutions, such as relying on wearable sensors, such as smart bracelets, sleeping straps, etc., belong to contact measurement, can realize measurement of vital sign information, but are uncomfortable to wear, and are easy to cause inconvenience to daily life. The other type of solution with the help of the camera can realize the position monitoring and gesture recognition of personnel, but needs good illumination condition, sight distance condition, can not work in all weather, has the risk of privacy disclosure, and does not have stable working performance.
However, due to the development of millimeter wave technology, the size of the millimeter wave radar can be made smaller, the cost is continuously reduced, and the millimeter wave radar-based monitoring equipment has high civil value, so that the millimeter wave radar-based monitoring equipment is expected to become a new solution, and has a large application prospect. The millimeter wave radar not only can acquire the position and the movement track of the personnel, but also can monitor vital sign information and identify the gesture of the personnel, can monitor in real time, is not influenced by factors such as illumination, sight distance, privacy and the like, is non-contact, and can work throughout the day. At present, monitoring equipment for identifying the positions and the movement tracks of people, vital signs and gestures does not exist, so that the monitoring equipment based on millimeter wave radar is researched, and the monitoring equipment can be applied to various scenes such as families and hospitals, is convenient and quick, can early warn vital sign abnormality and early warn falling of old people.
Disclosure of Invention
Aiming at the defects in the prior art, the application provides the personnel intelligent monitoring method and the related equipment based on the millimeter wave radar, which can monitor the positions, the motion trail, the vital signs and the personnel gestures of personnel at the same time, can be suitable for various living scenes such as families, hospitals and the like, and can increase convenience for daily life.
In order to achieve the above purpose, the present application provides the following technical solutions:
in a first aspect, the present application provides a personnel intelligent monitoring method based on millimeter wave radar, which includes:
transmitting millimeter wave signals and receiving radar original echo data;
preprocessing according to the radar original echo data to obtain time domain feature information of a target human body;
obtaining monitoring information of a target human body by processing according to the time domain characteristic information and adopting a deep learning method, wherein the monitoring information at least comprises the number, the position, the movement track, vital sign information and gesture recognition information of people;
and generating corresponding alarm information and control instructions according to the monitoring information, wherein the alarm information is sent to a set crowd, and the control instructions are sent to set household appliances.
In a second aspect, the present application provides a personnel intelligent monitoring system based on millimeter wave radar, comprising:
the millimeter wave radar module is used for transmitting millimeter wave signals and receiving radar original echo data;
the data preprocessing module is used for preprocessing according to the radar original echo data to obtain time domain characteristic information of a target human body;
the intelligent analysis module is used for obtaining monitoring information of a target human body according to the time domain characteristic information and by adopting a deep learning method, wherein the monitoring information at least comprises the number, the position, the movement track, vital sign information and gesture recognition information of people; the method comprises the steps of,
and the wireless communication module is used for generating corresponding alarm information and control instructions according to the monitoring information, the alarm information is sent to a set crowd, and the control instructions are sent to set household appliances.
In a third aspect, the present application also provides an electronic device, including a processor and a memory;
the memory is used for storing programs;
the processor executes the program to implement the method as described above.
In a fourth aspect, the present application also provides a computer-readable storage medium storing a program for execution by a processor to implement a method as described above.
In a fifth aspect, the present application also provides a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The computer instructions may be read from a computer-readable storage medium by a processor of a computer device, and executed by the processor, to cause the computer device to perform the foregoing method.
Compared with the prior art, the application has the beneficial effects that: the application can acquire various monitoring data of personnel in real time, analyze and process the monitoring data, send alarm information to a user through the wireless communication module, send control instructions to intelligent household appliances, and realize intelligent monitoring of the personnel and intelligent control of the household appliances, with no contact, simple and convenient.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following description will briefly explain the drawings needed in the embodiments, and it is obvious that the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flowchart of a personnel intelligent monitoring method based on millimeter wave radar in an embodiment of the application.
Fig. 2 is a block diagram of a personnel intelligent monitoring system based on millimeter wave radar in an embodiment of the application.
Fig. 3 is a schematic diagram of an electronic device according to an embodiment of the application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
It should be noted that the terms "first," "second," and the like in the description and the claims of the present application and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the application described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed or inherent to such process, method, article, or apparatus.
Example 1
Referring to fig. 1, a personnel intelligent monitoring method based on millimeter wave radar may include the steps of:
step 100: radar raw echo data is received.
In the step, millimeter wave signals can be transmitted through a millimeter wave radar and echoes can be received, so that radar original echo data can be obtained, wherein the millimeter wave radar module can adopt a multiple-input multiple-output array antenna, the transmitting antenna transmits the millimeter wave signals, the receiving antenna receives the reflected echo signals, the transmitting signals and the echo signals are mixed to obtain intermediate frequency signals, and the intermediate frequency signals are subjected to ADC sampling to obtain the original echo data.
More specifically, according to the practical application, the waveform, the transmitting power and the bandwidth of the transmitting signal of the millimeter wave radar can be configured, the measuring distance and the resolution of the radar are changed, and the personnel can be measured more accurately, so that the millimeter wave radar is suitable for different monitoring areas and covers the whole monitoring area as much as possible. Illustratively, the monitoring area may be a home or a hospital, including a living room, a ward. When a person forms an angle of 0 DEG with the beam of the millimeter wave radar, the obtained measurement result is the most accurate. The rotation of the millimeter wave radar can be controlled according to the position and the motion track of the personnel, and the beam direction of the millimeter wave radar is changed.
Step 200: preprocessing according to the radar original echo data to obtain time domain feature information of a target human body.
In this step, the time domain feature information of the target human body includes distance, velocity, azimuth, micro doppler features, and the like.
In some embodiments, this step may be further divided into the following sub-steps:
step 201, static clutter filtering is performed on an original echo data matrix R (m, n), specifically, a phasor average value cancellation method is adopted, data are accumulated and summed in a slow time dimension frame by frame, and then an average echo signal is obtained by averaging, and then the average echo signal is subtracted frame by frame to inhibit the static clutter;
step 202, performing short-time Fourier transform on echo data R (m, n) after removing impurities along a slow time dimension to obtain a micro Doppler characteristic matrix W (m, n);
step 203, performing a distance fast fourier transform on the echo data R (m, n) after removing the impurities along a fast time dimension to obtain a distance-time feature matrix D (m, n);
step 204, performing doppler fast fourier transform on the distance-time feature matrix D (m, n) along the slow time dimension to obtain a distance-doppler feature matrix V (m, n);
step 205, performing an angle fast fourier transform on the range-doppler feature matrix V (m, n) to obtain an azimuth-time feature matrix a (m, n).
In some embodiments, for step 202, the short-time Fourier transform may also take the following steps:
step 2021: performing short-time Fourier transform on the m-th line of echo data R (m, n) after removing the impurities along the slow time dimension:
where k is the Doppler frequency index, p is the window function movement step index, U is the window function movement step length, H is the length of the window function, and w (·) is the Kernel window function.
Step 2022: and accumulating short-time Fourier transform results of each row to obtain a micro Doppler characteristic matrix W (m, n):
step 300: and processing according to the time domain characteristic information by adopting a deep learning method to obtain monitoring information of the target human body, wherein the monitoring information at least comprises the number, the position, the movement track, vital sign information and gesture recognition information of personnel.
In some embodiments, this step may be further divided into the following sub-steps:
step 301, obtaining a distance image, a distance-azimuth angle image and a micro Doppler image according to the characteristic information;
step 302, inputting the distance-azimuth angle image into a deep learning model for processing, and determining the number, the positions and the motion trail of the target personnel according to the image detection result;
and 303, carrying out feature extraction and feature fusion on the distance image and the micro Doppler image to obtain a fused image, and inputting the fused image into a deep learning model for processing to obtain gesture recognition information.
Step 304: according to the distance-Doppler characteristic matrix V (m, n), determining the distance between the human body and the radar, and then sequentially carrying out phase extraction, ICEEMDAN decomposition, band-pass filtering and spectrum estimation to obtain vital sign information.
In some embodiments, for step 302, the number, the position and the motion trail of the target person are obtained, and the following steps may be further adopted:
step 3021: inputting the distance-azimuth image into a deep learning model, and dividing the image into 224×224 grids;
step 3022: determining that a target in a grid is detected when a target person falls on the grid;
step 3023: setting a detection threshold, regarding noise or interference when the target is smaller than the threshold, regarding target personnel when the target is larger than the threshold, counting the number of the target personnel, calculating the central point coordinates of the target personnel grids, and converting the central point coordinates into two-dimensional position coordinates;
step 324: and (3) carrying out association and matching on the two-dimensional position coordinates by utilizing a joint probability data association algorithm, and continuously updating the positions of the personnel to obtain the motion trail of the corresponding personnel.
In some embodiments, for step 303, the gesture recognition information is obtained, and the following steps may be further adopted:
step 3031: constructing a feature extraction layer to extract features, wherein the feature extraction layer comprises nine layers, namely a first convolution layer 3×3, a second convolution layer 3×3, a first maximum pooling layer 2×2, a third convolution layer 3×3, a fourth convolution layer 3×3, a second maximum pooling layer 2×2, a fifth convolution layer 3×3, a sixth convolution layer 3×3 and a third maximum pooling layer 2×2 in sequence;
step 3032: carrying out multi-domain feature fusion on the extracted features to obtain a fused feature image;
step 3033: and inputting the fusion image into a pre-trained deep learning model to obtain a gesture recognition result. The deep learning model is a mobiletv 3-Large model, and sequentially comprises a convolution layer 3×3, three bneck3×3 modules, eight bneck5×5 modules, a convolution layer 1×1, an average pooling layer 7×7, two fully connected layers and a softmax layer.
In some embodiments, for step 304, vital sign information is acquired, the following steps may also be employed:
step 3041: according to the distance-Doppler characteristic matrix V (m, n), determining the distance between the human body and the radar, and selecting a distance unit range corresponding to the human body;
step 3042: phase extraction is carried out on the distance-Doppler characteristic matrix by utilizing arctangent transformation, and phase information is obtained:
step 3043: and (3) carrying out phase unwrapping on the obtained phase information Vphase (m, n), wherein the phase unwrapping enables a phase value to be between [ -pi, pi ], if the phase exceeds the interval, subtracting or adding 2 pi, and obtaining the actual chest vibration displacement change through a phase unwrapping result.
Step 3044: decomposing the phase information Vphase (m, n) by using an ICEEMDAN decomposition method to obtain a plurality of IMF signals;
step 3045: carrying out signal separation on the IMF signals by utilizing a band-pass filter, reconstructing to obtain breathing signals and heartbeat signals, wherein the IMF signals with the frequency range of 0.8-2 Hz are heartbeat signals, and the IMF signals with the frequency range of 0.2-0.4 Hz are breathing signals;
step 3046: and performing spectrum estimation on the reconstructed respiratory signal and heartbeat signal to obtain the respiratory rate and the heartbeat rate of the personnel.
Step 400: and generating corresponding alarm information and control instructions according to the monitoring information, wherein the alarm information is sent to a set crowd, and the control instructions are sent to set household appliances.
In the step, the acquired monitoring data is analyzed and processed through the wireless communication module to generate corresponding alarm information and control instructions, and the wireless communication module can also be used for sending the alarm information to related users and sending the control instructions to intelligent household appliances so as to realize intelligent monitoring of personnel and intelligent control of the household appliances.
More specifically, whether the person goes out or not is judged according to the number, the positions and the movement track of the person, the person going out reminding can be sent, and real-time monitoring of special people, such as the old and children in home and the patients in hospitals can be realized; setting a vital sign information threshold, judging whether the vital sign information exceeds a corresponding threshold or not, and sending a vital sign alarm prompt to discover the illness state and prevent the illness state in an early stage; the gesture recognition information category comprises falling, left arm waving, right arm waving and double arm waving, according to the gesture recognition information, the personnel gesture is falling, then the falling alarm reminding is sent, the personnel gesture is left arm waving or right arm waving or double arm waving, and the like, and then the switch control instruction is sent to the intelligent household electrical appliance.
Example 2
Referring to fig. 2, a personnel intelligent monitoring system based on millimeter wave radar may include:
the millimeter wave radar module is used for transmitting millimeter wave signals and receiving radar original echo data;
the data preprocessing module is used for preprocessing according to the radar original echo data to obtain time domain characteristic information of a target human body;
the intelligent analysis module is used for obtaining monitoring information of a target human body according to the time domain characteristic information and by adopting a deep learning method, wherein the monitoring information at least comprises the number, the position, the movement track, vital sign information and gesture recognition information of people; the method comprises the steps of,
and the wireless communication module is used for generating corresponding alarm information and control instructions according to the monitoring information, the alarm information is sent to a set crowd, and the control instructions are sent to set household appliances.
In some embodiments, the system may further include a storage module, where the storage module is configured to store each item of monitoring data;
in some embodiments, the system may further include a display module, where the display module is configured to display power information, system operation information, and monitoring data in real time.
The wireless communication module supports communication protocols such as Bluetooth and WiFi,2G,3G,4G,5G, and can be directly connected with a user and intelligent household appliances respectively to send alarm information and control instructions.
Illustratively, the user includes a mobile device and a back-end platform, which may be a smart watch, a smart phone, a medical care center. The intelligent household appliance can be an intelligent lamp, an intelligent television, an intelligent air conditioner and the like.
The system can monitor the ingress and egress of personnel, can monitor a plurality of personnel in real time, and analyze, process and decompose the original echo data into monitoring data respectively belonging to the plurality of personnel and store the monitoring data.
Because the system is a system corresponding to the personnel intelligent monitoring method based on the millimeter wave radar in the embodiment of the application, and the principle of solving the problem of the system is similar to that of the method, the implementation of the system can refer to the implementation process of the embodiment of the method, and the repetition is omitted.
Example 3
Referring to fig. 3, based on the same inventive concept, an embodiment of the present application further provides an electronic device, which includes a processor and a memory, where at least one instruction, at least one section of program, a code set, or an instruction set is stored in the memory, and the at least one instruction, the at least one section of program, the code set, or the instruction set is loaded and executed by the processor, so as to implement the personnel intelligent monitoring method based on millimeter wave radar as described above.
It is understood that the Memory may include random access Memory (Random Access Memory, RAM) or Read-Only Memory (RAM). Optionally, the memory includes a non-transitory computer readable medium (non-transitory computer-readable storage medium). The memory may be used to store instructions, programs, code sets, or instruction sets. The memory may include a stored program area and a stored data area, wherein the stored program area may store instructions for implementing an operating system, instructions for at least one function, instructions for implementing the various method embodiments described above, and the like; the storage data area may store data created according to the use of the server, etc.
The processor may include one or more processing cores. The processor uses various interfaces and lines to connect various portions of the overall server, perform various functions of the server, and process data by executing or executing instructions, programs, code sets, or instruction sets stored in memory, and invoking data stored in memory. Alternatively, the processor may be implemented in hardware in at least one of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), programmable logic array (Programmable Logic Array, PLA). The processor may integrate one or a combination of several of a central processing unit (Central Processing Unit, CPU) and a modem etc. Wherein, the CPU mainly processes an operating system, application programs and the like; the modem is used to handle wireless communications. It will be appreciated that the modem may not be integrated into the processor and may be implemented by a single chip.
Because the electronic device is the electronic device corresponding to the personnel intelligent monitoring method based on the millimeter wave radar in the embodiment of the application, and the principle of solving the problem of the electronic device is similar to that of the method, the implementation of the electronic device can refer to the implementation process of the embodiment of the method, and the repetition is omitted.
Example 4
Based on the same inventive concept, the embodiment of the present application further provides a computer readable storage medium, in which at least one instruction, at least one section of program, a code set, or an instruction set is stored, and the at least one instruction, the at least one section of program, the code set, or the instruction set is loaded and executed by a processor to implement the personnel intelligent monitoring method based on millimeter wave radar as described above.
Those of ordinary skill in the art will appreciate that all or part of the steps of the various methods of the above embodiments may be implemented by a program to instruct related hardware, the program may be stored in a computer readable storage medium including Read-Only Memory (ROM), random access Memory (Random Access Memory, RAM), programmable Read-Only Memory (Programmable Read-Only Memory, PROM), erasable programmable Read-Only Memory (Erasable Programmable Read Only Memory, EPROM), one-time programmable Read-Only Memory (One-time Programmable Read-OnlyMemory, OTPROM), electrically erasable programmable Read-Only Memory (EEPROM), compact disc Read-Only Memory (CD-ROM) or other optical disc Memory, magnetic disk Memory, tape Memory, or any other medium capable of being used for carrying or storing data that is readable by a computer.
Because the storage medium is a storage medium corresponding to the personnel intelligent monitoring method based on the millimeter wave radar in the embodiment of the application, and the principle of solving the problem by the storage medium is similar to that of the method, the implementation of the storage medium can refer to the implementation process of the embodiment of the method, and the repetition is omitted.
Example 5
In some possible implementations, aspects of the method of the embodiments of the present application may also be implemented in the form of a program product comprising program code for causing a computer device to carry out the steps of the method for person intelligent monitoring based on millimeter wave radar according to the various exemplary embodiments of the application as described in this specification, when the program product is run on a computer device. Wherein executable computer program code or "code" for performing the various embodiments may be written in a high-level programming language such as C, C ++, c#, smalltalk, java, javaScript, visual Basic, structured query language (e.g., act-SQL), perl, or in a variety of other programming languages.
It is to be understood that portions of the present application may be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, the various steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, may be implemented using any one or combination of the following techniques, as is well known in the art: discrete logic circuits having logic gates for implementing logic functions on data signals, application specific integrated circuits having suitable combinational logic gates, programmable Gate Arrays (PGAs), field Programmable Gate Arrays (FPGAs), and the like.
In the description of the present specification, a description referring to terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present application. In this specification, schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, the different embodiments or examples described in this specification and the features of the different embodiments or examples may be combined and combined by those skilled in the art without contradiction.
The above embodiments are only for illustrating the technical concept and features of the present application, and are intended to enable those skilled in the art to understand the content of the present application and implement the same, and are not intended to limit the scope of the present application. All equivalent changes or modifications made in accordance with the essence of the present application are intended to be included within the scope of the present application.
Claims (10)
1. The personnel intelligent monitoring method based on the millimeter wave radar is characterized by comprising the following steps of:
transmitting millimeter wave signals and receiving radar original echo data;
preprocessing according to the radar original echo data to obtain time domain feature information of a target human body;
obtaining monitoring information of a target human body by processing according to the time domain characteristic information and adopting a deep learning method, wherein the monitoring information at least comprises the number, the position, the movement track, vital sign information and gesture recognition information of people;
and generating corresponding alarm information and control instructions according to the monitoring information, wherein the alarm information is sent to a set crowd, and the control instructions are sent to set household appliances.
2. The personnel intelligent monitoring method based on millimeter wave radar according to claim 1, wherein the preprocessing obtains time domain feature information of a target human body, and specifically comprises the steps of:
static clutter filtering is carried out on the matrix of the original echo data, wherein the average echo signals are obtained by accumulating and summing the original echo data in a slow time dimension frame by frame and then averaging the accumulated and summed slow time dimension frame by frame, and then the average echo signals are subtracted frame by frame to inhibit the static clutter;
performing short-time Fourier transform on the echo data matrix R (m, n) after removing the impurities along a slow time dimension to obtain a micro Doppler characteristic matrix W (m, n);
performing distance fast Fourier transform on the echo data matrix R (m, n) after removing the impurities along a fast time dimension to obtain a distance-time characteristic matrix D (m, n);
doppler fast Fourier transform is carried out on the distance-time characteristic matrix D (m, n) along the slow time dimension to obtain a distance-Doppler characteristic matrix V (m, n);
and performing angle fast Fourier transform on the range-Doppler characteristic matrix V (m, n) to obtain an azimuth-time characteristic matrix A (m, n).
3. The personnel intelligent monitoring method based on millimeter wave radar according to claim 2, wherein,
the short-time fourier transform, comprising:
performing short-time Fourier transform on the m-th row of the matrix R (m, n) of echo data after removing the impurities along the slow time dimension:
wherein k is Doppler frequency index, p is window function moving step index, U is window function moving step length, H is window function length, and w (·) is Kazier window function;
and accumulating short-time Fourier transform results of each row to obtain a micro Doppler characteristic matrix W (m, n):
4. the personnel intelligent monitoring method based on millimeter wave radar according to claim 2, wherein,
generating a distance image, a distance-azimuth image and a micro Doppler image according to the time domain feature information;
inputting the distance-azimuth angle image into a deep learning model for processing, and determining the number, the positions and the motion trail of target personnel according to an image detection result;
performing feature extraction and feature fusion on the distance image and the micro Doppler image to obtain a fusion image, and inputting the fusion image into a deep learning model for processing to obtain gesture recognition information;
and determining the distance between the human body and the radar according to the distance-Doppler characteristic matrix V (m, n), and then sequentially carrying out phase extraction, ICEEMDAN decomposition, band-pass filtering and spectrum estimation to obtain vital sign information.
5. The method for intelligently monitoring personnel based on millimeter wave radar according to claim 2, wherein the steps of obtaining the number, the position and the movement track of the target personnel comprise the following steps:
inputting the range-azimuth image into a deep learning model while dividing the range-azimuth image into 224×224 grids;
determining that a target in a grid is detected when a target person falls on the grid;
setting a detection threshold, wherein when the target is smaller than the threshold, the target is regarded as noise or interference; when the target is greater than the threshold, the target person is considered; counting the number of target personnel, calculating the coordinates of the central points of the grids of the target personnel, and converting the coordinates of the central points into two-dimensional position coordinates;
and (3) carrying out association and matching on the two-dimensional position coordinates by utilizing a joint probability data association algorithm, and continuously updating the position of the target person to acquire the motion trail of the corresponding target person.
6. The method for intelligently monitoring personnel based on millimeter wave radar according to claim 4, wherein the step of acquiring the gesture recognition information comprises the steps of:
constructing a feature extraction layer to extract features, wherein the feature extraction layer comprises nine layers, namely a first convolution layer 3×3, a second convolution layer 3×3, a first maximum pooling layer 2×2, a third convolution layer 3×3, a fourth convolution layer 3×3, a second maximum pooling layer 2×2, a fifth convolution layer 3×3, a sixth convolution layer 3×3 and a third maximum pooling layer 2×2 in sequence;
carrying out multi-domain feature fusion on the extracted features to obtain a fusion feature image;
inputting the fusion image into a pre-trained deep learning model to obtain a gesture recognition result, wherein the deep learning model is a mobiletv 3-Large model, and sequentially comprises a convolution layer 3×3, three bnck 3×3 modules, eight bnck 5×5 modules, a convolution layer 1×1, an average pooling layer 7×7, two fully connected layers and a softmax layer.
7. The method for intelligently monitoring personnel based on millimeter wave radar according to claim 4, wherein the step of acquiring the vital sign information comprises the steps of:
determining the distance between the human body and the radar according to the distance-Doppler characteristic matrix V (m, n), and selecting a distance unit range corresponding to the human body;
phase extraction is carried out on the distance-Doppler characteristic matrix by utilizing inverse tangent transformation to obtain phase information:
for the obtained phase information V phase(m，n) Performing phase unwrapping to obtain a phase value of [ -pi, pi [ -pi ]]The actual chest vibration displacement change is obtained through the phase unwrapping result;
phase information V by ICEEMDAN decomposition method phase(m，n) Decomposing to obtain a plurality of IMF signals;
carrying out signal separation on the IMF signals by utilizing a band-pass filter, and reconstructing to obtain breathing signals and heartbeat signals, wherein the IMF signals with the frequency range of 0.8-2 Hz are heartbeat signals, and the IMF signals with the frequency range of 0.2-0.4 Hz are breathing signals;
and carrying out frequency spectrum estimation on the reconstructed respiratory signal and heartbeat signal to obtain the respiratory rate and the heartbeat rate of the personnel.
8. Personnel intelligent monitoring system based on millimeter wave radar, characterized by comprising:
the millimeter wave radar module is used for transmitting millimeter wave signals and receiving radar original echo data;
the data preprocessing module is used for preprocessing according to the radar original echo data to obtain time domain characteristic information of a target human body;
the intelligent analysis module is used for obtaining monitoring information of a target human body according to the time domain characteristic information and by adopting a deep learning method, wherein the monitoring information at least comprises the number, the position, the movement track, vital sign information and gesture recognition information of people; the method comprises the steps of,
and the wireless communication module is used for generating corresponding alarm information and control instructions according to the monitoring information, the alarm information is sent to a set crowd, and the control instructions are sent to set household appliances.
9. An electronic device comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a set of codes, or a set of instructions, the at least one instruction, the at least one program, the set of codes, or the set of instructions being loaded and executed by the processor to implement the millimeter wave radar-based personnel intelligent monitoring method of any one of claims 1 to 7.
10. A computer-readable storage medium having stored therein at least one instruction, at least one program, code set, or instruction set, the at least one instruction, the at least one program, the code set, or instruction set being loaded and executed by a processor to implement the millimeter wave radar-based person intelligent monitoring method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310673735.XA CN116942142A (en) | 2023-06-08 | 2023-06-08 | Personnel intelligent monitoring method based on millimeter wave radar and related equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310673735.XA CN116942142A (en) | 2023-06-08 | 2023-06-08 | Personnel intelligent monitoring method based on millimeter wave radar and related equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116942142A true CN116942142A (en) | 2023-10-27 |
Family
ID=88460953
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310673735.XA Pending CN116942142A (en) | 2023-06-08 | 2023-06-08 | Personnel intelligent monitoring method based on millimeter wave radar and related equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116942142A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117148309A (en) * | 2023-11-01 | 2023-12-01 | 德心智能科技(常州)有限公司 | Millimeter wave radar human body sensing method and system applied to community grid inspection |
| CN117908018A (en) * | 2024-03-19 | 2024-04-19 | 清澜技术(深圳)有限公司 | Method, system, equipment and storage medium for warning waving hand |
-
2023
- 2023-06-08 CN CN202310673735.XA patent/CN116942142A/en active Pending
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117148309A (en) * | 2023-11-01 | 2023-12-01 | 德心智能科技(常州)有限公司 | Millimeter wave radar human body sensing method and system applied to community grid inspection |
| CN117148309B (en) * | 2023-11-01 | 2024-01-30 | 德心智能科技(常州)有限公司 | Millimeter wave radar human body sensing method and system applied to community grid inspection |
| CN117908018A (en) * | 2024-03-19 | 2024-04-19 | 清澜技术(深圳)有限公司 | Method, system, equipment and storage medium for warning waving hand |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN116942142A (en) | Personnel intelligent monitoring method based on millimeter wave radar and related equipment | |
| Sachs et al. | M‐sequence‐based ultra‐wideband sensor network for vitality monitoring of elders at home | |
| CN111695420A (en) | Gesture recognition method and related device | |
| US20230341518A1 (en) | Millimeterwave radar system for determining an activity record | |
| Yang et al. | Unsupervised detection of apnea using commodity RFID tags with a recurrent variational autoencoder | |
| CN115244586A (en) | Microwave identification method and system | |
| CN115308734A (en) | Respiratory data calculation method and related equipment | |
| CN113313040A (en) | Human body posture identification method based on FMCW radar signal | |
| CN110520935A (en) | Learn sleep stage from radio signal | |
| Goldfine et al. | Respiratory rate monitoring in clinical environments with a contactless ultra-wideband impulse radar-based sensor system | |
| Chen et al. | Respiration and activity detection based on passive radio sensing in home environments | |
| CN114305355B (en) | Breathing heartbeat detection method, system and device based on millimeter wave radar | |
| CN114818788A (en) | Tracking target state identification method and device based on millimeter wave perception | |
| CN115343704A (en) | Gesture recognition method of FMCW millimeter wave radar based on multi-task learning | |
| CN115721294B (en) | Millimeter wave perception-based respiration monitoring method and device, electronic equipment and medium | |
| Mauro et al. | Few-shot user-adaptable radar-based breath signal sensing | |
| CN116602640A (en) | Millimeter wave radar heart rate detection method based on adaptive clutter filtering | |
| Zhou et al. | Md-pose: Human pose estimation for single-channel uwb radar | |
| Choi et al. | Remote respiration monitoring of moving person using radio signals | |
| Raeis et al. | InARMS: Individual activity recognition of multiple subjects with FMCW radar | |
| Gao et al. | A new direction for biosensing: RF sensors for monitoring cardio-pulmonary function | |
| CN114190913A (en) | Millimeter wave radar-based driver driving state monitoring system and method | |
| Amin | Micro-Doppler classification of activities of daily living incorporating human ethogram | |
| GB2600975A (en) | Optimization of dataset | |
| Li et al. | ECG signal detection method based on millimeter wave radar |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |