CN117892202A - Nursing action monitoring method based on millimeter wave signals and electronic equipment - Google Patents
Nursing action monitoring method based on millimeter wave signals and electronic equipment Download PDFInfo
- Publication number
- CN117892202A CN117892202A CN202410293315.3A CN202410293315A CN117892202A CN 117892202 A CN117892202 A CN 117892202A CN 202410293315 A CN202410293315 A CN 202410293315A CN 117892202 A CN117892202 A CN 117892202A
- Authority
- CN
- China
- Prior art keywords
- millimeter wave
- nursing
- point cloud
- wave signals
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000000474 nursing effect Effects 0.000 title claims abstract description 112
- 238000000034 method Methods 0.000 title claims abstract description 68
- 238000012544 monitoring process Methods 0.000 title claims abstract description 62
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 59
- 238000012545 processing Methods 0.000 claims abstract description 25
- 238000013136 deep learning model Methods 0.000 claims abstract description 14
- 239000011159 matrix material Substances 0.000 claims description 60
- 230000009471 action Effects 0.000 claims description 21
- 238000004590 computer program Methods 0.000 claims description 13
- 238000006243 chemical reaction Methods 0.000 claims description 11
- 238000001514 detection method Methods 0.000 claims description 11
- 230000008569 process Effects 0.000 claims description 6
- 238000013528 artificial neural network Methods 0.000 claims description 4
- 238000004364 calculation method Methods 0.000 claims description 4
- 238000013527 convolutional neural network Methods 0.000 claims description 4
- 125000004122 cyclic group Chemical group 0.000 claims description 4
- 230000006870 function Effects 0.000 description 5
- 230000004048 modification Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 238000007792 addition Methods 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000005021 gait Effects 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 239000000779 smoke Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Chemical compound O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
- G01S13/88—Radar or analogous systems specially adapted for specific applications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/243—Classification techniques relating to the number of classes
- G06F18/2431—Multiple classes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/254—Fusion techniques of classification results, e.g. of results related to same input data
- G06F18/256—Fusion techniques of classification results, e.g. of results related to same input data of results relating to different input data, e.g. multimodal recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Molecular Biology (AREA)
- Radar, Positioning & Navigation (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Remote Sensing (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Evolutionary Biology (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Electromagnetism (AREA)
- Computer Networks & Wireless Communication (AREA)
- Radar Systems Or Details Thereof (AREA)
Abstract
The application provides a nursing action monitoring method and electronic equipment based on millimeter wave signals, wherein the method comprises the following steps: periodically transmitting millimeter wave signals with preset frequency to a space where a user is located; respectively receiving millimeter wave signals and forming a plurality of different target millimeter wave signals after the millimeter wave signals are reflected by a user; processing each target millimeter wave signal based on an FFT algorithm, a DOA estimation algorithm and a CFAR algorithm to obtain point cloud information corresponding to the whole of each target millimeter wave signal; and sending each point cloud data frame in the point cloud information to a computing terminal, so that the computing terminal receives a plurality of point cloud data frames in a preset time window, judges whether the time window is a nursing window, if so, inputs each point cloud data frame in the nursing window to a pre-trained deep learning model, and outputs a corresponding nursing action result. The nursing monitoring method and the nursing monitoring system can effectively reduce the flow complexity of the nursing monitoring method, effectively reduce the nursing monitoring cost, and further effectively improve the experience of patients.
Description
Technical Field
The application relates to the field of nursing monitoring, in particular to a nursing action monitoring method based on millimeter wave signals and electronic equipment.
Background
Millimeter wave radars have been widely used in the fields of autopilot, industry, unmanned aerial vehicles, and medical applications. The millimeter wave radar has higher sensing precision and stronger anti-interference capability. The FMCW modulation technique is used to obtain the distance, angle and speed information of the target at low cost. In addition, millimeter wave signals are not easily affected by external environment, can penetrate through smoke and water vapor, and are not easily affected by illumination conditions. Millimeter wave sensing technology has been widely used, such as millimeter wave-based gesture recognition, gait recognition, heartbeat and breath recognition, etc., and can provide more intelligent, convenient and high-quality product experience for us.
The existing nursing monitoring method is high in complexity, high in cost and poor in user experience.
Disclosure of Invention
In view of this, embodiments of the present application provide a method and apparatus for monitoring care actions based on millimeter wave signals, which obviate or ameliorate one or more of the disadvantages of the prior art.
A first aspect of the present application provides a method for monitoring a care action based on millimeter wave signals, the method comprising:
periodically transmitting millimeter wave signals with preset frequency to a space where a user is located;
respectively receiving the millimeter wave signals and forming a plurality of different target millimeter wave signals after the millimeter wave signals are reflected by a user; the user comprises a nursing staff and a nursed person;
processing each target millimeter wave signal based on an FFT algorithm, a DOA estimation algorithm and a CFAR algorithm to obtain point cloud information corresponding to the whole target millimeter wave signals;
and sending each point cloud data frame in the point cloud information to a computing terminal, so that the computing terminal receives a plurality of point cloud data frames in a preset time window, judges whether the time window is a nursing window, if so, inputs each point cloud data frame in the nursing window to a pre-trained deep learning model, and outputs a corresponding nursing action result.
In some embodiments of the present application, the processing, based on the FFT algorithm, the DOA estimation algorithm, and the CFAR algorithm, the target millimeter wave signals to obtain point cloud information corresponding to the whole target millimeter wave signals includes:
performing FFT calculation on the frame length dimension of each target millimeter wave signal to obtain a Doppler feature matrix and a distance feature matrix corresponding to each target millimeter wave signal;
obtaining a distance angle matrix corresponding to the whole of each target millimeter wave signal based on a DOA estimation algorithm and each target millimeter wave signal;
and processing the Doppler feature matrix and the corresponding distance angle matrix based on the CFAR target detection algorithm to obtain point cloud information corresponding to the whole millimeter wave signals of the targets.
In some embodiments of the present application, the obtaining, based on the DOA estimation algorithm and each target millimeter wave signal, a distance angle matrix corresponding to each target millimeter wave signal as a whole includes:
obtaining a conversion coefficient matrix corresponding to each receiving antenna based on the arrangement condition of the receiving antenna corresponding to each target millimeter wave signal;
and multiplying each conversion coefficient matrix by a corresponding distance characteristic matrix, and then adopting a DOA estimation algorithm to process so as to obtain the distance angle matrix.
In some embodiments of the present application, the processing, based on the CFAR target detection algorithm, the doppler feature matrix and the corresponding distance angle matrix to obtain point cloud information corresponding to the whole millimeter wave signal of each target includes:
calculating the position information of the weighted distance angle matrix based on a CFAR target detection algorithm;
and obtaining point cloud information corresponding to the whole millimeter wave signals of each target based on the Doppler feature matrixes and the position information.
A second aspect of the present application provides a method for monitoring a care action based on millimeter wave signals, the method comprising:
receiving each point cloud data frame in the point cloud information sent by the millimeter wave radar in a preset time window; the point cloud information periodically transmits millimeter wave signals with preset frequency to a space where a user is located by the millimeter wave radar, and the millimeter wave signals are respectively received and reflected by the user to form a plurality of different target millimeter wave signals; processing each target millimeter wave signal based on an FFT algorithm, a DOA estimation algorithm and a CFAR algorithm to obtain the target millimeter wave signal, wherein the user comprises a nursing staff and a nursed person;
judging whether the time window is a nursing window or not; if yes, inputting each point cloud data frame in the nursing window into a pre-trained deep learning model, and outputting to obtain a corresponding nursing action result.
In some embodiments of the present application, in the method for monitoring a nursing action based on millimeter wave signals provided in the second aspect of the present application, the determining whether the time window is a nursing window includes:
judging whether the number of the point cloud data frames received in each second in the time window is larger than a preset number threshold value, and whether the proportion of the number of the point cloud data frames in the preset space range in each second to the total number of the point cloud data frames received in each second is larger than a preset proportion threshold value, if so, determining that the time window is a nursing window.
In some embodiments of the present application, in the method for monitoring a nursing action based on millimeter wave signals provided in the second aspect of the present application, the inputting each point cloud data frame in the nursing window to a pre-trained deep learning model, outputting and obtaining a corresponding nursing action result includes:
inputting each point cloud data frame in the care window into the convolutional neural network, and extracting the corresponding multidimensional feature of each point cloud data frame;
and sequentially inputting the multi-dimensional features into a cyclic neural network according to a time sequence, extracting the associated information of the multi-dimensional features in the time dimension, and outputting a nursing action feature matrix.
And obtaining the nursing action result based on the nursing action feature matrix and a preset nursing action label.
A third aspect of the present application provides an electronic device, if the electronic device is a millimeter wave radar, configured to implement the method for monitoring a nursing action based on millimeter wave signals according to the foregoing first aspect;
if the electronic device is a computing terminal, the computing terminal includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor, where the processor implements the method for monitoring nursing actions based on millimeter wave signals according to the second aspect when executing the computer program.
A fourth aspect of the present application provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the millimeter wave signal-based care action monitoring method of the foregoing second aspect.
A fifth aspect of the present application provides a computer program product comprising a computer program which, when executed by a processor, implements the millimeter wave signal based care action monitoring method of the second aspect described above.
In summary, the present application provides a nursing action monitoring method based on millimeter wave signals, the method including: periodically transmitting millimeter wave signals with preset frequency to a space where a user is located; respectively receiving millimeter wave signals and forming a plurality of different target millimeter wave signals after the millimeter wave signals are reflected by a user; processing each target millimeter wave signal based on an FFT algorithm, a DOA estimation algorithm and a CFAR algorithm to obtain point cloud information corresponding to the whole of each target millimeter wave signal; and sending each point cloud data frame to a computing terminal, so that the computing terminal receives a plurality of point cloud data frames in a preset time window, judges whether the time window is a nursing window, if so, inputs each point cloud data frame in the nursing window to a pre-trained deep learning model, and outputs a corresponding nursing action result. The nursing monitoring method and the nursing monitoring system can effectively reduce the flow complexity of the nursing monitoring method, effectively reduce the nursing monitoring cost, and further effectively improve the experience of patients.
Additional advantages, objects, and features of the application will be set forth in part in the description which follows and in part will become apparent to those having ordinary skill in the art upon examination of the following or may be learned from practice of the application. The objectives and other advantages of the application may be realized and attained by the structure particularly pointed out in the written description and drawings.
It will be appreciated by those skilled in the art that the objects and advantages that can be achieved with the present application are not limited to the above-detailed description, and that the above and other objects that can be achieved with the present application will be more clearly understood from the following detailed description.
Drawings
The accompanying drawings are included to provide a further understanding of the application, and are incorporated in and constitute a part of this application. The components in the figures are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the application. Corresponding parts in the drawings may be exaggerated, i.e. made larger relative to other parts in an exemplary device actually manufactured according to the present application, for convenience in showing and describing some parts of the present application. In the drawings:
fig. 1 is a flowchart of a first care action monitoring method based on millimeter wave signals in an embodiment of the present application.
Fig. 2 is a flowchart of a second nursing action monitoring method based on millimeter wave signals according to another embodiment of the present application.
Fig. 3 is a schematic overall architecture of a second nursing action monitoring method based on millimeter wave signals according to another embodiment of the present application.
Detailed Description
For the purposes of making the objects, technical solutions and advantages of the present application more apparent, the present application will be described in further detail with reference to the embodiments and the accompanying drawings. The exemplary embodiments of the present application and their descriptions are used herein to explain the present application, but are not intended to be limiting of the present application.
It should be noted here that, in order to avoid obscuring the present application due to unnecessary details, only structures and/or processing steps closely related to the solution according to the present application are shown in the drawings, while other details not greatly related to the present application are omitted.
It should be emphasized that the term "comprises/comprising" when used herein is taken to specify the presence of stated features, elements, steps or components, but does not preclude the presence or addition of one or more other features, elements, steps or components.
It is also noted herein that the term "coupled" may refer to not only a direct connection, but also an indirect connection in which an intermediate is present, unless otherwise specified.
Hereinafter, embodiments of the present application will be described with reference to the drawings. In the drawings, the same reference numerals represent the same or similar components, or the same or similar steps.
The following examples are provided to illustrate the invention in more detail.
The embodiment of the application provides a first nursing action monitoring method based on millimeter wave signals, which can be executed by a millimeter wave radar, referring to fig. 1, wherein the nursing action monitoring method based on millimeter wave signals specifically comprises the following contents:
step 110: periodically transmitting millimeter wave signals with preset frequency to the space where the user is located.
Step 120: respectively receiving the millimeter wave signals and forming a plurality of different target millimeter wave signals after the millimeter wave signals are reflected by a user; the user includes caregivers and caregivers.
Step 130: and processing each target millimeter wave signal based on an FFT algorithm, a DOA estimation algorithm and a CFAR algorithm to obtain point cloud information corresponding to the whole target millimeter wave signals.
Step 140: and sending each point cloud data frame in the point cloud information to a computing terminal, so that the computing terminal receives a plurality of point cloud data frames in a preset time window, judges whether the time window is a nursing window, if so, inputs each point cloud data frame in the nursing window to a pre-trained deep learning model, and outputs a corresponding nursing action result.
Specifically, a millimeter wave radar (e.g., texas instruments IWR 6843) periodically transmits a millimeter wave signal of a predetermined frequency to a space in which a user is located. Then, a plurality of receiving antennas are adopted to receive millimeter wave signals, and the millimeter wave signals are reflected by a user to form a plurality of different target millimeter wave signals; and then processing each target millimeter wave signal based on an FFT (fast Fourier transform) algorithm, a DOA (Direction of Arrival, direction arrival) estimation algorithm and a CFAR (Constant False Alarm Rate ) target detection algorithm to obtain point cloud information corresponding to the whole of each target millimeter wave signal. And finally, sending each point cloud data frame in the point cloud information to a computing terminal, so that the computing terminal receives a plurality of point cloud data frames in a preset time window, judges whether the time window is a nursing window, if so, inputs each point cloud data frame in the nursing window to a pre-trained deep learning model, and outputs a corresponding nursing action result, thereby effectively reducing the flow complexity of a nursing monitoring method, effectively reducing the nursing monitoring cost and further effectively improving the experience of a patient.
The user comprises nursing staff and a nursed person. Millimeter wave signals represent a two-dimensional message comprising segments of signals having a fixed number of sampling points with a frequency that varies linearly with time. The target millimeter wave information is a two-dimensional signal and comprises frame length and discrete sampling points. The point cloud data frame includes X, Y, Z three-dimensional coordinates, speed, and signal strength information for a plurality of points.
To further reduce the flow complexity of the care monitoring method, step 130 includes:
step 131: and respectively carrying out FFT calculation on the frame length dimension of each target millimeter wave signal twice to obtain Doppler characteristic matrixes and distance characteristic matrixes corresponding to the target millimeter wave signals.
For step 131, the doppler characteristic (i.e., velocity characteristic) is shown in equation 1, and the distance characteristic is shown in equation 2:
(1)
wherein,for wavelength, < >>The phase difference between the millimeter wave signal and the target millimeter wave signal is represented, and t represents time.
(2)
Where c is the speed of light, f is the frequency difference between the millimeter wave signal and the target millimeter wave signal, d is the distance between the user and the receiving antenna, and s is the frequency increase slope of the millimeter wave signal.
Step 132: and obtaining a distance angle matrix corresponding to the whole target millimeter wave signals based on the DOA estimation algorithm and the target millimeter wave signals.
Step 133: and processing the Doppler feature matrix and the corresponding distance angle matrix based on the CFAR target detection algorithm to obtain point cloud information corresponding to the whole millimeter wave signals of the targets.
Specifically, firstly, performing FFT calculation on frame length dimensions of each target millimeter wave signal to obtain Doppler feature matrixes and distance feature matrixes corresponding to each target millimeter wave signal; then, based on the DOA estimation algorithm and each target millimeter wave signal, obtaining a distance angle matrix corresponding to each target millimeter wave signal; and finally, processing each Doppler characteristic matrix and the corresponding distance angle matrix based on the CFAR target detection algorithm to obtain point cloud information corresponding to the whole millimeter wave signals of each target, so that the flow complexity of the nursing monitoring method can be further reduced.
To effectively acquire the distance angle matrix, step 132 includes:
obtaining a conversion coefficient matrix corresponding to each receiving antenna based on the arrangement condition of the receiving antenna corresponding to each target millimeter wave signal, wherein the conversion coefficient matrix is shown in a formula 3;
（3）
j represents an imaginary symbol and,indicating the phase difference between the mth receive antenna and the first receive antenna.
And multiplying each conversion coefficient matrix with the corresponding distance characteristic matrix, and then adopting a DOA estimation algorithm to process the multiplied conversion coefficient matrix to obtain the distance angle matrix, wherein the distance angle matrix is shown in a formula 4.
（4）
Wherein,is a distance characteristic matrix corresponding to the target millimeter wave signal received by the mth receiving antenna,representing the conversion coefficient matrix corresponding to the mth receiving antenna.
Specifically, the millimeter wave radar firstly obtains a conversion coefficient matrix corresponding to each receiving antenna based on the arrangement condition of the receiving antennas corresponding to each target millimeter wave signal; and finally multiplying each conversion coefficient matrix with the corresponding distance characteristic matrix, and processing by adopting a DOA estimation algorithm to obtain a distance angle matrix.
To effectively acquire a point cloud data frame, step 133 includes:
calculating the position information of the weighted distance angle matrix based on a CFAR target detection algorithm;
and obtaining point cloud information corresponding to the whole millimeter wave signals of each target based on the Doppler feature matrixes and the position information.
Specifically, the millimeter wave radar calculates the position information of each Doppler feature matrix and the corresponding distance angle matrix based on a CFAR target detection algorithm; and then, respectively obtaining the point cloud information corresponding to the whole millimeter wave signals of each target based on the position information of each Doppler characteristic matrix and the position information of the corresponding distance angle matrix.
The embodiment of the application also provides a second nursing action monitoring method based on millimeter wave signals, which can be executed by the computing terminal, referring to fig. 2, wherein the nursing action monitoring method based on millimeter wave signals specifically comprises the following contents:
step 210: receiving each point cloud data frame in the point cloud information sent by the millimeter wave radar in a preset time window; the point cloud information periodically transmits millimeter wave signals with preset frequency to a space where a user is located by the millimeter wave radar, and the millimeter wave signals are respectively received and reflected by the user to form a plurality of different target millimeter wave signals; and processing each target millimeter wave signal based on an FFT algorithm, a DOA estimation algorithm and a CFAR algorithm, wherein the user comprises a nursing staff and a nursed person.
Step 220: judging whether the time window is a nursing window or not; if yes, inputting each point cloud data frame in the nursing window into a pre-trained deep learning model, and outputting to obtain a corresponding nursing action result.
Specifically, the computing terminal receives each point cloud data frame in the point cloud information sent by the millimeter wave radar in a preset time window;
judging whether the time window is a nursing window or not; if yes, each point cloud data frame in the nursing window is input to a pre-trained deep learning model, and a corresponding nursing action result is output and obtained, so that the flow complexity of a nursing monitoring method can be effectively reduced, the nursing monitoring cost is effectively reduced, and the experience of a patient can be effectively improved.
The point cloud data frame periodically transmits millimeter wave signals with preset frequency to a space where a user is located by the millimeter wave radar, and respectively receives the millimeter wave signals to form a plurality of different target millimeter wave signals after being reflected by the user; and processing each target millimeter wave signal based on an FFT algorithm, a DOA estimation algorithm and a CFAR algorithm, wherein the user comprises a nursing staff and a nursed person.
To effectively acquire a care window, the determining in step 220 whether the time window is a care window includes:
judging whether the number of the point cloud data frames received in each second in the time window is larger than a preset number threshold value, and whether the proportion of the number of the point cloud data frames in the preset space range in each second to the total number of the point cloud data frames received in each second is larger than a preset proportion threshold value, if so, determining that the time window is a nursing window.
Specifically, referring to fig. 3, the computing terminal determines whether the number of the point cloud data frames received in each second in the time window is greater than a preset number threshold, and whether the ratio of the number of the point cloud data frames in the preset spatial range in each second to the total number of the point cloud data frames received in each second is greater than a preset ratio threshold, if both are, determining that the time window is a care window.
In order to further reduce the complexity of the flow of the nursing monitoring method and effectively reduce the cost of nursing monitoring, in step 220, the step of inputting each point cloud data frame in the nursing window to a pre-trained deep learning model, and outputting to obtain a corresponding nursing action result includes:
and inputting each point cloud data frame in the nursing window into the convolutional neural network, and extracting the corresponding multidimensional characteristics of each point cloud data frame, including the position, doppler speed and signal intensity of the point cloud.
And sequentially inputting the multi-dimensional features into a cyclic neural network according to a time sequence, extracting the associated information of the multi-dimensional features in the time dimension, and outputting a nursing action feature matrix, wherein the matrix comprises a plurality of nursing action types and a plurality of nursing actions corresponding to the nursing action types.
And obtaining the nursing action result based on the nursing action feature matrix and a preset nursing action label.
Specifically, referring to fig. 3, the computing terminal firstly inputs each point cloud data frame (including spatial information, namely X, Y, Z coordinates, doppler information and signal intensity information) in the care window into a convolutional neural network (such as a res net network, and only uses the first three layers of networks) in a frame unit, and extracts multidimensional features corresponding to each point cloud data frame and related to care actions. And sequentially inputting the multi-dimensional features into a cyclic neural network (such as an LSTM network, wherein the number of layers is 1, the number of hidden layer nodes is 128) according to the time sequence, extracting the associated information of the multi-dimensional features in the time dimension, and outputting a nursing action feature matrix of the millimeter wave point cloud continuous frame sequence. And finally, obtaining the type of the nursing action based on the corresponding relation between the serial number of the maximum normalized probability value in each row in the nursing action feature matrix and the preset nursing action label (serial number), and storing the nursing action in a local system, so that the flow complexity of the nursing monitoring method can be further reduced, and the nursing monitoring cost can be effectively reduced.
In summary, the present application provides a nursing action monitoring method based on millimeter wave signals, the method including: periodically transmitting millimeter wave signals with preset frequency to a space where a user is located; respectively receiving millimeter wave signals and forming a plurality of different target millimeter wave signals after the millimeter wave signals are reflected by a user; processing each target millimeter wave signal based on an FFT algorithm, a DOA estimation algorithm and a CFAR algorithm to obtain point cloud information corresponding to the whole of each target millimeter wave signal; and sending each point cloud data frame to a computing terminal, so that the computing terminal receives a plurality of point cloud data frames in a preset time window, judges whether the time window is a nursing window, if so, inputs each point cloud data frame in the nursing window to a pre-trained deep learning model, and outputs a corresponding nursing action result. The nursing monitoring method and the nursing monitoring system can effectively reduce the flow complexity of the nursing monitoring method, effectively reduce the nursing monitoring cost, and further effectively improve the experience of patients.
The embodiment of the application further provides an electronic device, such as a central server, where the electronic device may include a processor, a memory, a receiver, and a transmitter, where the processor is configured to perform the second care action monitoring method based on millimeter wave signals mentioned in the foregoing embodiment, and the processor and the memory may be connected by a bus or other manners, for example, through a bus connection. The receiver may be connected to the processor, memory, by wire or wirelessly.
The processor may be a central processing unit (Central Processing Unit, CPU). The processor may also be any other general purpose processor, digital signal processor (Digital Signal Processor, DSP), application specific integrated circuit (Application Specific Integrated Circuit, ASIC), field programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof.
The memory, as a non-transitory computer readable storage medium, may be configured to store a non-transitory software program, a non-transitory computer executable program, and a module, such as program instructions/modules corresponding to the second care action monitoring method based on millimeter wave signals in the embodiments of the present application. The processor executes various functional applications and data processing of the processor by running non-transitory software programs, instructions, and modules stored in the memory, i.e., to implement the second care action monitoring method based on millimeter wave signals in the above-described method embodiments.
The memory may include a memory program area and a memory data area, wherein the memory program area may store an operating system, at least one application program required for a function; the storage data area may store data created by the processor, etc. In addition, the memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, the memory may optionally include memory located remotely from the processor, the remote memory being connectable to the processor through a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The one or more modules are stored in the memory that, when executed by the processor, perform the second care action monitoring method based on millimeter wave signals in embodiments.
In some embodiments of the present application, the user equipment may include a processor, a memory, and a transceiver unit, where the transceiver unit may include a receiver and a transmitter, and the processor, the memory, the receiver, and the transmitter may be connected by a bus system, the memory storing computer instructions, and the processor executing the computer instructions stored in the memory to control the transceiver unit to transmit and receive signals.
As an implementation manner, the functions of the receiver and the transmitter in the present application may be considered to be implemented by a transceiver circuit or a dedicated chip for transceiver, and the processor may be considered to be implemented by a dedicated processing chip, a processing circuit or a general-purpose chip.
As another implementation manner, a manner of using a general-purpose computer may be considered to implement the server provided in the embodiments of the present application. I.e. program code for implementing the functions of the processor, the receiver and the transmitter are stored in the memory, and the general purpose processor implements the functions of the processor, the receiver and the transmitter by executing the code in the memory.
Embodiments of the present application also provide a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the aforementioned second care action monitoring method based on millimeter wave signals. The computer readable storage medium may be a tangible storage medium such as Random Access Memory (RAM), memory, read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, floppy disks, hard disk, a removable memory disk, a CD-ROM, or any other form of storage medium known in the art.
Those of ordinary skill in the art will appreciate that the various illustrative components, systems, and methods described in connection with the embodiments disclosed herein can be implemented as hardware, software, or a combination of both. The particular implementation is hardware or software dependent on the specific application of the solution and the design constraints. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application. When implemented in hardware, it may be, for example, an electronic circuit, an Application Specific Integrated Circuit (ASIC), suitable firmware, a plug-in, a function card, or the like. When implemented in software, the elements of the present application are the programs or code segments used to perform the required tasks. The program or code segments may be stored in a machine readable medium or transmitted over transmission media or communication links by a data signal carried in a carrier wave.
It should be clear that the present application is not limited to the particular arrangements and processes described above and illustrated in the drawings. For the sake of brevity, a detailed description of known methods is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method processes of the present application are not limited to the specific steps described and illustrated, and those skilled in the art can make various changes, modifications, and additions, or change the order between steps, after appreciating the spirit of the present application.
The features described and/or illustrated in this application for one embodiment may be used in the same way or in a similar way in one or more other embodiments and/or in combination with or instead of the features of the other embodiments.
The foregoing description is only a preferred embodiment of the present application and is not intended to limit the present application, and various modifications and variations may be made to the embodiment of the present application by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.
Claims (10)
1. A millimeter wave signal-based care action monitoring method, comprising:
periodically transmitting millimeter wave signals with preset frequency to a space where a user is located;
respectively receiving the millimeter wave signals and forming a plurality of different target millimeter wave signals after the millimeter wave signals are reflected by a user; the user comprises a nursing staff and a nursed person;
processing each target millimeter wave signal based on an FFT algorithm, a DOA estimation algorithm and a CFAR algorithm to obtain point cloud information corresponding to the whole target millimeter wave signals;
and sending each point cloud data frame in the point cloud information to a computing terminal, so that the computing terminal receives a plurality of point cloud data frames in a preset time window, judges whether the time window is a nursing window, if so, inputs each point cloud data frame in the nursing window to a pre-trained deep learning model, and outputs a corresponding nursing action result.
2. The method for monitoring the nursing action based on the millimeter wave signals according to claim 1, wherein the processing the target millimeter wave signals based on the FFT algorithm, the DOA estimation algorithm and the CFAR algorithm to obtain point cloud information corresponding to the whole target millimeter wave signals comprises:
performing FFT calculation on the frame length dimension of each target millimeter wave signal to obtain a Doppler feature matrix and a distance feature matrix corresponding to each target millimeter wave signal;
obtaining a distance angle matrix corresponding to the whole of each target millimeter wave signal based on a DOA estimation algorithm and each target millimeter wave signal;
and processing the Doppler feature matrix and the corresponding distance angle matrix based on the CFAR target detection algorithm to obtain point cloud information corresponding to the whole millimeter wave signals of the targets.
3. The method for monitoring a nursing action based on millimeter wave signals according to claim 2, wherein the step of obtaining a distance angle matrix corresponding to each target millimeter wave signal as a whole based on the DOA estimation algorithm and each target millimeter wave signal comprises the steps of:
obtaining a conversion coefficient matrix corresponding to each receiving antenna based on the arrangement condition of the receiving antenna corresponding to each target millimeter wave signal;
and multiplying each conversion coefficient matrix by a corresponding distance characteristic matrix, and then adopting a DOA estimation algorithm to process so as to obtain the distance angle matrix.
4. The method for monitoring nursing actions based on millimeter wave signals according to claim 2, wherein the CFAR-based target detection algorithm processes the doppler feature matrix and the corresponding distance angle matrix respectively to obtain point cloud information corresponding to the millimeter wave signals of each target, respectively, and the method comprises the following steps:
calculating the position information of the weighted distance angle matrix based on a CFAR target detection algorithm;
and obtaining point cloud information corresponding to the whole millimeter wave signals of each target based on the Doppler feature matrixes and the position information.
5. A millimeter wave signal-based care action monitoring method, comprising:
receiving each point cloud data frame in the point cloud information sent by the millimeter wave radar in a preset time window; the point cloud information periodically transmits millimeter wave signals with preset frequency to a space where a user is located by the millimeter wave radar, and the millimeter wave signals are respectively received and reflected by the user to form a plurality of different target millimeter wave signals; processing each target millimeter wave signal based on an FFT algorithm, a DOA estimation algorithm and a CFAR algorithm to obtain the target millimeter wave signal, wherein the user comprises a nursing staff and a nursed person;
judging whether the time window is a nursing window or not; if yes, inputting each point cloud data frame in the nursing window into a pre-trained deep learning model, and outputting to obtain a corresponding nursing action result.
6. The millimeter wave signal based care action monitoring method according to claim 5, wherein the determining whether the time window is a care window comprises:
judging whether the number of the point cloud data frames received in each second in the time window is larger than a preset number threshold value, and whether the proportion of the number of the point cloud data frames in the preset space range in each second to the total number of the point cloud data frames received in each second is larger than a preset proportion threshold value, if so, determining that the time window is a nursing window.
7. The millimeter wave signal-based care action monitoring method according to claim 5, wherein the inputting each point cloud data frame in the care window into a pre-trained deep learning model and outputting to obtain a corresponding care action result comprises:
inputting each point cloud data frame in the nursing window into a convolutional neural network, and extracting the multidimensional features corresponding to each point cloud data frame;
sequentially inputting the multi-dimensional features into a cyclic neural network according to a time sequence, extracting the associated information of the multi-dimensional features in a time dimension, and outputting a nursing action feature matrix;
and obtaining the nursing action result based on the nursing action feature matrix and a preset nursing action label.
8. An electronic device, wherein if the electronic device is a millimeter wave radar, the electronic device is configured to implement the millimeter wave signal-based care action monitoring method according to any one of claims 1 to 4;
if the electronic device is a computing terminal, the computing terminal comprises a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the millimeter wave signal-based care action monitoring method according to any one of claims 5 to 7 when executing the computer program.
9. A computer-readable storage medium, on which a computer program is stored, characterized in that the computer program, when executed by a processor, implements the millimeter wave signal-based care action monitoring method as claimed in any one of claims 5 to 7.
10. A computer program product comprising a computer program, characterized in that the computer program, when executed by a processor, implements the millimeter wave signal based care action monitoring method of any one of claims 5 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410293315.3A CN117892202B (en) | 2024-03-14 | 2024-03-14 | Nursing action monitoring method based on millimeter wave signals and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410293315.3A CN117892202B (en) | 2024-03-14 | 2024-03-14 | Nursing action monitoring method based on millimeter wave signals and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117892202A true CN117892202A (en) | 2024-04-16 |
| CN117892202B CN117892202B (en) | 2024-07-16 |
Family
ID=90652096
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410293315.3A Active CN117892202B (en) | 2024-03-14 | 2024-03-14 | Nursing action monitoring method based on millimeter wave signals and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117892202B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10310073B1 (en) * | 2018-02-07 | 2019-06-04 | Infineon Technologies Ag | System and method for determining engagement level of a human being using a millimeter-wave radar sensor |
| CN116110122A (en) * | 2022-12-19 | 2023-05-12 | 万达信息股份有限公司 | Nursing behavior recognition method under privacy scene |
| CN116125458A (en) * | 2023-02-22 | 2023-05-16 | 北京科技大学 | Personnel positioning method based on millimeter wave radar |
| CN116338684A (en) * | 2023-05-29 | 2023-06-27 | 长沙莫之比智能科技有限公司 | Human body falling detection method and system based on millimeter wave radar and deep learning |
-
2024
- 2024-03-14 CN CN202410293315.3A patent/CN117892202B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10310073B1 (en) * | 2018-02-07 | 2019-06-04 | Infineon Technologies Ag | System and method for determining engagement level of a human being using a millimeter-wave radar sensor |
| CN116110122A (en) * | 2022-12-19 | 2023-05-12 | 万达信息股份有限公司 | Nursing behavior recognition method under privacy scene |
| CN116125458A (en) * | 2023-02-22 | 2023-05-16 | 北京科技大学 | Personnel positioning method based on millimeter wave radar |
| CN116338684A (en) * | 2023-05-29 | 2023-06-27 | 长沙莫之比智能科技有限公司 | Human body falling detection method and system based on millimeter wave radar and deep learning |
Non-Patent Citations (4)
| Title |
|---|
| FISCHIONE CARLO 等: ""Guest Editorial Millimeter-Wave Networking"", 《IEEE》, 1 December 2019 (2019-12-01) * |
| 王宁 等: ""基于毫米波感知的小型无源物体定位追踪"", 《物联网学报》, vol. 4, no. 2, 30 June 2020 (2020-06-30) * |
| 郑元杰 等: ""基于时空注意的毫米波雷达人体活动识别网络"", 《计算机应用研究》, vol. 40, no. 8, 31 August 2023 (2023-08-31) * |
| 饶雯 等: ""基于计算机视觉摔跤检测技术研究"", 《南方论坛》, 31 May 2020 (2020-05-31) * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117892202B (en) | 2024-07-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP4131064A1 (en) | Gesture recognition method and related apparatus | |
| CN112859033B (en) | Target detection method and device and related equipment | |
| CN114661142B (en) | Gesture recognition method and device | |
| CN115308734A (en) | Respiratory data calculation method and related equipment | |
| CN113311428A (en) | Intelligent human body falling monitoring system based on millimeter wave radar and identification method | |
| Bhatia et al. | Object classification technique for mmWave FMCW radars using range-FFT features | |
| EP4158378A1 (en) | Clustering in automotive imaging | |
| Yang et al. | Environment adaptive RFID-based 3D human pose tracking with a meta-learning approach | |
| CN113064483A (en) | Gesture recognition method and related device | |
| CN115343704A (en) | Gesture recognition method of FMCW millimeter wave radar based on multi-task learning | |
| CN112198507B (en) | Method and device for detecting human body falling features | |
| Pho et al. | Radar-based face recognition: One-shot learning approach | |
| Xie et al. | Lightweight midrange arm-gesture recognition system from mmwave radar point clouds | |
| Jung et al. | Digit Recognition Using FMCW and UWB Radar Sensors: A Transfer Learning Approach | |
| CN108805112A (en) | A kind of motion recognition system combined based on machine learning and radar | |
| KR20220141748A (en) | Method and computer readable storage medium for extracting target information from radar signal | |
| Kong et al. | A Survey of mmWave Radar-Based Sensing in Autonomous Vehicles, Smart Homes and Industry | |
| CN117892202B (en) | Nursing action monitoring method based on millimeter wave signals and electronic equipment | |
| Kang et al. | Body gesture recognition based on polarimetric micro-Doppler signature and using deep convolutional neural network | |
| CN117452398A (en) | Human body action recognition method based on FMCW radar | |
| CN113514796B (en) | Passive positioning method, system and medium | |
| Zhong et al. | Face Recognition Based on Point Cloud Data Captured by Low-cost mmWave Radar Sensors | |
| Yousaf et al. | Through-the-wall human activity recognition using radar technologies: A review | |
| WO2024119954A1 (en) | Target detection method and apparatus, and target detection model training method and apparatus | |
| Jin | Millimeter-wave radar point cloud classification and interference mitigation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |