CN117158967A - Personnel pressure non-sensing continuous monitoring method and system based on millimeter wave sensing - Google Patents
Personnel pressure non-sensing continuous monitoring method and system based on millimeter wave sensing Download PDFInfo
- Publication number
- CN117158967A CN117158967A CN202310918703.1A CN202310918703A CN117158967A CN 117158967 A CN117158967 A CN 117158967A CN 202310918703 A CN202310918703 A CN 202310918703A CN 117158967 A CN117158967 A CN 117158967A
- Authority
- CN
- China
- Prior art keywords
- point cloud
- input vector
- vector
- neural network
- millimeter wave
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 104
- 238000000034 method Methods 0.000 title claims abstract description 49
- 238000003062 neural network model Methods 0.000 claims abstract description 49
- 230000008447 perception Effects 0.000 claims abstract description 16
- 230000007704 transition Effects 0.000 claims description 41
- 230000008569 process Effects 0.000 claims description 19
- 238000012549 training Methods 0.000 claims description 11
- 230000001133 acceleration Effects 0.000 claims description 10
- 230000001364 causal effect Effects 0.000 claims description 6
- 238000012216 screening Methods 0.000 claims description 6
- 230000000694 effects Effects 0.000 abstract description 46
- 230000007774 longterm Effects 0.000 abstract description 10
- 238000006073 displacement reaction Methods 0.000 description 9
- 238000009826 distribution Methods 0.000 description 9
- 238000012545 processing Methods 0.000 description 8
- 230000000007 visual effect Effects 0.000 description 8
- 238000001914 filtration Methods 0.000 description 7
- 238000010586 diagram Methods 0.000 description 6
- 238000003860 storage Methods 0.000 description 5
- 238000013528 artificial neural network Methods 0.000 description 4
- 230000006399 behavior Effects 0.000 description 4
- 230000008901 benefit Effects 0.000 description 4
- 238000001514 detection method Methods 0.000 description 4
- 230000035882 stress Effects 0.000 description 4
- 206010038743 Restlessness Diseases 0.000 description 3
- 238000009825 accumulation Methods 0.000 description 3
- 230000008859 change Effects 0.000 description 3
- 238000013461 design Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 238000006748 scratching Methods 0.000 description 3
- 230000002393 scratching effect Effects 0.000 description 3
- 208000001431 Psychomotor Agitation Diseases 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 238000002592 echocardiography Methods 0.000 description 2
- 230000009323 psychological health Effects 0.000 description 2
- 208000019901 Anxiety disease Diseases 0.000 description 1
- 208000024172 Cardiovascular disease Diseases 0.000 description 1
- 208000017667 Chronic Disease Diseases 0.000 description 1
- 208000018737 Parkinson disease Diseases 0.000 description 1
- 206010044565 Tremor Diseases 0.000 description 1
- 208000003443 Unconsciousness Diseases 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 230000037328 acute stress Effects 0.000 description 1
- 238000007792 addition Methods 0.000 description 1
- 230000036506 anxiety Effects 0.000 description 1
- 238000005452 bending Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000037326 chronic stress Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 230000003862 health status Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000009545 invasion Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000003340 mental effect Effects 0.000 description 1
- 230000037081 physical activity Effects 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
Landscapes
- Image Analysis (AREA)
Abstract
The application provides a personnel pressure non-sensing continuous monitoring method and a system based on millimeter wave perception, which are used for acquiring point cloud data in each monitoring period based on millimeter wave radars deployed in a user active area, wherein the point cloud data in each monitoring period comprises a plurality of Zhang Dianyun images; calculating the positions of centroids of the point cloud images based on the positions of the point clouds in the point cloud images, constructing a track input image based on the positions of the centroids, and constructing a track input vector based on the track input image; constructing a point cloud input vector based on the point cloud of the point cloud data in each monitoring period; constructing a time stamp input vector based on the acquisition time of each point cloud image; and inputting the track input vector, the point cloud input vector and the time stamp input vector into a pre-trained neural network model, and outputting a pressure grade monitoring result corresponding to a monitoring period based on a classification layer of the neural network model. The application does not need to wear equipment by a user, and is suitable for long-term pressure monitoring of free activity scenes of people in daily home environments.
Description
Technical Field
The application relates to the technical field of psychological pressure assessment, in particular to a personnel pressure non-sensing continuous monitoring method and system based on millimeter wave sensing.
Background
People are under tremendous pressure due to the acceleration of modern work and life pace. Although occasional stress can be relieved by various means (e.g., sleep, entertainment), the accumulation of long-term stress can compromise a person's mental or even physical health. In particular, chronic stress is highly associated with depression, anxiety, etc., and people with great stress are more likely to suffer from cardiovascular diseases and chronic diseases such as parkinson's disease. Therefore, long-term pressure monitoring is beneficial to people in assessing their health status in time and providing doctors with more personalized medical data.
The existing long-term pressure monitoring schemes are mainly divided into two categories, namely questionnaire filling and wearable device measurement. The questionnaire-based regimen records the stress level of subjects by periodically filling a carefully designed questionnaire, however, it is cumbersome and burdensome for the user, and the user easily loses compliance and returns unreliable results during long-term monitoring; the scheme based on the wearable device for measuring the physiological signals of the human body requires people to wear the device for a long time and charge the device regularly, and the discomfort caused by the scheme limits the wide use of the wearable device.
Disclosure of Invention
In view of the foregoing, embodiments of the present application provide a method for continuous monitoring of human pressure without sense based on millimeter wave sensing to obviate or ameliorate one or more of the disadvantages of the prior art.
The application provides a millimeter wave perception-based personnel pressure non-sensing continuous monitoring method, which comprises the following steps:
acquiring point cloud data in each monitoring period based on millimeter wave radar deployed in a user active area, wherein the point cloud data in each monitoring period comprises a plurality of Zhang Dianyun images;
calculating the positions of centroids of the point cloud images based on the positions of the point clouds in the point cloud images, constructing a track input image based on the positions of a plurality of centroids, and constructing a track input vector based on the track input image;
constructing a point cloud input vector based on the point cloud of the point cloud data in each monitoring period;
constructing a time stamp input vector based on the acquisition time of each point cloud image;
and inputting the track input vector, the point cloud input vector and the timestamp input vector into a pre-trained neural network model, and outputting a pressure grade monitoring result corresponding to a monitoring period based on a classification layer of the neural network model.
By adopting the scheme, the millimeter wave radar can be utilized to sense the daily life activities of the user, and the replacement activities can be extracted from the activities, namely, the subjects with the people under pressure unconsciously show the behavior of restlessness, such as scratching, loitering, leg shaking and the like, and the relation between the activities and the pressure level is established. The application does not need to wear equipment by a user, and is suitable for long-term pressure monitoring in a free activity scene of a person in a daily household environment. In addition, since the millimeter wave radar generates only coarse-grained point cloud data of the user, which describes coordinates of the most active part of the human body, instead of visual images generated by a camera, the privacy intrusion of the application is much less. The millimeter wave signal is insensitive to light conditions, can work even in darkness, and can provide personalized psychological health condition references for users according to detection of the millimeter wave radar.
In some embodiments of the present application, the neural network model includes a classification module, the classification module is provided with a full connection layer and a classification layer, the full connection layer outputs a third transition vector, the trajectory input vector, the point cloud input vector and the timestamp input vector are input into a pre-trained neural network model, and the step of outputting a pressure level monitoring result corresponding to a monitoring period based on the classification layer of the neural network model includes:
calculating a balance vector based on the training data set of the neural network model in the pre-training process, the type of the data label and the number of data under each type of the data label;
and calculating a fourth transition vector based on the balance vector and the third transition vector, outputting the fourth transition vector to a classification layer, and outputting a pressure grade monitoring result corresponding to a monitoring period by the classification layer.
In some embodiments of the present application, in the step of calculating a fourth transition vector based on the balance vector and the third transition vector, the fourth transition vector is calculated according to the following formula:
Δ＝argmax y∈L [(f y (x)-τln(π y )]；
wherein Δ represents a fourth transition vector, L represents a class of data tags, y represents a class of any data tag, f y (x) Represents a third transition vector, pi y Representing the balance vector, τ representing the preset parameter.
In the specific implementation process, τ∈ (0, 1).
In some embodiments of the present application, the neural network model includes a data imbalance processing module, where the data imbalance processing module calculates a balance vector based on a training data set of the neural network model in a pre-training process, a class of data labels and a number of data under each class of data labels, and calculates τln (pi) based on a preset parameter y )。
In the implementation process, the fourth transition vector is input into the classification layer to obtain a pressure grade monitoring result corresponding to the monitoring period.
In some embodiments of the present application, in the step of calculating the position of the centroid of the point cloud image based on the position of the point cloud in the point cloud image, the position of the centroid of the point cloud image based on the point cloud is calculated using a DBSCAN clustering algorithm and a kalman filtering algorithm.
In the specific implementation process, if the point cloud image has the point cloud after collection, calculating the centroid position of the point cloud by adopting a DBSCAN clustering algorithm; if the point cloud image does not have the acquired point cloud, calculating the centroid position of the frame based on the centroid position and centroid speed of the last frame by adopting a Kalman filtering algorithm.
In some embodiments of the present application, the constructing a trajectory input image based on the positions of the plurality of centroids, the constructing a trajectory input vector based on the trajectory input image includes:
connecting the positions of the centroids sequentially based on the time acquired by the point cloud images to obtain the track input image, wherein the track input image comprises the position, the speed and the acceleration data of each centroid;
and taking the position, the speed and the acceleration data of each centroid in the track input image as parameters of each dimension in the track input vector to construct the track input vector.
In some embodiments of the present application, in the step of constructing a point cloud input vector based on the point clouds of the point cloud data in each monitoring period, a cartesian coordinate, a radial distance, a horizontal angle, an azimuth angle, and a radial velocity of each point cloud in each point cloud image are taken as parameters of respective dimensions in the point cloud input vector, and a point cloud input vector is constructed;
in the step of constructing the time stamp input vector based on the acquisition time of each point cloud image, the week, hour, minute, second and millisecond of the acquisition time of each point cloud image are respectively used as parameters of each dimension of the time stamp input vector, and the time stamp input vector is constructed.
In some embodiments of the present application, the step of constructing the point cloud input vector based on the point clouds of the point cloud data in each monitoring period further includes screening the point clouds in each point cloud image based on the signal intensity of the point clouds in each point cloud image, screening each point cloud image to obtain a preset number of point clouds, and constructing the screened point clouds as the point cloud input vector.
In some embodiments of the present application, in the step of inputting the trajectory input vector, the point cloud input vector, and the timestamp input vector into a pre-trained neural network model, outputting a pressure level monitoring result corresponding to a monitoring period based on a classification layer of the neural network model, the neural network model includes a dimension raising module, a global view module, a local view module, and a classification module, and the trajectory input vector, the point cloud input vector, and the timestamp input vector are sequentially processed by the dimension raising module, the global view module, the local view module, and the classification module to obtain the pressure level monitoring result.
In some embodiments of the present application, in the step of inputting the trajectory input vector, the point cloud input vector, and the timestamp input vector into a pre-trained neural network model, outputting a pressure level monitoring result corresponding to a monitoring period based on a classification layer of the neural network model, the trajectory input vector, the point cloud input vector, and the timestamp input vector are respectively raised to a preset dimension through a dimension raising module, and the trajectory input vector, the point cloud input vector, and the timestamp input vector raised to the preset dimension are combined into a first transition vector and input into a global view module.
In some embodiments of the present application, the global view module is provided with a transform layer, the local view module is provided with a causal expansion convolution layer, and in the step of inputting the trajectory input vector, the point cloud input vector and the timestamp input vector into a pre-trained neural network model, outputting a pressure level monitoring result corresponding to a monitoring period based on a classification layer of the neural network model, the first transition vector is sequentially processed through the transform layer and the causal expansion convolution layer to obtain a second transition vector.
In some embodiments of the present application, in the step of inputting the trajectory input vector, the point cloud input vector, and the timestamp input vector into a pre-trained neural network model, the classification layer based on the neural network model outputs the pressure level monitoring result corresponding to the monitoring period, the second transition vector is sequentially processed through the full connection layer and the classification layer, and the classification layer outputs the pressure level monitoring result corresponding to the monitoring period.
The second aspect of the present application also provides a millimeter wave wireless sensing based pressure system comprising a computer device comprising a processor and a memory, the memory having stored therein computer instructions for executing the computer instructions stored in the memory, the system implementing the steps implemented by the method as described above when the computer instructions are executed by the processor.
The third aspect of the present application also provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps implemented by the aforementioned millimeter wave perception based continuous monitoring method of person pressure insensitivity.
Additional advantages, objects, and features of the application will be set forth in part in the description which follows and in part will become apparent to those having ordinary skill in the art upon examination of the following or may be learned from practice of the application. The objectives and other advantages of the application may be realized and attained by the structure particularly pointed out in the written description and drawings.
It will be appreciated by those skilled in the art that the objects and advantages that can be achieved with the present application are not limited to the above-described specific ones, and that the above and other objects that can be achieved with the present application will be more clearly understood from the following detailed description.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate and together with the description serve to explain the application.
FIG. 1 is a schematic diagram of one embodiment of a continuous monitoring method for personnel pressure based on millimeter wave perception;
FIG. 2 is a schematic diagram of another embodiment of a continuous monitoring method of personnel pressure based on millimeter wave perception according to the present application;
FIG. 3 is a schematic diagram of a continuous monitoring method of personnel pressure based on millimeter wave perception;
FIG. 4 is a schematic diagram of a usage scenario of the continuous monitoring method of personnel pressure non-sensing based on millimeter wave sensing of the application;
FIG. 5 is a schematic diagram of an implementation flow of a continuous monitoring method of personnel pressure based on millimeter wave perception;
fig. 6 is a schematic diagram of a neural network structure of the continuous monitoring method of personnel pressure non-sensing based on millimeter wave sensing.
Detailed Description
The present application will be described in further detail with reference to the following embodiments and the accompanying drawings, in order to make the objects, technical solutions and advantages of the present application more apparent. The exemplary embodiments of the present application and the descriptions thereof are used herein to explain the present application, but are not intended to limit the application.
It should be noted here that, in order to avoid obscuring the present application due to unnecessary details, only structures and/or processing steps closely related to the solution according to the present application are shown in the drawings, while other details not greatly related to the present application are omitted.
In order to solve the above problems, as shown in fig. 1 and 5, the present application provides a millimeter wave sensing-based continuous monitoring method for pressure of a person, which comprises the following steps:
step S100, acquiring point cloud data in each monitoring period based on millimeter wave radar deployed in a user active area, wherein the point cloud data in each monitoring period comprises a plurality of Zhang Dianyun images;
in some embodiments of the present application, the millimeter wave radar may be installed in a user activity area (may be placed in a corner of a house) through a bracket, the millimeter wave emits signals and receives the signals, different activities of the user may affect the millimeter wave signals to different extents, and we may generate point cloud data (describing coordinates of the most active part of the human body) of the human body by analyzing echoes. The change of the human activity type can cause the difference of the point cloud distribution, the displacement activity caused by pressure can be extracted through long-time data accumulation and analysis, the pressure change condition of the user in a period of time is presumed through the occurrence frequency and the amplitude of the displacement activity of the human, and the pressure grade monitoring result of the user is output. The user can not perceive the existence of the millimeter wave sensor in the whole process, the activity of the user can not be limited, and the privacy of the user can not be disturbed, so that the long-term pressure monitoring in a home scene can be realized.
The millimeter wave radar may be a frequency modulated continuous wave radar (Frequency Modulated Continuous Wave Radar, FMCW) that, in an implementation, transmits FMCW millimeter wave signals using a frequency of 60GHz FMCW radar.
Step S200, calculating the positions of centroids of the point cloud images based on the positions of the point clouds in the point cloud images, constructing a track input image based on the positions of a plurality of centroids, and constructing a track input vector based on the track input image;
in some embodiments of the present application, a DBSCAN clustering algorithm and a kalman filtering algorithm may be used to calculate a position of a centroid based on a point cloud image, each point cloud image is calculated to obtain a position of a centroid, and the positions of the corresponding centroids are sequentially connected based on a time acquired by the point cloud images to obtain the track input image, where the track input image includes position, speed and acceleration data of each centroid, and the centroid speed and acceleration data are determined by the positions of centroids, a relative position between centroids, and an acquisition time of a point cloud image corresponding to the centroid.
Step S300, constructing a point cloud input vector based on the point cloud of the point cloud data in each monitoring period;
step S400, constructing a time stamp input vector based on the acquisition time of each point cloud image;
and S500, inputting the track input vector, the point cloud input vector and the timestamp input vector into a pre-trained neural network model, and outputting a pressure grade monitoring result corresponding to a monitoring period based on a classification layer of the neural network model.
By adopting the scheme, the millimeter wave radar can be utilized to sense the daily life activities of the user, and the replacement activities can be extracted from the activities, namely, the subjects with the people under pressure unconsciously show the behavior of restlessness, such as scratching, loitering, leg shaking and the like, and the relation between the activities and the pressure level is established. The application does not need to wear equipment by a user, and is suitable for long-term pressure monitoring in a free activity scene of a person in a daily household environment. In addition, since the millimeter wave radar generates only coarse-grained point cloud data of the user, which describes coordinates of the most active part of the human body, instead of visual images generated by a camera, the privacy intrusion of the application is much less. The millimeter wave signal is insensitive to light conditions, can work even in darkness, and can provide personalized psychological health condition references for users according to detection of the millimeter wave radar.
As shown in fig. 3, the millimeter wave sensor is placed at a position capable of sensing the whole user activity area, millimeter waves emit signals and receive the signals, different activities of the user can generate different degrees of influence on the millimeter wave signals, and point cloud data of human body activities can be generated by analyzing echoes. The variation of the human activity type can cause the difference of the point cloud distribution, the displacement activity caused by the pressure can be extracted through long-time data accumulation and analysis, as shown in fig. 4, the pressure variation condition of the user in a period of time is presumed through the occurrence frequency and the amplitude of the displacement activity of the human, and the pressure of the user is output. The user can not perceive the existence of the millimeter wave sensor in the whole process, the activity of the user can not be limited, and the privacy of the user can not be disturbed, so that the long-term pressure monitoring in a home scene can be realized.
As shown in fig. 2, in some embodiments of the present application, the neural network model includes a classification module, the classification module is provided with a full connection layer and a classification layer, the full connection layer outputs a third transition vector, the trajectory input vector, the point cloud input vector and the timestamp input vector are input into a pre-trained neural network model, and the step of outputting a pressure level monitoring result corresponding to a monitoring period based on the classification layer of the neural network model includes:
step S510, calculating a balance vector based on the category of the data label and the number of data under each data label in a training data set of the neural network model in the pre-training process;
and step S520, calculating a fourth transition vector based on the balance vector and the third transition vector, and outputting the fourth transition vector to a classification layer, wherein the classification layer outputs a pressure grade monitoring result corresponding to a monitoring period.
In some embodiments of the present application, in the step of calculating a fourth transition vector based on the balance vector and the third transition vector, the fourth transition vector is calculated according to the following formula:
Δ＝argmax y∈L (f y (x)-τln(π y )]；
wherein Δ represents a fourth transition vector, L represents a class of data tags, y represents a class of any data tag, f y (x) Represents a third transition vector, pi y Representing the balance vector, τ representing the preset parameter.
In the specific implementation process, τ∈ (0, 1).
In some embodiments of the present application, the neural network model includes a data imbalance processing module, which calculates a balance vector based on a class of data labels and a number of data under each class of data labels based on a training dataset of the neural network model in a pre-training process, and calculates τln (pi) based on a preset parameter y )。
As shown in fig. 6, in some embodiments of the present application, the data unbalance processing module performs a vector adjustment and a vector bias, in which a balance vector is calculated based on a class of data tags and the number of data under each class of data tags, and in which τln (pi) is calculated based on a preset parameter y )。
In the specific implementation process, in the step of calculating the balance vector based on the class of the data tag and the number of data under each class of the data tag, the ratio of the number of data under each class of the data tag to the total number of data is taken as a parameter of each dimension of the balance vector, and the balance vector is constructed.
By adopting the scheme, in order to solve the problem of unbalanced pressure data distribution, the scheme is based onClass of data labels and number of data under each class of data labels calculate balance vector, fourth transition vector is calculated through formula, and delta=argmax y∈L [f y (x)-τln(π y )]The prior distribution of the pressure labels is used for giving different weights to the middle characteristics of the network, so that the network pays attention to the middle-level pressure and high-level pressure characteristics with less quantity, the average accuracy of the pressure of each level is optimal, a fourth more accurate transition vector is obtained, and classification is more accurate.
In some embodiments of the present application, in the step of calculating the position of the centroid of the point cloud image based on the position of the point cloud in the point cloud image, the position of the centroid of the point cloud image based on the point cloud is calculated using a DBSCAN clustering algorithm and a kalman filtering algorithm.
In a specific implementation process, DBSCAN is a clustering algorithm capable of clustering point clouds to generate centroids, and Kalman filtering algorithm is a recursive predictive filtering algorithm capable of being used for centroid estimation of the point clouds. Specifically, the DBSCAN clustering algorithm clusters point clouds to generate centroids, and the Kalman filtering algorithm can enhance track continuity by predicting and updating coordinates of the point clouds.
In some embodiments of the present application, the constructing a trajectory input image based on the positions of the plurality of centroids, the constructing a trajectory input vector based on the trajectory input image includes:
connecting the positions of the centroids sequentially based on the time acquired by the point cloud images to obtain the track input image, wherein the track input image comprises the position, the speed and the acceleration data of each centroid;
and taking the position, the speed and the acceleration data of each centroid in the track input image as parameters of each dimension in the track input vector to construct the track input vector.
In a specific implementation process, each of the position, the speed and the acceleration data of the centroid consists of three dimensional data of x, y and z, and the trajectory input vector comprises parameters of 9 dimensions.
In some embodiments of the present application, in the step of constructing a point cloud input vector based on the point clouds of the point cloud data in each monitoring period, a cartesian coordinate, a radial distance, a horizontal angle, an azimuth angle, and a radial velocity of each point cloud in each point cloud image are taken as parameters of respective dimensions in the point cloud input vector, and a point cloud input vector is constructed;
in a specific implementation process, the cartesian coordinates of the point cloud are composed of three dimensions of x, y and z, and if each point cloud image includes 20 point clouds, the point cloud input vector includes parameters of 7×20=140 dimensions.
In the step of constructing the time stamp input vector based on the acquisition time of each point cloud image, the week, hour, minute, second and millisecond of the acquisition time of each point cloud image are respectively used as parameters of each dimension of the time stamp input vector, and the time stamp input vector is constructed.
In a specific implementation, the timestamp input vector includes parameters of 5 dimensions.
In some embodiments of the present application, the step of constructing the point cloud input vector based on the point clouds of the point cloud data in each monitoring period further includes screening the point clouds in each point cloud image based on the signal intensity of the point clouds in each point cloud image, screening each point cloud image to obtain a preset number of point clouds, and constructing the screened point clouds as the point cloud input vector.
In the implementation process, the preset number may be 20, and 20 point clouds with stronger signal strength are screened out from each point cloud image.
In the implementation process, if the number of the point clouds in the point cloud image is less than 20, the original point clouds are duplicated to enable the number of the point clouds to reach 20.
In some embodiments of the present application, in the step of inputting the trajectory input vector, the point cloud input vector, and the timestamp input vector into a pre-trained neural network model, outputting a pressure level monitoring result corresponding to a monitoring period based on a classification layer of the neural network model, the neural network model includes a dimension raising module, a global view module, a local view module, and a classification module, and the trajectory input vector, the point cloud input vector, and the timestamp input vector are sequentially processed by the dimension raising module, the global view module, the local view module, and the classification module to obtain the pressure level monitoring result.
By adopting the scheme, the processed time, point cloud and track characteristics are input into the neural network designed by us. The neural network structure is shown in fig. 6, and consists of four parts, namely a dimension raising module, a global visual field module, a local visual field module and a data unbalance processing module. The dimension-increasing module is used for increasing the dimension of the human activity characteristics and amplifying the difference between the activity characteristics; the global visual field module is used for extracting displacement activities (the displacement activities are unconsciously performed by a subject under pressure, such as actions of scratching, loitering, leg trembling and the like) from the activity sequence as much as possible by using a self-attention mechanism, so that the sensitivity of the model to the displacement activities is improved; the displacement activity is usually continuous and of an indefinite length, the local visual field module can be regarded as a sliding window with variable length, so that the correctness of the characteristic of the displacement activity is checked by the local visual field, and the accuracy of the model is improved; in order to solve the problem of unbalanced pressure data distribution, the prior distribution of the pressure labels is used for giving different weights to the middle characteristics of the network, so that the network pays attention to the middle-level pressure and high-level pressure characteristics with less quantity, and the model is trained through the data unbalance processing module, so that the average accuracy of the pressures of all levels is optimal.
In some embodiments of the present application, in the step of inputting the trajectory input vector, the point cloud input vector, and the timestamp input vector into a pre-trained neural network model, outputting a pressure level monitoring result corresponding to a monitoring period based on a classification layer of the neural network model, the trajectory input vector, the point cloud input vector, and the timestamp input vector are respectively raised to a preset dimension through a dimension raising module, and the trajectory input vector, the point cloud input vector, and the timestamp input vector raised to the preset dimension are combined into a first transition vector and input into a global view module.
In a specific implementation process, the preset dimension is 1024 dimensions.
In some embodiments of the present application, the dimensions of the trajectory input vector, the point cloud input vector, and the timestamp input vector may be raised using a dimension raising module provided with a convolution layer and a pooling layer.
In some embodiments of the present application, the global view module is provided with a transform layer, the local view module is provided with a causal expansion convolution layer, and in the step of inputting the trajectory input vector, the point cloud input vector and the timestamp input vector into a pre-trained neural network model, outputting a pressure level monitoring result corresponding to a monitoring period based on a classification layer of the neural network model, the first transition vector is sequentially processed through the transform layer and the causal expansion convolution layer to obtain a second transition vector.
In some embodiments of the present application, the classification module is provided with a full connection layer and a classification layer, and in the step of inputting the trajectory input vector, the point cloud input vector, and the timestamp input vector into the pre-trained neural network model, the classification layer based on the neural network model outputs the pressure level monitoring result corresponding to the monitoring period, the second transition vector is processed sequentially through the full connection layer and the classification layer, and the classification layer outputs the pressure level monitoring result corresponding to the monitoring period.
In a specific implementation, the classification layer may use a softmax classifier for classification.
In some embodiments of the application, the pressure level monitoring results include three results, high, medium, and low.
By adopting the scheme, the pressure level of the person is identified by utilizing the millimeter wave wireless signals, the daily life activities of the person are perceived on the premise of non-contact and no invasion of privacy, and the pressure of the person is extracted from the daily life activities.
The present solution is able to extract "replacement activities" from redundant daily activities for pressure speculation, while "replacement activities" are typically temporary and submerged in other normal daily activities. For example, a subject under acute stress may flex his head from time to time, but each flexing usually lasts only a few seconds, and inclusions of other normal activities occur irregularly. Furthermore, the "replacement activities" vary widely from individual to individual, and appear as different restless behaviors. For example, when one subject is bending his head under pressure, the other subject may shake his legs, etc. The scheme designs a neural network, which can learn human activities on macroscopic and microscopic time scales, capture replacement activities and perform pressure level speculation.
According to the scheme, sparse millimeter wave point cloud can be effectively utilized, and the distribution of the processing pressure data is unbalanced. Millimeter wave perception data, i.e., point clouds, are much more sparse than visual images, which presents additional difficulties for behavior detection and recognition, and in order to handle the sparsity of millimeter wave point clouds, we increase the dimension of the data to amplify the differences between the active features. Furthermore, we have found that most of the pressure data is distributed at low pressure, which can lead to a bias in the final pressure estimation results towards low pressure, and that the results of high and medium pressures are confused. To avoid this problem, we use the prior distribution of pressure at each level during model training, giving more weight to medium and high level pressures with less data distribution to optimize the average accuracy of each pressure level.
The beneficial effect of this scheme includes:
1. the application can robustly realize pressure speculation with higher accuracy in the free activity scene of the user, and is realized based on the recognition of unconscious activity (replacement activity) expressed by human under the pressure state;
2. the application discloses the relation between the replacement activity and the pressure of people, provides a brand-new scheme for the pressure detection in the birth period, and designs a pressure monitoring algorithm based on the identification of the replacement activity;
3. the application has higher recognition precision, and can realize recognition accuracy of more than 80% of pressure speculation for long-term deployment of a single user in a home environment;
4. the application builds a system prototype by using the millimeter wave radar in the range of 60GHz-64GHz, can perform higher-precision pressure speculation in a non-contact manner, and is suitable for the deployment in the production period in a real home environment.
The embodiment of the application also provides a pressure system based on millimeter wave wireless sensing, which comprises computer equipment, wherein the computer equipment comprises a processor and a memory, the memory is stored with computer instructions, the processor is used for executing the computer instructions stored in the memory, and the system realizes the steps realized by the method when the computer instructions are executed by the processor.
The embodiment of the application also provides a computer readable storage medium, on which a computer program is stored, which when being executed by a processor, is used for realizing the steps realized by the personnel pressure non-sensing continuous monitoring method based on millimeter wave perception. The computer readable storage medium may be a tangible storage medium such as Random Access Memory (RAM), memory, read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, floppy disks, hard disk, a removable memory disk, a CD-ROM, or any other form of storage medium known in the art.
Those of ordinary skill in the art will appreciate that the various illustrative components, systems, and methods described in connection with the embodiments disclosed herein can be implemented as hardware, software, or a combination of both. The particular implementation is hardware or software dependent on the specific application of the solution and the design constraints. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application. When implemented in hardware, it may be, for example, an electronic circuit, an Application Specific Integrated Circuit (ASIC), suitable firmware, a plug-in, a function card, or the like. When implemented in software, the elements of the application are the programs or code segments used to perform the required tasks. The program or code segments may be stored in a machine readable medium or transmitted over transmission media or communication links by a data signal carried in a carrier wave.
It should be understood that the application is not limited to the particular arrangements and instrumentality described above and shown in the drawings. For the sake of brevity, a detailed description of known methods is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method processes of the present application are not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between steps, after appreciating the spirit of the present application.
In this disclosure, features that are described and/or illustrated with respect to one embodiment may be used in the same way or in a similar way in one or more other embodiments and/or in combination with or instead of the features of the other embodiments.
The above description is only of the preferred embodiments of the present application and is not intended to limit the present application, and various modifications and variations can be made to the embodiments of the present application by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the protection scope of the present application.
Claims (10)
1. The staff pressure non-sensing continuous monitoring method based on millimeter wave sensing is characterized by comprising the following steps of:
acquiring point cloud data in each monitoring period based on millimeter wave radar deployed in a user active area, wherein the point cloud data in each monitoring period comprises a plurality of Zhang Dianyun images;
calculating the positions of centroids of the point cloud images based on the positions of the point clouds in the point cloud images, constructing a track input image based on the positions of a plurality of centroids, and constructing a track input vector based on the track input image;
constructing a point cloud input vector based on the point cloud of the point cloud data in each monitoring period;
constructing a time stamp input vector based on the acquisition time of each point cloud image;
and inputting the track input vector, the point cloud input vector and the timestamp input vector into a pre-trained neural network model, and outputting a pressure grade monitoring result corresponding to a monitoring period based on a classification layer of the neural network model.
2. The millimeter wave perception based personnel pressure non-sensing continuous monitoring method according to claim 1, wherein the neural network model comprises a classification module, the classification module is provided with a full connection layer and a classification layer, the full connection layer outputs a third transition vector, the trajectory input vector, the point cloud input vector and the timestamp input vector are input into a pre-trained neural network model, and the step of outputting a pressure level monitoring result corresponding to a monitoring period based on the classification layer of the neural network model comprises the following steps:
calculating a balance vector based on the training data set of the neural network model in the pre-training process, the type of the data label and the number of data under each type of the data label;
and calculating a fourth transition vector based on the balance vector and the third transition vector, outputting the fourth transition vector to a classification layer, and outputting a pressure grade monitoring result corresponding to a monitoring period by the classification layer.
3. The millimeter wave perception based person pressure non-sensing continuous monitoring method according to claim 2, wherein in the step of calculating a fourth transition vector based on the balance vector and the third transition vector, the fourth transition vector is calculated according to the following formula:
Δ＝argmax y∈L [f y (x)-τln(π y )]；
wherein Δ represents a fourth transition vector, L represents a class of data tags, y represents a class of any data tag, f y (x) Represents a third transition vector, pi y Representing the balance vector, τ representing the preset parameter.
4. The millimeter wave perception based person pressure non-sensing continuous monitoring method according to claim 1, wherein the step of constructing a trajectory input image based on the positions of the plurality of centroids, and constructing a trajectory input vector based on the trajectory input image comprises:
connecting the positions of the centroids sequentially based on the time acquired by the point cloud images to obtain the track input image, wherein the track input image comprises the position, the speed and the acceleration data of each centroid;
and taking the position, the speed and the acceleration data of each centroid in the track input image as parameters of each dimension in the track input vector to construct the track input vector.
5. The continuous monitoring method of personnel pressure based on millimeter wave perception according to claim 1, wherein in the step of constructing a point cloud input vector based on the point cloud of the point cloud data in each monitoring period, cartesian coordinates, radial distance, horizontal angle, azimuth angle and radial velocity of each point cloud in each point cloud image are taken as parameters of each dimension in the point cloud input vector, and the point cloud input vector is constructed;
in the step of constructing the time stamp input vector based on the acquisition time of each point cloud image, the week, hour, minute, second and millisecond of the acquisition time of each point cloud image are respectively used as parameters of each dimension of the time stamp input vector, and the time stamp input vector is constructed.
6. The continuous monitoring method of personnel pressure non-sensing based on millimeter wave sensing according to claim 1, wherein the step of constructing a point cloud input vector based on the point cloud of the point cloud data in each monitoring period further comprises screening the point clouds in each point cloud image based on the signal intensity of the point clouds in each point cloud image, screening each point cloud image to obtain a preset number of point clouds, and constructing the screened point clouds as the point cloud input vector.
7. The continuous monitoring method of personnel pressure based on millimeter wave perception according to any one of claims 1 to 6, wherein in the step of inputting the trajectory input vector, the point cloud input vector and the timestamp input vector into a pre-trained neural network model, outputting a pressure level monitoring result corresponding to a monitoring period based on a classification layer of the neural network model, the neural network model comprises a dimension raising module, a global view module, a local view module and a classification module, and the trajectory input vector, the point cloud input vector and the timestamp input vector are sequentially processed by the dimension raising module, the global view module, the local view module and the classification module to obtain the pressure level monitoring result.
8. The continuous monitoring method of personnel pressure based on millimeter wave perception according to claim 7, wherein in the step of inputting the trajectory input vector, the point cloud input vector and the time stamp input vector into a pre-trained neural network model and outputting the pressure level monitoring result corresponding to the monitoring period based on the classification layer of the neural network model, the trajectory input vector, the point cloud input vector and the time stamp input vector are respectively raised to a preset dimension through a dimension raising module, and the trajectory input vector, the point cloud input vector and the time stamp input vector raised to the preset dimension are combined into a first transition vector to be input into a global view module.
9. The continuous monitoring method of personnel pressure based on millimeter wave perception according to claim 8, wherein the global view module is provided with a transducer layer, the local view module is provided with a causal expansion convolution layer, and in the step of inputting the trajectory input vector, the point cloud input vector and the timestamp input vector into a pre-trained neural network model, outputting a pressure level monitoring result corresponding to a monitoring period based on a classification layer of the neural network model, the first transition vector is sequentially processed through the transducer layer and the causal expansion convolution layer to obtain a second transition vector.
10. A millimeter wave wireless sensing based pressure system, characterized in that the system comprises a computer device comprising a processor and a memory, said memory having stored therein computer instructions for executing the computer instructions stored in said memory, the system realizing the steps of the method according to any of claims 1-9 when said computer instructions are executed by the processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310918703.1A CN117158967B (en) | 2023-07-25 | Personnel pressure non-sensing continuous monitoring method and system based on millimeter wave sensing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310918703.1A CN117158967B (en) | 2023-07-25 | Personnel pressure non-sensing continuous monitoring method and system based on millimeter wave sensing |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117158967A true CN117158967A (en) | 2023-12-05 |
| CN117158967B CN117158967B (en) | 2024-06-04 |
Family
ID=
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20220198806A1 (en) * | 2020-12-21 | 2022-06-23 | Beihang University | Target detection method based on fusion of prior positioning of millimeter-wave radar and visual feature |
| CN114818788A (en) * | 2022-04-07 | 2022-07-29 | 北京邮电大学 | Tracking target state identification method and device based on millimeter wave perception |
| CN115205891A (en) * | 2022-05-31 | 2022-10-18 | 北京邮电大学 | Personnel behavior recognition model training method, behavior recognition method and device |
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20220198806A1 (en) * | 2020-12-21 | 2022-06-23 | Beihang University | Target detection method based on fusion of prior positioning of millimeter-wave radar and visual feature |
| CN114818788A (en) * | 2022-04-07 | 2022-07-29 | 北京邮电大学 | Tracking target state identification method and device based on millimeter wave perception |
| CN115205891A (en) * | 2022-05-31 | 2022-10-18 | 北京邮电大学 | Personnel behavior recognition model training method, behavior recognition method and device |
Non-Patent Citations (2)
| Title |
|---|
| 王宁等: "基于毫米波感知的小型无源物体定位追踪", 物联网学报, vol. 4, no. 2, 18 February 2020 (2020-02-18), pages 105 - 112 * |
| 陶丹，马华东: "视频传感器网络中基于相关性图像融合算法", 计算机辅助设计与图形学学报, vol. 19, no. 5, 15 May 2007 (2007-05-15), pages 656 - 666 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Vallabh et al. | Fall detection monitoring systems: a comprehensive review | |
| Saboor et al. | Latest research trends in gait analysis using wearable sensors and machine learning: A systematic review | |
| Deep et al. | A survey on anomalous behavior detection for elderly care using dense-sensing networks | |
| Nooruddin et al. | Sensor-based fall detection systems: a review | |
| Alvarez et al. | Behavior analysis through multimodal sensing for care of Parkinson’s and Alzheimer’s patients | |
| CN102427765B (en) | Sensing device for detecting a wearing position | |
| CN107423549A (en) | Healthy tracking equipment | |
| Tunçel et al. | Classifying human leg motions with uniaxial piezoelectric gyroscopes | |
| Lin et al. | Innovative head-mounted system based on inertial sensors and magnetometer for detecting falling movements | |
| Han et al. | IR-UWB sensor based fall detection method using CNN algorithm | |
| CN109864704A (en) | Bio signal quality evaluation equipment and bio signal quality evaluating method | |
| Aich et al. | Design of a Machine Learning‐Assisted Wearable Accelerometer‐Based Automated System for Studying the Effect of Dopaminergic Medicine on Gait Characteristics of Parkinson’s Patients | |
| CN110456320A (en) | A kind of ULTRA-WIDEBAND RADAR personal identification method based on free space gait temporal aspect | |
| CN111208508A (en) | Motion quantity measuring method and device and electronic equipment | |
| Rafferty et al. | Fall detection through thermal vision sensing | |
| Zhao et al. | Sensor-based fall risk assessment: a survey | |
| Pham et al. | Application of ultrasonic sensors in a smart environment | |
| Liang et al. | mmStress: Distilling Human Stress from Daily Activities via Contact-less Millimeter-wave Sensing | |
| CN117158967B (en) | Personnel pressure non-sensing continuous monitoring method and system based on millimeter wave sensing | |
| Zhang et al. | AutoQual: task-oriented structural vibration sensing quality assessment leveraging co-located mobile sensing context | |
| US20240032820A1 (en) | System and method for self-learning and reference tuning activity monitor | |
| Su et al. | Radar placement for fall detection: Signature and performance | |
| CN117158967A (en) | Personnel pressure non-sensing continuous monitoring method and system based on millimeter wave sensing | |
| US20230123815A1 (en) | Stability scoring of individuals utilizing inertial sensor device | |
| Wu et al. | Unobtrusive monitoring of sedentary behaviors with fusion of bluetooth and ballistocardiogram signals |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |