CN117281516A - Mental stress monitoring system, method, medium and device - Google Patents
Mental stress monitoring system, method, medium and device Download PDFInfo
- Publication number
- CN117281516A CN117281516A CN202311580851.3A CN202311580851A CN117281516A CN 117281516 A CN117281516 A CN 117281516A CN 202311580851 A CN202311580851 A CN 202311580851A CN 117281516 A CN117281516 A CN 117281516A
- Authority
- CN
- China
- Prior art keywords
- mental stress
- stress monitoring
- hrv
- signal
- feature extraction
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000003340 mental effect Effects 0.000 title claims abstract description 52
- 238000000034 method Methods 0.000 title claims abstract description 48
- 238000012544 monitoring process Methods 0.000 title claims abstract description 37
- 238000000605 extraction Methods 0.000 claims abstract description 30
- 238000004458 analytical method Methods 0.000 claims abstract description 28
- 238000012545 processing Methods 0.000 claims abstract description 11
- 238000012216 screening Methods 0.000 claims abstract description 9
- 238000004590 computer program Methods 0.000 claims description 13
- 238000010586 diagram Methods 0.000 claims description 11
- 238000005070 sampling Methods 0.000 claims description 8
- 230000001815 facial effect Effects 0.000 claims description 4
- 230000008859 change Effects 0.000 description 8
- 230000008901 benefit Effects 0.000 description 7
- 238000001514 detection method Methods 0.000 description 6
- 230000008569 process Effects 0.000 description 6
- 238000004364 calculation method Methods 0.000 description 5
- 230000000694 effects Effects 0.000 description 5
- 230000000747 cardiac effect Effects 0.000 description 4
- 230000007423 decrease Effects 0.000 description 4
- 230000002889 sympathetic effect Effects 0.000 description 4
- 238000003066 decision tree Methods 0.000 description 3
- 210000005037 parasympathetic nerve Anatomy 0.000 description 3
- 230000009467 reduction Effects 0.000 description 3
- 208000019901 Anxiety disease Diseases 0.000 description 2
- 208000013738 Sleep Initiation and Maintenance disease Diseases 0.000 description 2
- 210000003403 autonomic nervous system Anatomy 0.000 description 2
- 230000033228 biological regulation Effects 0.000 description 2
- 239000008280 blood Substances 0.000 description 2
- 210000004369 blood Anatomy 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 2
- 230000001684 chronic effect Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- JYGXADMDTFJGBT-VWUMJDOOSA-N hydrocortisone Chemical compound O=C1CC[C@]2(C)[C@H]3[C@@H](O)C[C@](C)([C@@](CC4)(O)C(=O)CO)[C@@H]4[C@@H]3CCC2=C1 JYGXADMDTFJGBT-VWUMJDOOSA-N 0.000 description 2
- 206010022437 insomnia Diseases 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 210000005036 nerve Anatomy 0.000 description 2
- 238000013186 photoplethysmography Methods 0.000 description 2
- 238000001228 spectrum Methods 0.000 description 2
- 238000007619 statistical method Methods 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 239000013598 vector Substances 0.000 description 2
- 206010003840 Autonomic nervous system imbalance Diseases 0.000 description 1
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 230000036506 anxiety Effects 0.000 description 1
- 230000002567 autonomic effect Effects 0.000 description 1
- 210000000467 autonomic pathway Anatomy 0.000 description 1
- 230000035581 baroreflex Effects 0.000 description 1
- 230000035559 beat frequency Effects 0.000 description 1
- 230000017531 blood circulation Effects 0.000 description 1
- 210000001715 carotid artery Anatomy 0.000 description 1
- 150000003943 catecholamines Chemical class 0.000 description 1
- 230000001149 cognitive effect Effects 0.000 description 1
- 230000001276 controlling effect Effects 0.000 description 1
- 239000002537 cosmetic Substances 0.000 description 1
- 238000007418 data mining Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000008451 emotion Effects 0.000 description 1
- 230000006397 emotional response Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 210000001061 forehead Anatomy 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 210000003128 head Anatomy 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 230000001121 heart beat frequency Effects 0.000 description 1
- 208000019622 heart disease Diseases 0.000 description 1
- 229960000890 hydrocortisone Drugs 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 230000003902 lesion Effects 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 230000004630 mental health Effects 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 230000001537 neural effect Effects 0.000 description 1
- 238000007427 paired t-test Methods 0.000 description 1
- 230000001734 parasympathetic effect Effects 0.000 description 1
- 210000001002 parasympathetic nervous system Anatomy 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 230000004962 physiological condition Effects 0.000 description 1
- 238000004321 preservation Methods 0.000 description 1
- 230000010349 pulsation Effects 0.000 description 1
- 230000001105 regulatory effect Effects 0.000 description 1
- 230000000241 respiratory effect Effects 0.000 description 1
- 230000029058 respiratory gaseous exchange Effects 0.000 description 1
- 230000035807 sensation Effects 0.000 description 1
- 210000001013 sinoatrial node Anatomy 0.000 description 1
- 208000011580 syndromic disease Diseases 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 230000001515 vagal effect Effects 0.000 description 1
- 230000002861 ventricular Effects 0.000 description 1
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
- A61B5/165—Evaluating the state of mind, e.g. depression, anxiety
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/0059—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence
- A61B5/0077—Devices for viewing the surface of the body, e.g. camera, magnifying lens
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7203—Signal processing specially adapted for physiological signals or for diagnostic purposes for noise prevention, reduction or removal
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
- A61B5/7267—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems involving training the classification device
Abstract
The application discloses a mental stress monitoring system, a mental stress monitoring method, a mental stress monitoring medium and mental stress monitoring equipment. The mental stress monitoring system may include: the video acquisition unit acquires rPPG signals of the face area of the human body; the image processing unit is used for extracting RGB video frame pictures from the collected face video and acquiring R, G, B single-channel images; the signal processing unit is used for denoising the acquired pixels of the R, G, B single-channel image; the HRV feature extraction unit is used for carrying out HRV feature extraction on the denoised rPPG signal; the HRV characteristic analysis unit is used for screening the HRV characteristics by adopting an SPSS method; and the identification unit is used for identifying the sample data and matching the corresponding mental stress level. Exemplary methods, media, and devices for mental stress monitoring are also disclosed.
Description
Technical Field
The application relates to the technical field of medical equipment, in particular to a mental stress monitoring system, a mental stress monitoring method, a mental stress monitoring medium and mental stress monitoring equipment.
Background
Recently, remote photoplethysmography (remote photoplethysmography, abbreviated rpg) has received increasing attention. The rpg technique is a technique of capturing a skin color cycle change caused by a cardiac cycle by a sensor such as a camera; the specific meaning is that a technique for extracting heart rate variability features is performed by acquiring periodic fine skin color changes invisible to the naked eye caused by a cardiac cycle through a camera. The rPPG technology has the characteristics of non-invasive performance, rapidness and the like, shows unique technical advantages in the biomedical field, and brings better detection experience and higher efficiency to patients. With mental health problems, it has become one of the main causes of "disability" (disability) of individuals, and in combination with the development trends of "global", "long-term" (chronic) and "epidemic"), the application of rpg technology to mental stress detection will help the stress assessment method to obtain a very wide application prospect in basic medical systems, even in home health systems.
In view of the current state of the art, means for assessing psychological stress have been developed in numerous professional fields, and the psychological stress assessment can be broadly divided into two types: one is based on questionnaires and psychological measures in the form of self-reports, with psychological sensations measured in terms of cognitive assessment and emotional response, typically in the form of self-reporting questionnaires or scales. Although this method is convenient and quick and easy to manage, the test questionnaire form cannot be measured twice in a short period of time because of memory residues in the human body. The other is psychological pressure detection based on physiological signals, and pressure evaluation can be realized by measuring cortisol, plasma catecholamine, electrocardiosignals, electroencephalogram signals and electromyographic signals, but the preservation cost of detection samples is higher, and the detection samples are invasive, and special acquisition instruments and professionals are needed, so that stress reaction of organisms can be caused to a great extent, the measurement structure is influenced, and the Shu Huoxing and convenience of the acquisition cannot be guaranteed. In view of this, application of the rpg technique to the measurement of mental stress has gradually become an optimal solution to the above-described problem.
Disclosure of Invention
The primary object of the present application is to provide a mental stress monitoring system, method, medium and apparatus that makes mental stress monitoring more convenient and efficient.
To achieve the above object, in a first aspect, the present application provides a mental stress monitoring system, the system comprising: the video acquisition unit acquires rPPG signals of the face area of the human body; the image processing unit is used for extracting RGB video frame pictures from the collected face video and acquiring R, G, B single-channel images; the signal processing unit is used for denoising the acquired pixels of the R, G, B single-channel image; the HRV feature extraction unit is used for carrying out HRV feature extraction on the denoised rPPG signal; the HRV characteristic analysis unit is used for screening the HRV characteristics by adopting an SPSS method; and the identification unit is used for identifying the sample data and matching the corresponding mental stress level.
In some embodiments, the human face region comprises: frontal area, left face area, right face area and chin area.
In some embodiments, HRV feature extraction is performed on the denoised rpg signal, further comprising: nonlinear characteristic parameters are extracted based on a Poincare scatter diagram analysis method.
In some embodiments, HRV feature extraction is performed on the denoised rpg signal, further comprising: performing cubic spline interpolation on the rPPG signal, up-sampling to 240Hz, extracting peak points, obtaining RR interval sequences by calculating time intervals among the peak points, removing outliers of the RR interval sequences, and extracting characteristic parameters of a time domain; and carrying out frequency domain analysis on the rPPG signal by adopting a Welch power spectrogram, and extracting characteristic parameters of a frequency domain.
In a second aspect, the present application also provides a method of mental stress monitoring, comprising: acquiring rPPG signals of a human face area; extracting RGB video frame pictures from the collected face video to obtain R, G, B single-channel images; denoising the obtained pixels of the R, G, B single-channel image; carrying out HRV feature extraction on the denoised rPPG signal; screening the HRV features by adopting an SPSS method; the sample data is identified, matching the corresponding mental stress level.
In some embodiments, the human face region comprises: frontal area, left face area, right face area and chin area.
In some embodiments, HRV feature extraction is performed on the denoised rpg signal, further comprising: nonlinear characteristic parameters are extracted based on a Poincare scatter diagram analysis method.
In some embodiments, HRV feature extraction is performed on the denoised rpg signal, further comprising: performing cubic spline interpolation on the rPPG signal, up-sampling to 240Hz, extracting peak points, obtaining RR interval sequences by calculating time intervals among the peak points, removing outliers of the RR interval sequences, and extracting characteristic parameters of a time domain; and carrying out frequency domain analysis on the rPPG signal by adopting a Welch power spectrogram, and extracting characteristic parameters of a frequency domain.
In a third aspect, the present application also provides a computer readable storage medium storing a computer program, characterized in that the computer program when executed by a processor implements the steps of the method of mental stress monitoring described above.
In a fourth aspect, the present application further provides an electronic device, including: a memory for storing a computer program product; a processor for executing the computer program product stored in the memory, and the computer program product when executed, performing the steps of the method of mental stress monitoring described above.
Compared with the prior art, the application has the following advantages:
the mental stress monitoring system, the mental stress monitoring method, the mental stress monitoring medium and the mental stress monitoring equipment are convenient and efficient to monitor mental stress.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this application, are included to provide a further understanding of the application and to provide a further understanding of the application with regard to the other features, objects and advantages of the application. The drawings of the illustrative embodiments of the present application and their descriptions are for the purpose of illustrating the present application and are not to be construed as unduly limiting the present application. In the drawings:
FIG. 1 shows a schematic diagram of a mental stress monitoring system according to an exemplary embodiment of the present application;
FIG. 2 illustrates a flow chart of a method of mental stress monitoring according to an exemplary embodiment of the present application;
fig. 3 shows a schematic diagram of an apparatus of an electronic device according to an exemplary embodiment of the present application.
Detailed Description
In order to make the present application solution better understood by those skilled in the art, the following description will be made in detail and with reference to the accompanying drawings in the embodiments of the present application, it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments herein without making any inventive effort, shall fall within the scope of the present application.
It should be noted that the terms "first," "second," and the like in the description and claims of the present application and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate in order to describe the embodiments of the present application described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
It should be noted that, in the case of no conflict, the embodiments and features in the embodiments may be combined with each other. The present application will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.
Fig. 1 shows a schematic diagram of a mental stress monitoring system according to an exemplary embodiment of the present application. Wherein the mental stress monitoring system 100 comprises: a video acquisition unit 102, an image processing unit 104, a signal processing unit 106, an HRV feature extraction unit 108, an HRV feature analysis unit 110, and an identification unit 112.
The video acquisition unit 102 is used for acquiring an rPPG signal of a human face area; alternatively, the rpg signal may be taken as the original data set.
In some embodiments, the facial region is selected for video capture because the human face is rich in capillaries, while near the carotid artery, resulting in higher blood signal strength, while taking into account the convenience of video acquisition. For example, a face may be divided into four regions: frontal area, left face area, right face area and chin area. The video acquisition requires that under the environment with sufficient light, dense makeup is avoided as much as possible, the use of cosmetic products is avoided, the forehead area, the left face area, the right face area and the chin area of the face are ensured, the brightness and the uniformity of the light are ensured, the fluctuation of the light source is prevented from influencing the faint skin color change of the face, the normal respiration is kept in the acquisition process, the movements of the head and the body are reduced as much as possible, and the large-amplitude expression and the like are avoided.
In some embodiments, rpg signal information of a human face region may be stored in xml files, resulting in a folder composed of a plurality of xml files, where each xml file has the same name as its corresponding human face image, forming an original dataset composed of xml files.
The image processing unit 104 extracts RGB video frame pictures from the acquired face video, and acquires R, G, B single-channel images. In order to obtain accurate results, the image processing unit needs to perform face detection on each frame of RGB graphics.
In some embodiments, in order to reduce interference of background factors, redundant shooting backgrounds need to be removed, and face RGB pictures are obtained. Based on the vascularity and blood flow condition of the face, the region of interest selects a part with dense vascularity and sufficient blood, a R, G, B single-channel image is obtained, and the change of the light of the face caused by heart pulsation is reflected by calculating the pixel mean value drawing curve of each ROI picture in three single channels, so that three facial pixel change curves are obtained.
And the signal processing unit 106 is used for denoising the acquired pixels of the R, G, B single-channel image.
In some embodiments, after obtaining a single-channel pixel ray change curve of the face, the effective heart beat information is easily covered by noise, interfering due to the large amount of noise present in the signal. The source signal contained in the signal is separated using the fastca algorithm, and the source signal having the highest pearson coefficient of the green channel is extracted as the rpg signal associated with the heart beat. In order to further remove high-frequency noise and baseline drift, a wavelet transformation is adopted to reconstruct signals in a normal human heartbeat frequency range, effective pulse wave frequency in the signals is obtained through fast Fourier transformation, and a narrow band-pass filter is arranged to obtain pure rPPG signals.
HRV feature extraction section 108 performs HRV feature extraction on the denoised rpg signal.
In some embodiments, performing cubic spline interpolation on the rPPG signal, up-sampling to 240Hz, performing peak point extraction, obtaining an RR interval sequence (the RR interval represents ventricular beat frequency) by calculating the time interval between the peak points, removing outliers of the RR interval sequence, and extracting characteristic parameters of a time domain; and carrying out frequency domain analysis on the rPPG signal by adopting a Welch power spectrogram, and extracting characteristic parameters of a frequency domain. Briefly stated, HRV is also referred to as heart rate variability, which refers to the small difference in time between successive normal (sinus) cardiac cycles. Under physiological conditions, HRV is produced primarily due to the modulation of cardiac sinus node autonomic activity by sympathetic and vagal nerves, neural centers, baroreflex, and respiratory activity, such that there is typically a few tens of milliseconds of difference in the interval between beats of the heart.
In some embodiments, the nonlinear feature parameters may be extracted based on poincare scattergram analysis. By the heart rate variability characteristic calculation method, 17 characteristic parameters are collected, and 9 time domain characteristic parameters are respectively: HR (mean heart rate of samples over time), SDNN (standard deviation of RR intervals, unit ms), PNN50 (percentage of the total number of RR intervals greater than 50 ms), PNN20 (percentage of the total number of RR intervals greater than 20 ms), RMSSD (square root of the mean of the sum of differences between adjacent RR intervals, unit ms), SDSD (standard deviation of differences between adjacent RRs, unit ms), CVSD (coefficient of variation of continuous differences), CVnni (coefficient of variation), std_hr (standard deviation of heart rate, unit bmp). The frequency domain characteristic parameters are 4, namely: LF (Low band Power, unit)Reference range 0.04-0.15 Hz), HF (high-band power, unit ∈ ->Reference range 0.15-0.4 Hz), TP (total signal power)Rate, units->Reference range 0 to 0.4 Hz), LF/HF (low-high frequency power ratio). The nonlinear characteristic parameters are respectively SD1 (standard deviation of Poincare plot on straight line perpendicular to mark line, unit ms), SD2 (standard deviation along mark line, unit ms), SD1/SD2 (ratio of SD2 and SD 1), and SampEn (sample entropy).
The following is an HRV feature extraction and calculation method:
1. time domain feature analysis:
SDNN represents the degree to which heart rate variability deviates from an average. Wherein the method comprises the steps ofIs the average of RR intervals for N heartbeats.Is the i-th RR interval.
PNN50 may reflect parasympathetic activity. PNN20 is identical to PNN 50.
RMSSD is used to measure the effects of parasympathetic nervous system on heart rate regulation.
SDSD more finely represents the overall change in HRV. Wherein the method comprises the steps of，/>。
CVSD represents the coefficient of variation of the continuous difference.
CVnni may be used to measure the HRV size.
std_hr represents the degree to which the heart rate deviates from the average. Wherein the method comprises the steps ofRepresenting the ith advanced heart rate.
2. Frequency domain feature analysis: the frequency domain analysis may reflect the respective activities of the sympathetic and parasympathetic nerves, as well as the state of balance of the autonomic nervous system. The formula is as follows:
in the method, in the process of the invention,-power spectrum of RR intervals; LF/HF-the ratio of the low frequency to high frequency power spectrum;
3. nonlinear analysis: using poincare scattergram analysis, the current heart cycle is affected by its previous time, and in order to express the correlation between the two, an RR interval sequence is used, the coordinates of two adjacent intervals as a factor of poincare points, i.e. the current interval is the abscissa and the next interval is the ordinate, and each point is plotted in the figure, thereby obtaining a scattergram. The scatter plot may draw an ellipse whose center is the average of the heart beat intervals, after which the major and minor axes of the ellipse may be found to be SD2 and SD1, respectively. The calculation is as follows:
the sample entropy is calculated as follows:
the time series for a known N data composition is noted asM-dimensional vector is formed according to serial numbers>:
Representing m consecutive x values starting from i. Calculating vectorsAnd->Distance of (2) is denoted as->:
After which the distance is countedThe number of (2) is recorded as->Wherein->To set a threshold. From this the probability is calculated:
calculating a mean value:
adding the dimension to m+1, and calculating the probability in the above stepsMean->. Sample entropy is defined as:
if N is a finite value, it can be noted that:
the HRV feature analysis unit 110 screens the HRV features using the SPSS method.
In some embodiments, the sample data with pressure tags is subjected to feature screening of the 17 sets of extracted features. The SPSS (SPSS is totally called a social science statistical software package, is a general term of software products and related services for statistical analysis operation, data mining, predictive analysis and decision support tasks) is adopted for data statistical analysis, and 14 features with significant differences are screened out through paired t-test to be used as related indexes of mental stress analysis. The three characteristic parameters were discarded because of insignificant differences among std_hr, SD1/SD2, and sampenn. When the device is in a pressure state for a long time, the autonomic nervous system can generate corresponding change, and the change is further shown in the characteristic value of the HRV. Pressure results in a decrease in HRV, which is primarily manifested as a decrease in the various time domain indicators. For example, the time domain index SDNN can be used for evaluating the state of the human body pressure resistance, and the good pressure resistance can often enable the human body to self-regulate as soon as possible when bearing pressure, so as to avoid the generation of excessive negative emotion; RMSSD may be used to measure abnormal lesions or syndrome of the human heart; SDNN and RMSSD values below 10ms at the same time often represent an increased incidence of heart disease. If the frequency domain characteristic TP is obviously reduced in a chronic pressure state, the reduction of the total energy represents the reduction of the autonomic nerve regulation capacity; LF/HF represents the overall balance between sympathetic and parasympathetic nerves, which are elevated when exposed to external pressure, and too high and too low are both states of autonomic imbalance, with sympathetic nerves responsible for regulating emergencies and parasympathetic nerves responsible for controlling various body functions and normal activities.
The identification unit 112 identifies the sample data, matching the corresponding pressure level.
In some embodiments, the mental stress identification model is built by training the HRV feature analyzed data using a decision tree algorithm. The decision tree is a tree model and is used for solving the problem of which value of which dimension is divided, so that the lower the information entropy is after division, the lower the information entropy is, and the stronger the classification certainty is.
In some embodiments, the HRV feature analyzed data is looped through from a parent node according to certain rules, finding the currently decisive feature, and splitting to child nodes according to the value of the feature. The child node continues to split as a new parent node according to the above rules until a condition is met or cannot split. So selecting proper root node is key to splitting, judging standard is to calculate the non-purity of sample set, and the calculation method uses information gain method. As the nodes of the decision tree divide, their uncertainty decreases, and as classification progresses, more and more information is known that the overall entropy decreases. There are n categories in the sample set S, where n corresponds to mental stress classified as anxiety, depression, insomnia, calm, happiness. Then the information entropy is calculated as:
in the method, in the process of the invention,-the duty cycle of the different classes of samples.
When feature F is selected as a splitting condition, this information entropy is as follows:
in the method, in the process of the invention,sample S is divided into k parts.
The information gain means the reduction of the information entropy under the action of the characteristic attribute F, and the calculation formula of the information gain is as follows:
in some embodiments, the sample data to be tested is identified using a mental stress identification model, identifying its corresponding stress level. After the sample data to be tested is processed, standard data suitable for model input is obtained, and the data can be classified by the trained recognition model. Such as: 1) The time domain indexes SDNN and RMSSD are obviously reduced, the HR is increased, and the frequency domain indexes LF/HF and LF are increased under the anxiety state; 2) The time domain index SDNN and RMSSD are obviously reduced in a depression state, the frequency domain index LF/HF is reduced to some extent, and the like; 3) Time domain indexes SDNN, RMSSD and the like are reduced in the insomnia state; 4) During calm, the indexes have no obvious deviation; 5) In a happy state, the time domain index SDNN, RMSSD, PNN rises, the frequency domain index HF rises, the nonlinear indexes SD1 and SD2 rise, and the like.
Fig. 2 illustrates a method 200 of mental stress monitoring, comprising: step 202, acquiring rPPG signals of a human face region; step 204, extracting RGB video frame pictures from the collected face video to obtain R, G, B single-channel images; step 206, denoising the obtained pixels of the R, G, B single-channel image; step 208, performing HRV feature extraction on the denoised rpg signal; step 210, screening the HRV features by adopting an SPSS method; at step 212, the sample data is identified, matching the corresponding mental stress level.
In some embodiments, the human face region comprises: frontal area, left face area, right face area and chin area.
In some embodiments, HRV feature extraction is performed on the denoised rpg signal, further comprising: nonlinear characteristic parameters are extracted based on a Poincare scatter diagram analysis method.
In some embodiments, HRV feature extraction is performed on the denoised rpg signal, further comprising: performing cubic spline interpolation on the rPPG signal, up-sampling to 240Hz, extracting peak points, obtaining RR interval sequences by calculating time intervals among the peak points, removing outliers of the RR interval sequences, and extracting characteristic parameters of a time domain; and carrying out frequency domain analysis on the rPPG signal by adopting a Welch power spectrogram, and extracting characteristic parameters of a frequency domain.
Fig. 3 shows a schematic diagram of an electronic device 300 according to an exemplary embodiment of the present application. The electronic device 300 may include: at least one processor 302; and at least one memory 304 including computer program code, the at least one memory 304 and the computer program code 306 configured to, with the at least one processor 302, cause the electronic device 300 to perform: acquiring rPPG signals of a human face area; extracting RGB video frame pictures from the collected face video to obtain R, G, B single-channel images; denoising the obtained pixels of the R, G, B single-channel image; carrying out HRV feature extraction on the denoised rPPG signal; screening the HRV features by adopting an SPSS method; the sample data is identified, matching the corresponding mental stress level.
In some embodiments, the human face region comprises: frontal area, left face area, right face area and chin area.
In some embodiments, HRV feature extraction is performed on the denoised rpg signal, further comprising: nonlinear characteristic parameters are extracted based on a Poincare scatter diagram analysis method.
In some embodiments, HRV feature extraction is performed on the denoised rpg signal, further comprising: performing cubic spline interpolation on the rPPG signal, up-sampling to 240Hz, extracting peak points, obtaining RR interval sequences by calculating time intervals among the peak points, removing outliers of the RR interval sequences, and extracting characteristic parameters of a time domain; and carrying out frequency domain analysis on the rPPG signal by adopting a Welch power spectrogram, and extracting characteristic parameters of a frequency domain.
The application also discloses a computer readable storage medium storing a computer program, characterized in that the computer program is realized when being executed by a processor: acquiring rPPG signals of a human face area; extracting RGB video frame pictures from the collected face video to obtain R, G, B single-channel images; denoising the obtained pixels of the R, G, B single-channel image; carrying out HRV feature extraction on the denoised rPPG signal; screening the HRV features by adopting an SPSS method; the sample data is identified, matching the corresponding mental stress level.
In some embodiments, the human face region comprises: frontal area, left face area, right face area and chin area.
In some embodiments, HRV feature extraction is performed on the denoised rpg signal, further comprising: nonlinear characteristic parameters are extracted based on a Poincare scatter diagram analysis method.
In some embodiments, HRV feature extraction is performed on the denoised rpg signal, further comprising: performing cubic spline interpolation on the rPPG signal, up-sampling to 240Hz, extracting peak points, obtaining RR interval sequences by calculating time intervals among the peak points, removing outliers of the RR interval sequences, and extracting characteristic parameters of a time domain; and carrying out frequency domain analysis on the rPPG signal by adopting a Welch power spectrogram, and extracting characteristic parameters of a frequency domain.
The basic principles of the present application have been described above in connection with specific embodiments, however, it should be noted that the advantages, benefits, effects, etc. mentioned in the present application are merely examples and not limiting, and these advantages, benefits, effects, etc. are not to be considered as necessarily possessed by the various embodiments of the present application. Furthermore, the specific details disclosed herein are for purposes of illustration and understanding only, and are not intended to be limiting, as the application is not intended to be limited to the details disclosed herein as such.
The foregoing description is only of the preferred embodiments of the present application and is not intended to limit the same, but rather, various modifications and variations may be made by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.
Claims (10)
1. A mental stress monitoring system, the system comprising:
the video acquisition unit acquires rPPG signals of the face area of the human body;
the image processing unit is used for extracting RGB video frame pictures from the collected face video and acquiring R, G, B single-channel images;
the signal processing unit is used for denoising the acquired pixels of the R, G, B single-channel image;
the HRV feature extraction unit is used for carrying out HRV feature extraction on the denoised rPPG signal;
the HRV characteristic analysis unit is used for screening the HRV characteristics by adopting an SPSS method;
and the identification unit is used for identifying the sample data and matching the corresponding mental stress level.
2. The mental stress monitoring system of claim 1, wherein the human facial area comprises: frontal area, left face area, right face area and chin area.
3. The mental stress monitoring system of claim 1, wherein HRV feature extraction is performed on the denoised rpg signal, further comprising: nonlinear characteristic parameters are extracted based on a Poincare scatter diagram analysis method.
4. The mental stress monitoring system of claim 1, wherein HRV feature extraction is performed on the denoised rpg signal, further comprising:
performing cubic spline interpolation on the rPPG signal, up-sampling to 240Hz, extracting peak points, obtaining RR interval sequences by calculating time intervals among the peak points, removing outliers of the RR interval sequences, and extracting characteristic parameters of a time domain;
and carrying out frequency domain analysis on the rPPG signal by adopting a Welch power spectrogram, and extracting characteristic parameters of a frequency domain.
5. A method of mental stress monitoring, comprising:
acquiring rPPG signals of a human face area;
extracting RGB video frame pictures from the collected face video to obtain R, G, B single-channel images;
denoising the obtained pixels of the R, G, B single-channel image;
carrying out HRV feature extraction on the denoised rPPG signal;
screening the HRV features by adopting an SPSS method;
the sample data is identified, matching the corresponding mental stress level.
6. The method of mental stress monitoring according to claim 5, wherein the human facial area comprises: frontal area, left face area, right face area and chin area.
7. The method of mental stress monitoring as recited in claim 5, wherein,
HRV feature extraction is performed on the denoised rpg signal, further comprising: nonlinear characteristic parameters are extracted based on a Poincare scatter diagram analysis method.
8. The method of mental stress monitoring according to claim 5, wherein HRV feature extraction is performed on the denoised rpg signal, further comprising:
performing cubic spline interpolation on the rPPG signal, up-sampling to 240Hz, extracting peak points, obtaining RR interval sequences by calculating time intervals among the peak points, removing outliers of the RR interval sequences, and extracting characteristic parameters of a time domain;
and carrying out frequency domain analysis on the rPPG signal by adopting a Welch power spectrogram, and extracting characteristic parameters of a frequency domain.
9. A computer readable storage medium storing a computer program, characterized in that the computer program when executed by a processor realizes the steps of the method of mental stress monitoring according to any of claims 5 to 8.
10. An electronic device, comprising: a memory for storing a computer program product; a processor for executing a computer program product stored in said memory, which when executed, performs the steps of the method of mental stress monitoring as claimed in any of the preceding claims 5 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311580851.3A CN117281516A (en) | 2023-11-24 | 2023-11-24 | Mental stress monitoring system, method, medium and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311580851.3A CN117281516A (en) | 2023-11-24 | 2023-11-24 | Mental stress monitoring system, method, medium and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117281516A true CN117281516A (en) | 2023-12-26 |
Family
ID=89258966
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311580851.3A Pending CN117281516A (en) | 2023-11-24 | 2023-11-24 | Mental stress monitoring system, method, medium and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117281516A (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111714144A (en) * | 2020-07-24 | 2020-09-29 | 长春理工大学 | Mental stress analysis method based on video non-contact measurement |
| CN112294282A (en) * | 2019-08-01 | 2021-02-02 | 天津工业大学 | Self-calibration method of emotion detection device based on RPPG |
| CN114913573A (en) * | 2022-04-19 | 2022-08-16 | 深圳市彼岸心智科技有限公司 | Expression recognition method and device, storage medium and equipment |
| CN115553777A (en) * | 2022-11-02 | 2023-01-03 | 济南大学 | Non-contact mental stress detection method and system |
-
2023
- 2023-11-24 CN CN202311580851.3A patent/CN117281516A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112294282A (en) * | 2019-08-01 | 2021-02-02 | 天津工业大学 | Self-calibration method of emotion detection device based on RPPG |
| CN111714144A (en) * | 2020-07-24 | 2020-09-29 | 长春理工大学 | Mental stress analysis method based on video non-contact measurement |
| CN114913573A (en) * | 2022-04-19 | 2022-08-16 | 深圳市彼岸心智科技有限公司 | Expression recognition method and device, storage medium and equipment |
| CN115553777A (en) * | 2022-11-02 | 2023-01-03 | 济南大学 | Non-contact mental stress detection method and system |
Non-Patent Citations (1)
| Title |
|---|
| 郑国华等: "健康状况与风险评估", 科学技术文献出版社, pages: 255 - 263 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Fernandes et al. | A novel nonintrusive decision support approach for heart rate measurement | |
| Bulagang et al. | A review of recent approaches for emotion classification using electrocardiography and electrodermography signals | |
| CN109843163B (en) | Method and system for marking sleep state | |
| US9872652B2 (en) | Method and apparatus for heart rate monitoring using an electrocardiogram sensor | |
| Karthikeyan et al. | Detection of human stress using short-term ECG and HRV signals | |
| Colomer Granero et al. | A comparison of physiological signal analysis techniques and classifiers for automatic emotional evaluation of audiovisual contents | |
| US7547279B2 (en) | System and method for recognizing user's emotional state using short-time monitoring of physiological signals | |
| KR101738278B1 (en) | Emotion recognition method based on image | |
| Tabei et al. | A novel personalized motion and noise artifact (MNA) detection method for smartphone photoplethysmograph (PPG) signals | |
| US20240081705A1 (en) | Non-contact fatigue detection system and method based on rppg | |
| CN114732418A (en) | High-frequency QRS waveform curve analysis method and device, computer equipment and storage medium | |
| Poddar et al. | Heart rate variability based classification of normal and hypertension cases by linear-nonlinear method | |
| Mitsuhashi et al. | Video-based stress level measurement using imaging photoplethysmography | |
| Gupta et al. | A support system for automatic classification of hypertension using BCG signals | |
| Khan et al. | Computer-aided diagnosis system for cardiac disorders using variational mode decomposition and novel cepstral quinary patterns | |
| Akbulut | Evaluating the effects of the autonomic nervous system and sympathetic activity on emotional states | |
| Athaya et al. | An efficient fingertip photoplethysmographic signal artifact detection method: A machine learning approach | |
| Tiwari et al. | Movement artifact-robust mental workload assessment during physical activity using multi-sensor fusion | |
| Wang et al. | A Novel Rapid Assessment of Mental Stress by Using PPG Signals Based on Deep Learning | |
| CN117281516A (en) | Mental stress monitoring system, method, medium and device | |
| JP7332723B2 (en) | System for detecting QRS complexes in an electrocardiogram (ECG) signal | |
| Corino et al. | A simple model to detect atrial fibrillation via visual imaging | |
| Bassiouni et al. | Combination of ECG and PPG Signals for Healthcare Applications: A Survey | |
| Rezaei et al. | Evaluating valence level of pictures stimuli in heart rate variability response | |
| Shuzan et al. | Machine-learning-based emotion recognition in arousal–valence space using photoplethysmogram signals |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |