CN116434959A

CN116434959A - Human body fatigue monitoring system based on mobile phone camera

Info

Publication number: CN116434959A
Application number: CN202310634011.4A
Authority: CN
Inventors: 张昱昊; 姚宇恒; 章杰; 徐晓燕; 王汉语; 陈星星
Original assignee: Wannan Medical College
Current assignee: Wannan Medical College
Priority date: 2023-05-31
Filing date: 2023-05-31
Publication date: 2023-07-14

Abstract

The invention discloses a human body fatigue monitoring system based on a mobile phone camera, which uses the mobile phone camera to collect pulse wave signals of a human body, parameters obtained after data processing are similar to heart rate variability parameters, then frequency domain analysis and time domain analysis are carried out, a human body fatigue evaluation model based on HRV is established by using an HMM theory, and experimental results show that the model has higher accuracy and can be used for evaluating the fatigue state of the human body so as to judge whether the human body is in the fatigue state.

Description

Human body fatigue monitoring system based on mobile phone camera

Technical Field

The invention belongs to the technical field of human health parameter measurement, and particularly relates to a human fatigue monitoring system based on a mobile phone camera.

Background

The pressure faced by the daily work and life of people in the current society is increasing, and the sudden death event caused by frequent overtime is frequent. Studies have shown that an important predisposition for most sudden death cases is mental fatigue due to excessive stress. In order to detect the mental fatigue of a human body in real time and give an alarm when the human body is in a fatigue state, a set of human body fatigue monitoring system is necessary to be designed. By the human body fatigue detection system, the human body fatigue state can be monitored, and an alarm is given when the human body is in the fatigue state, so that people are reminded to rest in time.

Foreign research on human body fatigue began in the 90 s of the 20 th century. In 1994, fatigue was defined by Brown as a physical response when a person was not willing to do his current work, and also reduced the person's acuity response when the person was in a state of fatigue. Research shows that fatigue has become an important cause of safety accidents or driving accidents when working in dangerous environments. It is counted that in some dangerous environments, about 10% -30% of accidents are caused by mental fatigue of the human body, which directly reflects the importance of mental fatigue detection of the human body. The american physicist Edward proposes that fatigue is the result of the combined action of various influencing factors, namely 'variation theory', and through continuous physical or mental consumption, the physical and concentration of people can be reduced, the reaction can become slow, the work efficiency can be reduced, the people are more difficult to concentrate on the state of insufficient concentration, and in addition, bored emotion can be generated on the basis, and the state is called as a human fatigue state. The theory concludes that: like people's daily cold, human mental fatigue is also a protective response of the human body, and its effect is to prevent people from excessively consuming their own energy. Chua et al studied HRV heart rate variability index and psychomotor alertness task of ECG electrocardiosignal signals acquired every two hours during 40h of sleep deprivation of subjects, and found that HRV index can be used for monitoring fatigue and can be used as index for predicting drowsiness of human body [3]. The heart rate variability is used as detection data of mental fatigue, so that the heart rate variability has a good detection effect, and colleagues have the advantages of noninvasive acquisition, high sensitivity and the like. Al-Libawy and the like obtain heart rate variability of a human body through a chest belt type heart monitor and a wrist watch capable of measuring heart rate and analyze the current fatigue condition of the human body. Tsai et al analyze the fatigue of construction workers by combining HRV and electroencephalogram signals, and expect to reduce safety accidents caused by fatigue. Domestic Li Mingai and the like prove that the brain electrical signals can be used for judging fatigue driving and are common means for evaluating the fatigue condition of a human body. Li Yanjun et al designed both reading and computing experiments and found that with increasing fatigue during reading or computing, heart rate decreased with increasing HRV. Guo Weizhen and the like extract HRV from electrocardiosignals, and perform time-frequency domain analysis on the HRV to quantitatively evaluate human fatigue. Zhou Rongxin et al have experimentally concluded that: the effect of analyzing the fatigue degree by using the electromyographic signals is inferior to that of the HRV signals, and meanwhile, the HRV signals can be comprehensively used with the electromyographic signals to judge the fatigue degree of the human body, so that the accuracy is higher.

In carrying out the invention, the inventors have found that the prior art has at least the following problems:

at present, most of domestic and foreign researches detect fatigue conditions by analyzing physiological signals. These biological signals mainly include: electrocardiograph (ECG), electro-oculogram (EOG), electroencephalogram (EEG), electromyogram (EMG), pulse wave signal (PPG), and the like. When the human body is in a fatigue state, the human body is accompanied by changes of some physiological characteristics, such as: reaction retardation, distraction, acceleration of heart beat, pulse enhancement, etc. These physiological changes are manifested as described aboveBy detecting changes in these signals, it is possible to determine whether or not the human body is in a tired state. However, the electroencephalogram signal acquisition equipment is too expensive; the electrooculogram signal acquisition is too complex and has low real-time performance. The signals are complex to operate during acquisition, which causes inconvenience to users ^[11] . The research at home and abroad is integrated, the existing human body fatigue detection system still has the defects of complex signal acquisition, inconvenient use and high price, and is not suitable for general popularization. And because of the limitation of signal acquisition, the normal operation and the service life of the equipment are seriously affected. It is difficult to meet the real-time requirements.

At present, fatigue conditions common in daily life are mostly caused by long-time driving or long-term mental labor. Most of major traffic accidents occur due to driver fatigue driving. By analyzing the results of the investigation of the related departments, it is found that adult drivers who have experienced sleep or drowsiness in the driving process occupy more than half of the total population, and the interviewees who have experienced fatigue driving are acknowledged to occupy a larger proportion; when mental fatigue occurs on a mental worker, it often results in a decrease in work efficiency with an increase in error rate. When mental fatigue occurs on the body of a worker working in dangerous situations such as high altitude, high pressure and the like, the occurrence probability of dangerous and even fatal situations such as falling, electric shock and the like can be greatly improved, and the safety hidden trouble of incapability of light sight is provided.

The detection of the mental fatigue degree of the human body is helpful for the worker to judge the mental condition of the worker, and the negative effects of work efficiency reduction and the like caused by mental fatigue are avoided to a certain extent. However, since most of the currently used fatigue monitoring systems have problems of complicated use, high cost, difficulty in real-time detection, etc., it is necessary to design a portable system capable of detecting mental fatigue state of a human body in real time in order to solve the problems and realize detection of fatigue of the human body.

Disclosure of Invention

The invention aims to solve the technical problem of providing a human body fatigue monitoring system based on a mobile phone camera, which is a portable system capable of detecting the mental fatigue state of a human body in real time.

In order to solve the technical problems, the invention adopts the following technical scheme: a human body fatigue monitoring system based on a mobile phone camera comprises a signal acquisition module, a signal processing module and a data analysis module;

the signal acquisition module acquires pulse wave signals by using a mobile phone camera;

the signal processing module is used for carrying out noise reduction processing on pulse wave signals acquired by the mobile phone camera;

and the data analysis module is used for carrying out data analysis on pulse wave signals acquired by the mobile phone camera.

The working process of the signal acquisition module comprises the following steps:

1) Selecting a certain number of detection personnel, and covering the mobile phone camera with the fingers to obtain a video; the shot video is subjected to dependent processing, namely the video is decomposed into a series of pictures, and the video time sequence is the picture sequence;

2) Intercepting pixel analysis of a central area of a picture;

3) Extracting a third quantile Q3 of each image gray value of a certain number of periods to average, and taking the third quantile Q3 as a reference value A;

4) Counting the number B that the gray value of the pixel in the central area of each image is larger than A, and reflecting the blood volume in the blood vessel at the moment;

5) The PPG curve time sequence diagram can be obtained by taking the number of tons as a horizontal axis variable and taking the number B of pixel gray values corresponding to each dependency as a vertical axis.

The working process of the signal processing module comprises the following steps:

1) EMD (empirical mode decomposition) decomposes the original signal into a series of components IMF 1-IMF 7 of an eigenmode function IMF (eigenmode function) and a sum res of residual components in order of frequency components from high to low;

2) Performing frequency domain analysis on each IMF generated by the analysis: the pulse wave frequency of a normal adult is 1-1.6 Hz, and the PPG signal with high-frequency sound exposure and baseline diffuse movement eliminated can be obtained by adding the IMF3 and IMF4 two-order IMFs in the pulse wave frequency range.

The data analysis module comprises the following steps: 1) Determining a pulse wave datum point; 2) Frequency domain analysis of heart rate variability; 3) Time domain analysis of heart rate variability; 4) And (5) establishing an HMM fatigue evaluation model.

The technical scheme has the advantages that the mobile phone camera is used for collecting pulse wave signals of a human body, parameters obtained after data processing are similar to heart rate variability parameters, then frequency domain analysis and time domain analysis are carried out, the HMM theory is used for establishing a human body fatigue evaluation model based on HRV, and experimental results show that the model has higher accuracy and can be used for evaluating the fatigue state of the human body so as to judge whether the human body is in the fatigue state.

Drawings

Fig. 1 is a schematic diagram of a human body fatigue monitoring system based on a mobile phone camera provided in an embodiment of the present invention;

FIG. 2 is a schematic diagram of the cell phone camera-based human fatigue monitoring system of FIG. 1;

FIG. 3 is a schematic diagram of the cell phone camera-based human fatigue monitoring system of FIG. 1;

FIG. 4 is a schematic diagram of the cell phone camera-based human fatigue monitoring system of FIG. 1;

FIG. 5 is a timing diagram of PPG curves;

fig. 6 is a graph of the PPG signal after EMD processing;

FIG. 7 is a schematic diagram of three defining methods of pulse wave reference points;

FIG. 8 is a schematic representation of a Markov chain.

Detailed Description

The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

Referring to fig. 1 to 8, a human body fatigue monitoring system based on a mobile phone camera comprises a signal acquisition module, a signal processing module and a data analysis module.

The signal acquisition module:

photoplethysmography rationale: imaging photoplethysmography is based on contact photoplethysmography. Photoplethysmography was first proposed by Hertzman in 1838. The realization of photoplethysmography is due to the nature of human skin tissue, when light is reflected out after being irradiated to human skin and received by a photosensitive sensor, the illumination intensity can generate certain attenuation, the blood has fluidity, and along with the contraction and the relaxation of cardiac muscle, the filling degree of blood in blood vessels is changed, so that the absorption of arterial blood to light is changed, and the periodicity is generated. Based on the detection principle of photoplethysmography, the acquisition part is set as a finger area, the acquisition device is a mobile phone camera, and two substances are generally considered to dominate the absorption of light in the visible light area due to the optical properties of human skin: hemoglobin and melanin. While hemoglobin is the primary light absorber in the dermis. Ambient light may permeate through the human skin and be re-released, where it may be absorbed by hemoglobin in the finger's blood, thereby causing a change in the re-released light compared to the incident light, which is difficult to detect by the human eye but may be captured by an optical sensor. The change will be manifest in pixels because the contraction and relaxation of the myocardium will change the degree of blood filling in the blood vessel, which in turn will cause the blood flow in the finger area to change, thereby affecting the intensity of the reflected light from the finger, which can be detected by the digital camera despite the subtle color change in the finger's skin that is not visible to the human eye. Therefore, the change of the pixel intensity can be captured by the camera, and further the change of the cardiac cycle of blood under the skin can be extracted, so that the non-contact detection is completely realized.

Each frame of image of the color video shot by the mobile phone is stored as a three-dimensional matrix, and 3 dimensions of the matrix respectively represent image data information of 3 color channels of red, green and blue. When selecting the color channel, two factors are mainly considered, namely, the absorptivity of blood to light in the corresponding wave band, and whether the illumination light in the selected wave band can penetrate the epidermis of the human body so as to reach the arteriole layer below the epidermis. Green light is at the peak of the blood absorption light and can penetrate the arteriole layer of the skin, so the study analyzes the pixels of the green channel in the video.

The specific experiment is set as follows: experiment one: 40 healthy volunteers (26 men and 14 women) of the southern Anhui medical college of medical science of 10 months of 2022 and 9 months of 2020 are selected, and a video is obtained by covering fingers on a mobile phone camera. And shooting a finger video schematic diagram. The experimenter remained motionless while the PPG signal was acquired, with 30s per acquisition time. At the same time, the electrocardiographic signals of volunteers were acquired using a standard 12-lead format (I-lead).

Experiment II: the 40 experimenters were subjected to PPG (photoplethysmogram) signal simultaneous acquisition by clamping the existing medical instrument CONTEC pulse oximeter on the market on the other finger. The CONTEC pulse oximeter measurement mode. After measurement, pulse wave data is transmitted to a computer through a USB data line.

Next, the following is performed:

step 1: and carrying out framing treatment on the video shot in the experiment I, namely decomposing the video into a series of pictures, wherein the frame sequence of the pictures is the time sequence of the video.

Step 2: and intercepting pixel analysis of the central area of the picture so as to reduce interference of ambient light, and analyzing pixels of the intercepted image.

Step 3: extracting a third quantile Q3 of the gray value of each frame of image in a plurality of periods to average, and taking the third quantile Q3 as a reference value A:

step 4: counting the number B of pixels with gray values larger than A in the central area of each frame of image, and reflecting the blood volume in the blood vessel at the moment;

step 5: and taking the frame number as a horizontal axis variable and the number B of pixel gray values corresponding to each frame as a vertical axis, so as to obtain the PPG curve timing diagram. As shown in fig. 5:

the PPG curve timing diagram shown in fig. 5 has low frequency baseline drift and some high frequency noise, which can greatly reduce accuracy, and the accuracy of measurement of human physiological parameters based on photoplethysmography waves can also greatly reduce, so that the measurement needs to be eliminated from the signal.

And a signal processing module:

the human physiological sign signals are very weak, so that pulse wave signals acquired by utilizing a mobile phone camera are easy to be interfered by noise, further the accuracy of experimental data is reduced, and therefore, the signal processing is particularly critical. The sources of noise are mainly two: high frequency noise and low frequency baseline drift. Currently, there are many methods for processing pulse wave signals, for example: an adaptive filter method, a wavelet transformation method, a polynomial difference method and an empirical mode decomposition method. The self-adaptive filter has slower signal processing speed, can not realize automatic tracking of filtering frequency, and is easy to influence the experiment progress; the wavelet transformation can be used for removing baseline drift in pulse waves, but the processing signals depend on the selection of wavelet bases, once the wavelet bases are not properly selected, the progress of the whole experiment can be influenced, the experiment result is greatly influenced, and the fault tolerance rate is low; finding a reference point in the polynomial difference is important; the empirical mode decomposition (Empirical Mode Decomposition, EMD) can perform adaptive decomposition according to the local characteristics of the signal, and has an excellent signal-to-noise ratio. Unlike the above method, the following steps are adopted: the decomposition of different frequency components of the signal can be realized without selecting and setting any reference point or base function in advance. Thus, the application uses EMD to process the PPG signal.

Step 1: the EMD decomposes the original signal into a number of eigenmode function (Intrinsic Mode Function, IMF) components (IMF 1-IMF 7) and a sum of residual components (res) in order of frequency components from high to low. Different IMF components cause the characteristics of the signal to manifest at different time scale resolutions.

Step 2: and carrying out frequency domain analysis on each IMF generated by the analysis. The pulse wave frequency of a normal adult is about 1-1.6 Hz, and the PPG signal with high-frequency noise and baseline drift eliminated can be obtained by adding the IMF3 and IMF4 two-order IMFs in the pulse wave frequency range. Fig. 6 shows the PPG signal after EMD processing.

And a data analysis module:

heart rate variability stems from the physiological phenomenon that the motion of the heart is irregular, which was suggested by Albrecht von Haller in the beginning of the 18 th century. Hon & Lee observed variability in RR intervals for the first time on fetuses. The study shows that the extremely short time heart rate variability shows good consistency with the pulse variability, the heart rate variability is equivalent to the pulse variability (pulae rate variability, PRV), and the heart rate variability refers to the change characteristic between successive heart beat intervals and is determined according to two adjacent heart beat periods. Heart rate variability is used to determine autonomic activity and measure the degree of balance between internal sympathetic and vagal regulation. Heart disease also reflects heart variability from which heart disease and sudden death risk can be determined. In physiological parameter calculation and autonomic nervous system assessment, PRV may be equivalent to HRV for subsequent calculation.

Determination of pulse wave reference points:

the time between two consecutive R-waves in the ECG waveform is defined as R wave to R wave Interval and is denoted RRI. The time between two consecutive feature bases in the pulse wave is defined as Pulse to Pusie Interval, denoted PPL, and the position of the feature base of the pulse wave depends on the definition of the feature base of the pulse wave, and there are three methods for defining the feature base of the pulse wave:

1. the second-order differential maximum method searches a second-order differential maximum point diagram in the pulse wave rising period, such as a number 2 mark point on the PPG signal in fig. 7, and the pulse rate interval obtained by the method is marked as PPI1.

2. The main wave peak method is the maximum point of the main wave pulse, such as the marked point 3 on the PPG signal in fig. 7. The pulse rate interval resulting from this method is noted as PPI2.

3. The intersection point of the tangent line of the maximum slope point and the base line in the pulse wave rising period by the tangent intersection point method is shown as a mark point 1 on the PPG signal in fig. 7, and the pulse rate interval obtained by the method is marked as PPI3.

Frequency domain analysis of heart rate variability:

the power spectral parameters of short-range HRV are Total Power (TP), high frequency power (high frequency power, HF), low frequency power (low frequency power, LF), and normalized low/high frequency power ratio (LF/HF). Among these parameters, LF/HF can reflect the relative ratio of high and low frequency power, so that the influence of factors such as data length, individual difference, modeling method and the like on the frequency domain HRV parameter calculation result can be reduced to the greatest extent, and the parameters are quantization parameters reflecting the balance and the equilibrium of the autonomous nervous system.

Step 1: the R-R interval sequence is extracted by a QRS wave recognition algorithm based on a second-order differential minimum value. By comparing the empirical formula method of whether the instantaneous R-R interval is between 0.75 times and 1.25 times of the average value of the R-R interval, the wrongly detected R wave can be effectively identified, and the accuracy of the detection result is ensured.

Step 2: the pulse wave main wave peak PP interval (PPI 1, PPI2, PPI 3) is extracted.

Step 3: and 4Hz cubic spline resampling is carried out on the RR interval sequence and the PP interval sequence to obtain a uniform sampling time sequence.

Step 4: the trend item component of the ultralow frequency is removed by adopting a smooth priori principle, and the data of 512 points are extracted to make the power spectrum estimation of the 19-order AR model based on the Burg algorithm (the AR model is widely used for heart rate variability frequency domain analysis due to good spectrum resolution).

Step 5: frequency domain parameters of HRV (heart rate variability parameters of RRI, PPI1, PPI2, PPI3 are denoted by HRV, HRV1, HRV2, HRV3, respectively) were obtained.

Step 6: and comparing the HRV results obtained by the two methods by using a regression statistical method and a Bland-Altman random analysis method to obtain the result.

Time domain analysis of heart rate variability:

time domain parameters:

the time domain analysis refers to calculating the change of RR intervals by a statistical discrete trend analysis method, and HRV time domain statistical indexes of short-range (2-5 min) electrocardio data have standard deviation SDNN of RR intervals and root mean square value r-RMSSD of RR interval difference values:

wherein: n is the total number of normal heart beats; RR (RR) _i Is the i-th R-R interval;

is the average of the R-R intervals of N heartbeats;

wherein: n is the total number of normal heart beats; RR (RR) _i Is the i-th R-R interval; RR (RR) _i And RR _i+1 Is the length of two adjacent sinus cardiac intervals.

Establishment of HMM fatigue evaluation model (hidden Markov model)

Step 1: establishing an HMM fatigue evaluation model, wherein λ= (pi, A, B) of initial model parameters is needed to be determined; pi=p (qi=si), 1 < i < N

Wherein pi is more than or equal to 0 and less than or equal to 1,

pi is an initial state probability vector, P is probability, N is natural number, and i is a numerical value obtained in an experiment; the following is the sum Σ;

a and B represent two matrixes, namely a state transition probability matrix and are distinguished and indicated by letters; the following ranges are the values of pi.

Since pattern recognition usually adopts a left-right model, the initial state probability vector pi i is not estimated and is set as: pi 1=1, pi i=0, (i=2, 3, …, N)

The number of states of the Markov chain of the human fatigue state evaluation model established herein is 2, as shown in fig. 8. State S1 indicates that the volunteer is in an awake state, and state S2 indicates that the volunteer is in a tired state.

Step 2: the number of states of the Markov chain is 2, i.e., the awake state and the fatigue state, and the number of transfer paths from each state is 2, so aij=0.5; aij is the element of the ith row and the jth column of the matrix, and represents the algebraic remainder corresponding to the element of the ith row and the jth column of the matrix. The initial value of the state transition probability matrix A is selected by utilizing the principle of uniform distribution, and the initial value is determined by utilizing the formula:

namely, the initial state transition probability matrix a is:

the aim of the study is to determine whether the human body is in mental fatigue state, thus classifying the hidden state into two kinds, namely, awake state and fatigue state. Because the person is in a fatigue or awake state at a certain moment, the person is likely to be in an awake or fatigue state at the next moment, and the HMM model is characterized in that: the state may be transferred to the next state or to the state itself. The number of states of the Markov chain of the human fatigue state evaluation model established herein was found to be 2 by this analysis.

From the above analysis, the number of states of the Markov chain is 2, i.e., the awake state and the fatigue state, and the number of transfer paths to be transferred from each state is 2, so aij=0.5.

The state split threshold for SDNN is set to 60. The awake state is set to 1 and the fatigue state is set to 2. I.e. the probability of the body transitioning from the awake state to the awake state at a moment, e.g. a12 the probability of the body transitioning from the awake state to the fatigue state, a21 the probability of the body transitioning from the fatigue state to the awake state, a22 the probability of the body transitioning from the fatigue state to the fatigue state.

And B, analyzing the collected PPG signals based on the obtained experimental data and extracting SDNN parameters to form a sample database for constructing the HMM human body fatigue evaluation model. Calculating the value of each element bij in the matrix B by using a mathematical statistics method, and when the bij is in a state of i, observing the probability of j.

If B11 in the initial observation probability matrix B is calculated, a group of observation sample sequences with the length of 24 is selected as statistical data:

sample: o= [2,2,1,1,1,1,2,2,2,2,1,1,2,2,2,2,2,2,2,2,1,1,1,1]

The corresponding state sequence of the experimental record at this time is:

Q＝[1，1，1，1，1，1，2，2，2，2，2，2，2，2，2，2，2，2，1，1，1，1，1，1]

b11 represents the probability that the observation variable is normal when the human body is in an awake state, namely the probability that the value in the corresponding observation sequence is 1 when the value in the state sequence is 1. The number of the values of 1 in the above state sequence is 12, the number of the states of 1 and the number of the observed values of 1 is 8, b11=8/12, and the values of other elements in the observation probability matrix B are obtained by adopting the same method.

Step 3: and training the initial model parameters by using a Baum-Welch algorithm to obtain proper model parameters.

Step 4: and inputting the observation value sequence into the established HMM fatigue evaluation model by using the Viterbi algorithm to obtain an optimal state sequence, and comparing the optimal state sequence with an actual state sequence to judge the accuracy of the model. Therefore, human body fatigue assessment is completed by the mobile phone camera.

In the current society, the pressure faced by people's daily work and life is increasingly high, news of sudden death caused by frequent overtime is frequently used, and psychological and physical fatigue caused by excessive pressure is an important cause of most sudden death cases. However, mental fatigue is more difficult to detect than physical exhaustion, and it is difficult for people to find themselves already in mental fatigue. Therefore, in order to detect the mental condition of the human body in real time and give an alarm when the human body is in a fatigue state, the research uses a mobile phone camera to collect pulse wave signals of the human body, parameters obtained after data processing are similar to heart rate variability parameters, then frequency domain analysis and time domain analysis are carried out, an HRV-based human body fatigue evaluation model is established by using the HMM theory, and an experimental result shows that the model has higher accuracy and can be used for evaluating the fatigue state of the human body so as to judge whether the human body is in the fatigue state.

In the description of the present invention, it should be understood that the terms "coaxial," "bottom," "one end," "top," "middle," "another end," "upper," "one side," "top," "inner," "front," "center," "two ends," etc. indicate orientations or positional relationships based on the orientation or positional relationships shown in the drawings, are merely for convenience in describing the present invention and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a specific orientation, be configured and operated in a specific orientation, and thus should not be construed as limiting the present invention.

In the present invention, unless explicitly specified and limited otherwise, the terms "mounted," "configured," "connected," "secured," "screwed," and the like are to be construed broadly and may be, for example, fixedly connected, detachably connected, or integrally formed; can be mechanically or electrically connected; either directly or indirectly through intermediaries, or in communication with each other or in interaction with each other, unless explicitly defined otherwise, the meaning of the terms described above in this application will be understood by those of ordinary skill in the art in view of the specific circumstances.

Although embodiments of the present invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made therein without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.

Claims

1. The human body fatigue monitoring system based on the mobile phone camera is characterized by comprising a signal acquisition module, a signal processing module and a data analysis module;

2. The human body fatigue monitoring system based on the mobile phone camera as claimed in claim 1, wherein the signal acquisition module working process comprises the following steps:

1) Selecting a certain number of detection personnel, and covering the mobile phone camera with the fingers to obtain a video; carrying out frame division processing on the shot video, namely decomposing the video into a series of pictures, wherein the frame sequence of the pictures is the time sequence of the video;

2) Intercepting pixel analysis of a central area of a picture;

5) The PPG-photoplethysmogram curve time sequence diagram can be obtained by taking the number of the tons as a horizontal axis variable and the number of the pixel gray values B corresponding to each frame as a vertical axis.

3. The human body fatigue monitoring system based on the mobile phone camera as claimed in claim 2, wherein the working process of the signal processing module comprises the following steps:

1) The EMD-empirical mode decomposition method is used for decomposing an original signal into a series of components IMF 1-IMF 7 of an eigenmode function IMF and a sum res of residual components according to the sequence of frequency components from high to low;

4. The human body fatigue monitoring system based on the mobile phone camera as claimed in claim 3, wherein the data analysis module comprises the following steps: 1) Determining a pulse wave datum point; 2) Frequency domain analysis of heart rate variability; 3) Time domain analysis of heart rate variability; 4) And (5) establishing a fatigue evaluation model of the HMM-hidden Markov model.

5. The mobile phone camera-based human body fatigue monitoring system according to claim 4, wherein in the step of determining the pulse wave reference point, a time between two consecutive R waves in the ECG electrocardiograph signal waveform is denoted as RRI; the time between two successive feature bases in the pulse wave is denoted PPL, and the position of the feature bases of the pulse wave depends on the definition of the feature bases of the pulse wave.

6. The system for monitoring fatigue of a human body based on a mobile phone camera according to claim 5, wherein in the step of determining the pulse wave reference point, there are three methods for defining the pulse wave reference point:

1) Searching a second-order differential maximum value point diagram in the pulse wave rising period by a second-order differential maximum value method, and marking the pulse rate interval obtained by the method as PPI1;

2) The pulse wave main peak method is characterized in that the pulse rate interval obtained by the method is marked as PPI2;

3) The pulse rate interval obtained by the method is marked as PPI3.

7. The mobile phone camera based human body fatigue monitoring system according to claim 6, wherein in the frequency domain analysis step of heart rate variability, the power spectrum parameters of short range HRV-heart rate variability include total power TP, high frequency power HF, low frequency power LF and normalized low/high frequency power ratio LF/HF, among which LF/HF is used as quantization parameter reflecting balance and equalization of autonomic nervous system because LF/HF reflects the relative ratio of high and low frequency power.

8. The cell phone camera based human fatigue monitoring system as in claim 7, the frequency domain analysis of heart rate variability comprising the steps of:

1) Extracting R-R, namely a distance interval sequence between two QRS wave R waves and the R wave by a QRS wave identification algorithm based on a second-order differential minimum value; by comparing the empirical formula of whether the instantaneous R-R interval is between 0.75 and 1.25 times the mean of the R-R interval,

identifying R waves of the false detection, and ensuring the accuracy of detection results;

2) Extracting the time-PPI 1, PPI2 and PPI3 of the main wave crest PP interval of the pulse wave, namely the primary cardiac cycle;

3) Resampling the RR interval and PP interval sequences with 4Hz cubic spline to obtain uniform sampling time sequence

4) Removing the trend item component of the ultralow frequency by adopting a smooth priori principle, extracting 512-point data, and performing power spectrum estimation of a 19-order AR model-autoregressive model based on a Burg algorithm-a Berg algorithm;

5) Frequency domain parameters of HRV-heart rate variability were obtained: heart rate variability parameters of RRI, PPI1, PPI2, PPI3 are denoted HRV, HRV1, HRV2, HRV3, respectively;

6) And comparing the HRV results obtained by the two methods by using a regression statistical method and a Bland-Altman random analysis method to obtain the result.

9. The human body fatigue monitoring system based on the mobile phone camera of claim 8, wherein in the time domain analysis step of heart rate variability, the change of the R-R interval is calculated by a statistical discrete trend analysis method, and the HRV-heart rate variability time domain statistical index of the short-range 2-5 min electrocardiograph data is as follows: root mean square value of standard deviation SDNN of R-R interval and R-R interval difference R-RMSSD:

wherein: n is the total number of normal heart beats; RR (RR) _i Is the i-th R-R interval; RR is the average of R-R intervals for N heartbeats;

10. The human body fatigue monitoring system based on the mobile phone camera of claim 9, the establishment of the HMM fatigue evaluation model comprises the following steps:

1) First, λ= (pi, a, B) of initial model parameters need to be determined;

πi＝P(qi＝Si)，1＜i＜N

wherein pi is more than or equal to 0 and less than or equal to 1,

2) The number of states of the Markov chain-hidden Markov chain is 2, i.e., the awake state and the fatigue state, and the number of transfer paths to be transferred from each state is 2, so aij=0.5; aij is the i-th row and the j-th column of the matrix, and represents the corresponding algebraic remainder of the i-th row and the j-th column of the matrix;

3) Setting the state division broad value of SDNN to 60; the awake state is set to 1 and the fatigue state is set to 2; i.e. the probability of the human body transferring from the awake state to the awake state at a moment;

4) B, analyzing the collected PPG signals and extracting SDNN parameters based on the obtained experimental data to form a sample database for constructing an HMM human epidemic assessment model; calculating the value of each element bij in the matrix B by using a mathematical statistics method, wherein when the state of bij is i, the probability that the observed value is j is obtained;

5) Training the initial model parameters by using a Baum-Welch algorithm, namely a Bom Welch algorithm, so as to obtain model parameters;

6) Inputting the observation value sequence into the established HMM fatigue evaluation model by using a Viterbi algorithm-Viterbi algorithm to obtain an optimal state sequence, and comparing the optimal state sequence with an actual state sequence to judge the accuracy of the model; therefore, human epidemic degree evaluation is completed by the mobile phone camera.