KR101834906B1 - System and method for detecting abnormal status of user based on bio-signal and wearable device - Google Patents

Abstract

The present invention relates to a system and a method for detecting an abnormal status of a user based on a bio-signal and a wearable device, which classify and detect an abnormal status of a driver in advance based on a bio-signal measured by using a wearable device and notify the abnormal status to the driver and a passenger, thereby preventing an accident in advance. The present invention detects the abnormal status of the driver in advance, notifies the abnormal status to the driver and the passenger, receives the abnormal status of the driver in advance, and takes an appropriate measure such as resting or indoor ventilation, thereby preventing a car accident due to the abnormal status and reducing human and material losses due to the accident. The system for detecting an abnormal status of a user based on a bio-signal and a wearable device comprises: the wearable device measuring the bio-signal and a movement signal of the user, and transmitting the measured signals to the outside; and an abnormal status detecting and displaying device receiving and processing the signal transmitted from the wearable device, and detecting and displaying the abnormal status of the user.

Description

Technical Field [0001] The present invention relates to a system and a method for detecting a user anomaly state based on a bio-signal and a wearable device,

The present invention relates to an abnormal state detection system and method, and more particularly, to a system and method for detecting an abnormal state of a driver based on a biological signal measured using a wearable device, and notifying the driver and the passenger of the accident, And a system and method for detecting a user anomaly state based on a wearable device.

During driving, abnormal conditions such as driver 's stress, sleepiness, and fatigue affect the driver' s response speed and judgment ability, which reduces driving ability and increases the risk of traffic accidents.

 Various methods have been proposed to monitor the driver's abnormal condition during driving. One of the methods is based on image processing based on detection of driver's drowsiness as in Patent Document 1, The driver's drowsiness is detected by capturing signs of drowsiness appearing on the face of the driver.

However, these image processing based methods are susceptible to environmental factors such as lighting conditions and driver wearing glasses, and have the disadvantage that drowsiness can be detected after the drowsiness symptoms appear on the driver 's face.

In addition, many systems have been proposed to detect abnormal conditions and generate notifications based on the collected bio-signals of various drivers including EEG. However, the bio-signal-based method has a disadvantage that the measurement device for collecting bio-signals must be attached to the driver's body. Accordingly, in order to solve such a problem, an attempt has been made to easily detect a driver's abnormality by collecting a bio-signal of a driver, and recently, a method as disclosed in Patent Document 2 based on a wearable device has been proposed.

Biological signals are signals that can be measured in our body. In addition to brain waves generated by brain activity, PPG (photoplethysmography), which measures blood flow by heartbeat, is used to measure the skin conductivity that changes with the activation of sweat glands And the temperature of the body in addition to the galvanic skin response (GSR).

As described above, in order to construct a practical driver abnormality detection system based on a bio-signal, it is essential to collect bio-signals such as a wearable device while minimizing driver's inconvenience.

However, the non-invasive sensor or signal measurement method used in wearable devices is very sensitive to the movement of the measurer and contains a lot of noise, so there is a high possibility of making an erroneous state judgment when the signal is used for state discrimination due to the inaccuracy of the obtained signal. Therefore, it is essential to determine the reliability of the signal according to the purpose of use in the system using the wearable device.

In addition, when the abnormal state is determined, the abnormal state such as stress, drowsiness, and fatigue is discriminated in addition to the normal state. Therefore, it is necessary to extract the characteristic that distinguishes each state from the bioelectric signal. The abnormal state can be classified and detected.

: Patent application 10-2004-0004677 : Patent Application No. 10-2014-0103845

SUMMARY OF THE INVENTION The present invention has been made in view of the above problems, and it is an object of the present invention to provide a biological signal detecting apparatus and method which can prevent an accident from occurring by informing a driver and a passenger of an anomaly of a user based on a biological signal measured using a wearable device, And a system and method for detecting a user abnormal state based on a wearable device.

According to another aspect of the present invention, there is provided a system for detecting an anomaly of a user, the system including: a bio-signal and a motion signal of the user; To a wearable device; And an abnormal state detection and display unit for receiving and processing a signal transmitted from the wearable device to detect and display an abnormal state of the user.

According to another aspect of the present invention, there is provided a method for detecting an anomaly of a user, the method comprising: detecting a user's biological signal and a motion signal through a wearable device; And a motion signal measuring step; A signal transmitting step of transmitting the measured signal to an anomaly detection and display device; A signal receiving step of receiving the transmitted signal; An abnormal state detecting step of detecting an abnormal state through a preprocessing process on the received signal; And an abnormal state notification step of informing the detected abnormal state to the outside.

As described above, the system and method for detecting a user abnormality state based on a biological signal and a wearable device according to the present invention can detect an abnormal state of a user, particularly a driver, and inform the driver and the passenger of the abnormality. So that appropriate measures such as rest and indoor ventilation can be taken to prevent a traffic accident caused by an abnormal situation, thereby reducing human and material loss caused by an accident.

Furthermore, in addition to the prevention of accidents, information on the driver's anomaly helps the driver to make various driving decisions by helping him / her to make various decisions that occur during driving such as driving route, destination, and break time.

In addition, the bio-signal processing method and the state detection algorithm included in the present invention as well as the abnormal state detection of the driver can be applied to various products such as various healthcare products and systems for monitoring the status of workers in other industrial sites have.

1 is a configuration diagram of a user abnormality detection system based on a wearable device and a bio-signal according to the present invention.
2 is a flowchart of a method of detecting a user anomaly state based on a wearable device and a bio-signal according to the present invention.
FIG. 3 is a flowchart illustrating the abnormal state detection step of FIG. 2; FIG.
4 is a diagram showing the waveforms of the measured PPG signals and the detected peaks.
FIG. 5 is a flowchart illustrating the preprocessing and signal discrimination steps of FIG. 2; FIG.

The present invention relates to a practical bio-signal-based driver anomaly state system and a detailed algorithm, and measures a driver's bio-signal and a motion signal using a wearable device in a clock shape for a practical system configuration.

A preprocessing process is performed to check whether the signal can be used to detect an abnormal condition by measuring various signals for detecting an abnormal condition from the driver and then checking the measured signal. The abnormal state detection is performed using the signal.

Adaboost, which is one of ensemble learning techniques that compiles the results of several classifiers and calculates the final result, is used to detect more anomalies in order to have more accurate detection performance.

The present invention proposed to achieve the above object is largely composed of a wearable device for measuring a bio-signal, an abnormal state detection and indicator for processing a measured signal and for informing a detected state, The details of the implementation shall be described in detail.

1 is a configuration diagram of a user abnormality detection system based on a wearable device and a bio-signal according to the present invention.

Referring to FIG. 1, a bio-signal and wearable device-based user anomaly detection system according to the present invention includes a wearable device 100 and an anomaly detection and display device 200.

 The wearable device 100 may include a smart watch type device for measuring information due to various vital signs or movements occurring in the body of a wearer and worn mainly on the wrist.

The wearable device 100 includes a PPG sensor 110, a temperature sensor 120, a GSR sensor 130, an accelerometer 140, a gyroscope 150, and a wireless communication unit 160 do. The PPG sensor 110, the temperature sensor 120 and the GSR sensor 130 sense the wearer's PPG signal, temperature, and GSR signal. The accelerometer 140 and the gyroscope 150 measure information due to the movement of the wearer. The wireless transmission unit 160 transmits signals measured by the sensors to the outside through wireless communication. At this time, the wireless communication may include Bluetooth, ZigBee or Radio Frequency (RF) communication.

The abnormality detection and display device 200 is a device that is placed in a position where it can be easily seen by a driver or a passenger inside the vehicle. The abnormality detection and display device 200 includes a wireless reception unit 210, hardware 220 for control and calculation, .

The wireless receiving unit 210 is a device for receiving a signal transmitted from the wearable device 100 and uses the same communication method as the wireless transmitting unit 160 of the wearable device.

The hardware 220 for control and calculation controls the operation of the wireless communication unit 210 and the abnormal state notification unit 230 and processes the signal received from the wireless transmission unit 210 to determine the current state of the driver. And then transmits the detected state to the abnormal state notifying unit 230 so that appropriate measures can be taken. Accordingly, the hardware 220 for control and operation may be configured as a micro controller unit (MCU), a digital signal processor (DSP), or a small computer for controlling and operating peripheral devices.

The abnormal state notification unit 230 receives the type of the user state detected by the hardware 220 for control and operation and generates an appropriate notification to notify the abnormal state to the outside. The notification may include a visual notification such as an LED light or a warning message output depending on the state, and an audible notification such as a warning sound or a voice message depending on the state. Also, background music for alleviating the detected condition, and follow-up measures such as ventilation, vibration and the like may be included.

2 is a flowchart of a method of detecting a user anomaly state based on a wearable device and a bio-signal according to the present invention.

As shown in FIG. 2, the bio-signal and wearable device-based user abnormal state detection method according to the present invention includes a bio-signal and motion signal measurement step S100, a signal transmission step S200, a signal reception step S300, A state detection step S400 and an abnormal state notification step S500.

In the bio-signal and motion signal measurement step S100, a wearable device of the clock type is used to measure the movement of the driver's bio-signals, acceleration and gyro signals such as the PPG signal, the temperature signal and the GSR signal Measure the signal.

In the signal transmission step S200, a measured signal is transmitted using the wireless communication unit in the wearable device, and in the signal reception step S300, a signal transmitted from the wearable device is detected using the abnormal state detection and wireless communication unit in the indicator .

In the abnormal state detection step S400, an abnormal state is detected through a preprocessing process on the received signal. In the abnormal state notification step S500, the abnormal state is informed to the driver or the passenger through the visual and auditory notification device.

FIG. 3 is a flowchart showing a detailed process of detecting the abnormal state (S400) of FIG. 2, and FIG. 4 is a diagram showing the waveform of the measured RF signal and the detected peak.

The abnormal state detection step S400 includes a peak detection step S410, a preprocessing and signal discrimination step S420, a feature extraction step S430 and an abnormal state extraction step S440.

In the peak detection step (S410) of the peak signal, the peak of the peak signal is detected from the signal transmitted from the wearable device. The peak of the RF signal is the peak of each pulse of the RF signal indicated by red in the RF signal of FIG. 4, and the blood is delivered to the blood vessel by the heartbeat, . The peak of the feature signal can be detected by selecting an algorithm suitable for the environment configured in the present invention, and the change in the heart rate can be detected through the interval of peaks when the peak is detected.

 In the preprocessing and signal determination step S420, a signal measured through the wearable device is pre-processed to determine whether or not it can be used for abnormal state detection.

When measuring a signal using a wearable device, non-invasive signal measurement causes a lot of noise. When a signal including such a noise is used for state discrimination, it is easy for an erroneous state judgment to be made. Therefore, it is essential to determine whether the signal can be used for abnormal condition detection by preprocessing.

The preprocessing and signal discrimination are performed on the PPG signal which is most sensitive to the noise. After detecting the peak in the PPG signal, it is checked whether the following four conditions are satisfied.

The first condition relates to the degree of increase / decrease of heart rate, and uses the inequality equation (1), where tk-1, tk, tk + 1 are consecutive three peak positions.

The second condition is to check whether the interval of peaks does not deviate greatly from the reference value. It is checked whether the difference between the average peak interval of the RF signal to be inspected and the reference value is within? Times of the reference value. More specifically, when the average peak interval (PI) of the RF signal to be tested is referred to as a tested interval (TI) and the reference peak interval (PI) is defined as an average peak interval (MPI) Similarly, it is checked whether the absolute value of the difference between the test interval TI and the average peak interval MPI is smaller than? Times the average peak interval MPI. The probability that noise will occur when the motion of the device measuring the signal is small is reduced.

Therefore, the third condition is to check whether the movement of wearable device for signal measurement is below constant. In order to obtain information about the movement of the wearable device, a high-pass filter is applied to exclude the value due to the gravity component from the acceleration values of the measured x, y, and z axes, and then the sum is added to the squares. Then, it is checked whether the average value of the calculated motion is less than the reference value to check whether the movement of the measuring device is small.

The last condition is based on the fact that the shape of the pulse of each RF signal is similar in a small noise situation. The correlation coefficient (CC) of the pulse of each RF signal in the interval And it is checked whether the minimum value is equal to or greater than a predetermined reference value. At this time, the correlation coefficient (CC) is calculated as shown in Equation (3), and the similarity of the two signals x and y is compared.

FIG. 5 is a flowchart illustrating the preprocessing and signal discrimination steps of FIG. 2; FIG.

FIG. 5 is a flow chart for judging whether or not an intra-section signal to be examined using the above-mentioned four conditions is available. When the first condition and the second condition or the first condition, the third condition and the fourth condition are satisfied, It is judged that the signal of If it is determined that the signal of the corresponding interval is available, the average peak interval (MPI) is updated as shown in Equation (4).

If it is determined that the signal is a usable signal in the preprocessing and signal discrimination step S420, a feature extraction step S430 for detecting an abnormal state is performed using the signal. The features are the target signal, temperature, ground signal, and gyro signal among the measured signals and extract the following seven features.

The first feature is the average peak interval (PI) of the FE signal, and the second feature is the average of each peak magnitude in the signal with the low frequency components below 0.5 Hz removed from the FE signal. The third feature is the median value of the RF signal to determine the magnitude of the DC component. The fourth characteristic is the ratio of the second AC characteristic to the DC characteristic. The fifth and sixth features are the average values of the temperature and the zygote signal, respectively. The last feature is the average of the values obtained by taking the squares of the gyro signals of the three axes followed by the square root of the sum and the degree of change of the gyro value.

The abnormal state extraction step S440 actually detects the abnormal state using the seven features extracted in the feature extraction step S430 for detecting the abnormal state. A classifier is used for state detection and classification. The classifier uses Adaboost, which is one of ensemble learning techniques, to derive reliable performance for normal, stress, drowsiness, and fatigue states that are difficult to distinguish. . Adaboost is a method to synthesize the results of several classifiers and collects the results of several classifiers designed, designed and trained one or more neural network based classifiers to perform final state classification and detection do.

Although the present invention has been described with reference to the embodiments shown in the drawings, it is to be understood that various modifications and equivalent embodiments may be made by those skilled in the art without departing from the scope of the present invention. Accordingly, the true scope of the present invention should be determined by the technical idea of the appended claims.

Claims (17)

A system for detecting an abnormal condition of a user,
A wearable device for measuring the user's bio-signal and motion signal and transmitting the measured signal to the outside; And
And an abnormal state detection and display unit for receiving and processing a signal transmitted from the wearable device to detect and display an abnormal state of the user,
The abnormality detection and display device
A wireless receiving unit for receiving a signal transmitted from the wearable device;
Hardware for control and operation for preprocessing the received signal and detecting an abnormal state; And
And an abnormality state notification unit for informing the abnormality detected from the hardware for the control and operation to the outside,
The hardware for the control and computation
A preprocessing for judging whether or not a peak of the measured RF signal and a signal of the corresponding section can be used for detecting an abnormal state, a feature extraction for detecting an abnormal state, and an abnormal state detection operation using AdaBoost And a wearable device based on the biological signal. The wearable device of claim 1, wherein the wearable device
An image sensor for measuring a user's image signal;
A temperature sensor for measuring the temperature of the user;
A geosynthetic sensor for measuring the user's signal;
An accelerometer for measuring acceleration according to user's motion;
A gyroscope for measuring a rotational motion signal according to a user's motion; And
And a radio transmitter for transmitting a bio-signal and a motion-related signal measured from the RF sensor, the temperature sensor, the GPS sensor, the accelerometer, and the gyroscope to the abnormal state detection and display device. Device based user abnormal state detection system. delete delete The wireless communication system according to claim 2, wherein the radio transmitter and the radio receiver
Wherein the signal is transmitted and received in a wireless communication system including Bluetooth, RF, and ZigBee. The method of claim 1, wherein the hardware for the control and computation comprises
And a microcomputer unit (MCU), a digital signal processor (DSP) or a small computer for controlling and operating peripheral devices. The apparatus according to claim 1, wherein the abnormality notification unit
Visual notifications such as lighting of LEDs and output of guidance messages;
Audible alarms such as beeps or voice messages; And
Wherein the abnormality state is informed to the outside via a tactile alert by vibration.
A method for detecting an abnormal state of a user,
A biological signal and a motion signal measuring step of measuring a user's biological signal and a motion signal through the wearable device;
A signal transmitting step of transmitting the measured signal to an anomaly detection and display device;
A signal receiving step of receiving the transmitted signal;
An abnormal state detecting step of detecting an abnormal state through a preprocessing process on the received signal; And
And an abnormal condition notification step of informing the detected abnormal condition to the outside,
The bio-signal and motion signal measurement step
The wearable device measures the user's feel signal, the temperature of the user, the user's earth signal, the acceleration according to the user's motion, and the rotational motion signal according to the user's motion,
The abnormal state detecting step
A peak detection step of detecting a peak of a feature signal from a signal transmitted from a wearable device;
A preprocessing and signal discrimination step of pre-processing a signal measured through the wearable device and discriminating whether a signal of the corresponding section can be used for detecting an abnormal state;
A feature extraction step of extracting a feature for detecting an anomaly state using a signal determined as a usable signal in the preprocessing and signal discrimination step; And
Extracting an abnormal state using the feature extracted in the feature extracting step,
The abnormal state extracting step
(Adaboost) technique that uses a plurality of classifiers to detect and classify a state in a normal state, a stress state, a drowsy state, or a tired state and synthesizes the results of the plurality of classifiers to calculate a final result And extracting an abnormal state based on the biological signal and the wearable device. delete 9. The method of claim 8, wherein the signal transmitting step and the signal receiving step
(RF), or Zigbee. The method of detecting a user anomaly based on a bio-signal and a wearable device. delete 9. The method of claim 8, wherein the preprocessing and signal determination step comprises:
A second condition according to the degree of similarity of the pulse intervals, a third condition according to the magnitude of the acceleration value, and a fourth condition according to the degree of similarity of the pulse shape, Wherein the step of determining whether or not the abnormal condition can be used for detecting an abnormal condition comprises the steps of: 9. The method according to claim 8, wherein the feature extracting step
A mean value of a peak value, an average value of a peak value, an intermediate value of a peak value signal to know a magnitude of a direct current component, a ratio of an average value of the peak size to an intermediate value of the peak value, Wherein the step of extracting the features comprises the step of extracting features constituted by the average value of the signals and the degree of change of the gyro value. delete 9. The method according to claim 8, wherein the abnormality notification step
And provides a visual notification such as lighting of an LED or output of a guidance message according to the detected abnormal state, providing an auditory notification such as a warning sound or a voice message, or providing a tactile notification by vibration A method for detecting a user anomaly state based on a biological signal and a wearable device. delete 13. The method of claim 12,
Wherein the first condition and the second condition are satisfied,
When the first condition, the third condition, and the fourth condition are satisfied, it is determined that the signal of the corresponding section can be used for detecting an abnormal condition. The method of detecting a user abnormal condition based on a biological signal and a wearable device .

Images