KR101938045B1 - System and method for determining drowsy state of passengers - Google Patents

Abstract

A sensor unit including a first sensor for sensing a passenger's bio-signal in a first mode and a second sensor for sensing a passenger's bio-signal in a second mode according to an embodiment of the present invention; And an abnormal state determiner for determining whether the occupant is in an abnormal state using the bio-signal.

Description

TECHNICAL FIELD [0001] The present invention relates to a system and method for determining an abnormal condition of a passenger,

The present invention relates to a system and method for determining an anomalous state of a passenger (for example, a driver), and more particularly, to a system and method for accurately determining whether a passenger is drowsy by utilizing a passenger's bio- ≪ / RTI >

Recently, it is known that the biggest cause of traffic accidents is drowsiness driving, and many methods for preventing drowsiness driving are being studied. Conventional driver drowsiness driving judgment techniques include a method of photographing the motion of a driver's pupil using a vehicle signal (a steering signal, a lateral position in a lane through a camera) or a camera for photographing a driver, Are mainly used.

However, it is difficult to distinguish the vehicle signal from the intentional vehicle behavior (driving due to intense driving and change of surrounding environment) pattern, and the video signal based method is accurate due to the poor performance of eye opening / It is difficult to determine fatigue drowsiness.

Korean Patent Application 10-2011-0093033

 An object of the present invention is to provide a system for determining an abnormal condition of a passenger who can ascertain sudden cardiac arrhythmia or cardiac dysfunction or drowsiness of a passenger in advance and secure the safety of the vehicle while driving.

Another object of the present invention is to provide a system for determining an anomalous condition of a passenger and a method of determining the anomalous state of a passenger by recognizing biometric collection information with low false positives by using two or more contactless / contact sensors driven in different forms.

A sensor unit including a first sensor for sensing a passenger's bio-signal in a first mode and a second sensor for sensing a passenger's bio-signal in a second mode according to an embodiment of the present invention; And an abnormal state determiner for determining whether the occupant is in an abnormal state using the bio-signal.

A method of determining an anomalous condition of an occupant according to an embodiment of the present invention includes: measuring a vital signal of a passenger in a different manner using first and second sensors and converting the measured vital signal into a digital signal; Collecting a reference signal in a steady state for a first time through the bio-signal; Measuring a bio-signal of a passenger for a second time after the first time elapses; And comparing the reference signal and the bio-signal measured during the second time to determine whether the occupant is in an abnormal state.

According to an embodiment of the present invention, a system for determining an anomalous state of a passenger and a method for determining the anomalous state of a passenger can ascertain the abrupt cardiac arrhythmia, cardiac dysfunction or drowsiness of a passenger in advance and secure the safety of the vehicle while driving.

In addition, a non-contact type bio-signal detection sensor capable of collecting by minimizing the influence of human motion and a contact type bio-signal detection sensor capable of maintaining a constant contact state are provided at the same time, Can be improved.

In addition, since the signals collected in the contact method are integrated in the non-contact bio-signal detector in the idle band in the radio wave receiving band of the non-contact type bio-signal detector, Signal processing is possible.

In addition, through analysis of characteristic peak periods of bio-signals appearing in the time domain, it is possible to grasp the driver's emergency situation and drowsiness, and it is possible to cope with various situations such as activation of safe driving mode and alarm notification for a sufficient time.

1 is a view showing a sensor unit according to an embodiment of the present invention
2 is a diagram illustrating a system for determining an abnormal condition of a driver according to an embodiment of the present invention.
3 is a diagram illustrating a first filtering of a signal according to an embodiment of the invention.
4 is a diagram showing a part of a system for determining an abnormal condition of a driver according to an embodiment of the present invention.
5 is a diagram showing the intervals between signals of processed data according to an embodiment of the present invention.
FIG. 6 is a diagram for explaining a method of determining a drowsy state of a driver according to an embodiment of the present invention.
7 is a flowchart illustrating a method of determining an abnormal state of a driver according to an embodiment of the present invention.
8 is a flowchart illustrating a method of converting a physical activity signal of a driver into a digital signal according to an embodiment of the present invention.

It is noted that the technical terms used in the present invention are used only to describe specific embodiments and are not intended to limit the present invention. In addition, the technical terms used in the present invention should be construed in a sense generally understood by a person having ordinary skill in the art to which the present invention belongs, unless otherwise defined in the present invention, Should not be construed to mean, or be interpreted in an excessively reduced sense. In addition, when a technical term used in the present invention is an erroneous technical term that does not accurately express the concept of the present invention, it should be understood that technical terms can be understood by those skilled in the art. In addition, the general terms used in the present invention should be interpreted according to a predefined or prior context, and should not be construed as being excessively reduced.

Furthermore, the singular expressions used in the present invention include plural expressions unless the context clearly dictates otherwise. In the present invention, terms such as "comprising" or "comprising" and the like should not be construed as encompassing various elements or various steps of the invention, Or may further include additional components or steps.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings, wherein like reference numerals refer to like or similar elements throughout the several views, and redundant description thereof will be omitted.

In the following description, well-known functions or constructions are not described in detail since they would obscure the invention in unnecessary detail. It is to be noted that the accompanying drawings are only for the purpose of facilitating understanding of the present invention, and should not be construed as limiting the scope of the present invention with reference to the accompanying drawings.

1 is a view showing a sensor unit according to an embodiment of the present invention. 2) according to an embodiment of the present invention may include a first sensor 110 and a second sensor 120.

The first sensor 110 and the second sensor 120 can collect biometric information of a passenger, for example, a driver, and can collect biometric information of a driver in a different manner. In the embodiment of the present invention, the driver is described as an example, but the present invention is not limited thereto. The biological signal may include at least one of heart rate and respiratory rate.

The first sensor 110 and the second sensor 120 can sense not only heart beat but also breathing.

The first sensor 110 may be a non-contact type radar sensor that transmits and receives an electromagnetic wave signal in the RF band and operates at a specific position to prevent a noise signal due to a driver's movement from entering the sensor .

The first sensor 110 is disposed near the headrest of the car seat, for example, under the headrest, and transmits a radar signal in the neck direction at the back of the driver's neck to detect movement of the carotid artery by the heartbeat .

The signal to noise ratio (SNR) can be increased because the radar sensor signal directly touches the body skin. In particular, the throat is the place where the carotid artery is located and the large signal waveform can be observed by the heartbeat.

Since the effects of the user's movement may be different from each other, the first sensors 110 may be configured in different arranging directions. For example, when two sensors are used, one sensor may be disposed on the rear side of the driver and a sensor may be disposed on the side to prevent saturation of collected signals due to movement.

In addition, the first sensor 110 can sense not only the heart rate, but also breathing. The first sensor 110 may be connected to the vehicle by wire to receive power.

The second sensor 120 may be configured in a contact manner such as a piezoelectric element, a capacitor coupling pad, or the like. By sensing the movement of the heartbeat from the surface, it can be placed in a position where it is kept in contact with the user at all times, and can be attached to the seatbelt and positioned near the driver's heart to collect vital signs.

The second sensor 120 may form a plurality of contact pads in the same substrate and improve the signal-to-noise ratio using the sum or difference of the signals appearing between the pads.

The second sensor 120 may be battery powered and may employ other energy supply and collection techniques such as wireless power transmission and energy harvesting and may also provide sustainable power externally along a flexible lead, A signal may be detected.

The second sensor 120 may include a communication unit, and may transmit the collected biometric information to the outside through the communication unit. For example, to the first sensor 110.

The second sensor 120 can be fixed to the seatbelt using a fixing member such as a clip, and the position of the second sensor 120 can be changed according to the body shape of the driver.

The signal collected by the second sensor 120 is transmitted to the first sensor 11 through a flexible communication line on the wireless communication system or the safety belt without performing signal processing for low power driving. In this case, in order to distinguish the signal from the signal collected by the first sensor, a signal including a different modulation signal or a different carrier frequency is required.

Therefore, the transmission frequency is set to a frequency band in which the bio-signal detection signal can not exist in the collection frequency band of the first sensor. For example, in the case of a radar sensor for detecting a biological signal having a frequency band of 2.4 GHz to 2.5 GHz, a transmission frequency of 2.49 GHz can be set when the biological signal from the second sensor has a bandwidth of 5 MHz.

FIG. 2 is a diagram illustrating a sleeping (or emergency) state determination system of a driver according to an embodiment of the present invention, and FIG. 3 is a diagram illustrating a first filtering of a signal according to an embodiment of the present invention. 4 is a diagram showing a part of an abnormality state determination system for a driver according to an embodiment of the present invention.

The system for determining an abnormal condition of a driver according to an embodiment of the present invention may include a sensor unit 100, a control unit 200, and a service providing unit 300.

The sensor unit 100 may include a plurality of sensors that operate in different ways, that is, a contact mode and a non-contact mode, by sensing a biosignal of a driver. The sensor unit 100 may include a first sensor 110 and a second sensor 120 and may transmit the measured biometric signal to the controller 200 through a communication unit.

The control unit 200 receives the biological signal from the sensor unit 100, converts the biological signal into a digital signal, compares the biological signal with computed and stored data, determines the state of the driver, And to output a signal for controlling.

The controller 200 includes a communication unit 210, a first filter unit 215, a mix unit 220, a second filter unit 230, a correlator 240, an analog / digital conversion unit 250, (260) and an abnormal state determination unit (270).

The communication unit 210 receives the biosignal from the sensor unit 100. Antenna, but it is not limited thereto.

The first filter unit 215 is connected to the communication unit 210 and receives the bio-signal measured by the sensor unit 100 from the communication unit 210.

The biological signal measured by the sensor unit 100 may be a signal measured by the first sensor and the second sensor, that is, the non-contact sensor and the contact sensor. The first filter unit 215 may measure the contact signal and the non- Can be added.

The first filter unit 215 filters the summed signals to select the frequencies of the touch sensor signal and the non-contact sensor signal. Therefore, the frequency output from the first filter unit 215 is the sum of the frequency fc of the touch sensor signal and the frequency fnc of the non-contact sensor signal.

The mixer unit 220 is connected to the first filter unit 215 and outputs f _LO1 and f _LO2 , which are frequencies near fc and fnc, respectively, to fc + fnc, which is an output signal of the first filter unit 215, Add and subtract. f _LO1 and f _LO2 are reference frequencies of fc and fnc, respectively, in order to modulate the measured signals fc and fnc.

Therefore, a value obtained by adding / subtracting f _LO1 to fc + fnc becomes (fc + f _LO1 ) + (fnc + f _LO1 ). That is, (fc- _fLO1 ) + (fnc- _fLO1 ) + (fc + _fLO1 ) + (fnc + _fLO1 ). Further, a value obtained by adding / subtracting f _LO2 to fc + fnc becomes (fc + f _LO2 ) + (fnc + f _LO2 ). That is, (fnc- _fLO2 ) + (fc- _fLO2 ) + (fc + _fLO2 ) + (fnc + _fLO2 ).

The second filter unit 230 may include two low-pass filters, and filters the frequency components of the lowest frequency band among the signals output from the mixer 220. Therefore, the signals output from the second filter unit 230 become (fc-f _LO1 ) and (fnc-f _LO2 ).

Since the first sensor 110 and the second sensor 120 measure the same person's biological signal in the same time zone, data of the same time is input when time synchronization is performed. The correlator 240 receives (fc-f _LO1 ) And (fnc-f _LO2 ), and outputs the same signal to the analog-to-digital converter 250. Thus, the reliability can be improved.

The analog-to-digital conversion unit 250 converts the analog signal into a digital signal. The anomaly state determination unit 270 computes the digital signal converted by the analog-to-digital conversion unit 250, To determine the current driver status during driving.

The storage unit 260 may store data and set values related to the bio-signal obtained using the sensor unit.

The service providing unit 300 includes not only a simple alarm function but also can be connected to an autonomous running system of the vehicle to control the vehicle.

That is, the controller receives the abnormal state signal determined by the controller and displays a warning to the driver. For example, when the driver is determined to be in a drowsy state or an abnormal state, an alarm service is provided to the driver in such a manner that the windshield is opened or the music is reproduced over a certain volume.

In addition, when it is determined that the driver is in the drowsy state or the abnormal state, the autonomous operation mode may be changed.

3 is a diagram illustrating a first filtering of a signal according to an embodiment of the invention. As shown, the signal filtered by the first filter unit 215 may have a predetermined bandwidth and may filter the contact sensor signal and the non-contact sensor signal.

5 is a diagram showing the intervals between signals of processed data according to an embodiment of the present invention. 5 (a) shows the bio-signals measured by the sensor unit 100 and FIG. 5 (b) shows the peak intervals L1, L2, L3, L4 ...).

It is shown that the heart rate decreases as the peak intervals (L1, L2, L3, L4 ...) are widened, and the heart rate increases as the peak intervals (L1, L2, L3, L4 ...) .

FIG. 6 is a diagram for explaining a method of determining a drowsy state of a driver according to an embodiment of the present invention. The abnormal state determination unit 270 determines that the driver is in a drowsy state when the peak intervals L1, L2, L3, L4, ... are wider than the reference value.

That is, if the signal interval RR is HBR _reference And the ratio value of the HBR _current exceeds the reference value, it can be determined that the driver is in a drowsy state.

FIG. 7 is a view for explaining a method of determining a sleeping state of a driver according to an embodiment of the present invention.

First, the physical activity of the driver is measured through the first and second sensors and converted into a digital signal (S100). The first and second sensors may be configured to be in contact and non-contact, and may include filtering and frequency summation. S100 will be described in detail with reference to FIG.

Next, a reference signal in a steady state is collected for a first time through the measurement signal converted in S100 (S200). In step S200, reference values of heart rate and respiration rate are determined. The first time may be a predetermined time from the start of the operation, and the reference signal may be based on the heart rate and the breath count collected during the first time.

Accordingly, even when the driver changes the biorhythm even when the driver changes the biorhythm, the change of the biological signal is detected using the steady state at the initial stage of the operation as the reference signal. In addition, it is possible to integrate analysis using personal data for each driver.

In the frequency domain, the time domain signal is detected in the form of FFT, and the heart rate and respiration rate per minute can be grasped, and the driver state can be grasped by sensing the heart rate and respiratory rate change over a certain period of time.

In the frequency domain conversion, since the frequency band containing the biomedical signals of the heart rate and respiratory rate per minute is very low, sufficient data is required in advance, but new data is accumulated and analyzed in the previous data to minimize the analysis time required for signal processing .

Next, a bio-signal according to physical activity for a second time period, for example, a breathing volume and a heart rate, is measured and calculated (S300). The difference from S200 is that S200 is to measure the heart rate and respiratory rate collected during a certain period of time after starting the operation, whereas the second time in S300 means a certain time collected after the first time in S200 to be. The driver's real-time heart rate and respiration rate can be measured by S300.

Next, it is determined whether the current heart rate and respiration rate measured in S300 exceeds a predetermined range in comparison with the reference heart rate and respiration rate measured in S200 (S400). If the change is out of a predetermined range, A notification service is performed (S500).

In FIG. 6, it is determined that the current heart rate and respiration rate measured at S300 are drowsy when the measured heart rate and respiration rate are lower than the reference heart rate and respiration rate measured at S200. However, the present invention is not limited thereto. For example, if the current heart rate and respiration rate measured at S300 is higher than the reference heart rate and respiratory rate measured at S200, it may be determined that the driver's heart rate increases due to a traffic accident or the like.

8 is a flowchart illustrating a method of converting a physical activity signal of a driver into a digital signal according to an embodiment of the present invention.

First, the signals collected through the first and second sensors are summed and firstly filtered by a filter having a predetermined bandwidth (S110). The frequency of the touch sensor signal and the non-contact sensor signal is selectively filtered by the first filtering.

Next, the reference frequency is added to and outputted from the signal filtered by S110 (S120). The reference frequencies of the contact sensor signal and the non-contact sensor signal are respectively added to and outputted from the signals collected through the first and second sensors by S120.

Next, the frequency output in step S120 is secondly filtered (S130). A touch sensor frequency (fc) of the f _LO1 of the reference frequency of the touch sensor signal subtraction to the frequency (fc-f _LO1) and the measured non-contact sensor frequency (fnc) measured by S130 of the reference frequency of the non-contact sensor signal f _LO2 (Fnc-f _LO2 ) is output.

Next, the correlator extracts the intersection of the first and second signals outputted in S130 (S140), and converts it into an analog digital signal (S150).

The flow shown in FIG. 8 corresponds to FIG. 4, S110 is the first filter 215, S120 is the mixer 220, S130 is the second filter 230, (240), respectively.

As described above, according to an embodiment of the present invention, a system for determining an abnormal condition of a passenger and a method of determining the same can prevent a sudden cardiac arrest or cardiac malfunction or drowsiness of a passenger in advance, .

In addition, a non-contact type bio-signal detection sensor capable of collecting by minimizing the influence of human motion and a contact type bio-signal detection sensor capable of maintaining a constant contact state are provided at the same time, Can be improved.

The features, structures, effects and the like described in the foregoing embodiments are included in at least one embodiment of the present invention and are not necessarily limited to one embodiment. Further, the features, structures, effects, and the like illustrated in the embodiments may be combined or modified in other embodiments by those skilled in the art to which the embodiments belong.

Therefore, it should be understood that the present invention is not limited to these combinations and modifications. While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is clearly understood that the same is by way of illustration and example only and is not to be construed as limiting the scope of the present invention. It can be seen that various modifications and applications are possible. For example, each component specifically shown in the embodiments may be modified and implemented. It is to be understood that the present invention may be embodied in many other specific forms without departing from the spirit or essential characteristics thereof.

100:
110: first sensor
120: second sensor
200:
210:
215:
220: Mix part
230: second filter portion
240: Correlator
250: analog-to-digital conversion section
260:
270:
300: Service Offering

Claims (12)

A sensor unit including a first sensor for sensing a passenger's bio-signal in a first mode and a second sensor for sensing a passenger's bio-signal in a second mode; And
And an abnormal state determiner for modulating the frequency of the first type of living body signal and the second type of living body signal to modulate the frequency of the living body signal and then determining whether the occupant is abnormal based on the selected signal, Of the occupant.
The method according to claim 1,
Wherein the bio-signal includes at least one of a heart rate and a respiratory rate.
The method according to claim 1,
And a service providing unit for receiving the abnormal condition signal determined by the control unit and warning the passenger of the abnormal condition signal.
The method according to claim 1,
Wherein the first sensor is a non-contact type, and the second sensor is a contact type.
5. The method of claim 4,
Wherein the first sensor is a radar sensor mounted on a headrest and irradiates a radar near a rear end of the occupant, and the second sensor is a piezoelectric element or a capacitor coupling pad mounted on a seatbelt Judgment system.
The method according to claim 1,
Wherein the control unit includes a first filter unit for outputting a summed frequency of the first sensor signal and the second sensor signal.
The method according to claim 1,
The control unit of the first sensor and the reference frequency of fc and fnc respectively to the frequency (fc + fnc) the summation of the second sensor signal is f _LO1 and the acceleration a _{f LO2 (fc-f LO1)} + (fnc-f LO1 ) + (including fc + f _LO1) + (fnc + f _LO1) and (fnc-f _LO2) + (fc-f _LO2) + (fc + f _LO2) + (fnc + f _LO2), the output mix section for Wherein the vehicle occupant is a driver.
8. The method of claim 7,
Wherein the controller is _{(fc-f LO1) + (} fnc-f LO1) + (fc + f LO1) + (fnc + f LO1) and _{(fnc-f LO2) + (} fc-f LO2) the mix of negative output signal _{+ (fc + f LO2) +} (fnc + f LO2) for filtering (fc-f _LO1) and (fnc-f _LO2) to output the abnormality determination system of the occupant, characterized in that it comprises second filter unit for.
9. The method of claim 8,
The control unit and the second filter section with (fc-f _LO1) and (fnc-f _LO2) an abnormal state determination system of the occupant, characterized in that it comprises a correlator for outputting a signal matching by comparing the output from.
Measuring the passenger's vital sign in a different manner using the first and second sensors and converting it into a digital signal;
Collecting a reference signal in a steady state for a first time through the bio-signal;
Measuring a bio-signal of a passenger for a second time after the first time elapses; And
Comparing the reference signal and the bio-signal measured during the second time to determine whether the occupant is in an abnormal state,
Wherein the judging step comprises modulating the bio-signal of the first scheme and the bio-signal of the second scheme by frequency-summing and modulating the bio-signal, and judging whether or not the abnormal state is based on the selected signal,
Wherein the reference signal is based on a steady state at the initial stage of operation of the same driver.
11. The method of claim 10,
The step of measuring and converting the passenger's bio-signal into a digital signal in a different manner using the first and second sensors comprises:
Summing and first filtering the bio-signals collected using the first and second sensors; And
And adding and outputting a reference frequency of a signal measured using the first and second sensors to the first filtered signal, respectively, and outputting the resultant signal.
12. The method of claim 11,
And a step of adding and outputting a reference frequency of a signal measured using the first and second sensors to the first filtered signal, respectively,
And outputting a frequency at which the reference frequency is subtracted from the frequency measured by the first sensor through the second filtering and a frequency at which the reference frequency is subtracted from the frequency measured by the second sensor. State determination method.

Images