KR20190023555A - Apparatus and method of determining driver state - Google Patents

Abstract

An apparatus for determining a driver state includes: a driving environment sensing unit including a peripheral distraction measuring sensor for receiving a driver image during driving of a vehicle to measure peripheral distraction information, a biometric sensor for measuring driver biometric information, and a vehicle measuring sensor for measuring vehicle state information; a driver recognizing unit for recognizing the driver based on the driver image; a personalized driver model selecting unit for selecting a personalized driver model when the driver is recognized; a driver running state estimating unit for estimating a driver state by reflecting a state probability value and a state reference value, which are acquired on the basis of perimeters on the peripheral distraction information, the driver biometric information, or the vehicle state information, to the personalized driver model; and a personalized driver model updating unit for updating a state probability table and state reference table including the state probability value and the state reference value, respectively, through reinforced learning on the driver state estimation.

Description

[0001] APPARATUS AND METHOD OF DETERMINING DRIVER STATE [0002]

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a driver state determination technique, and more particularly, to a driver state determination device and method capable of effectively determining a driver's state by providing a personalized driver model.

One of the reasons for the high proportion of traffic accidents is human error. Careful attention to safe driving leads to mistakes by the driver, delays in information processing, and increases the possibility of traffic accidents. Accordingly, various research institutes and commercial vehicle companies are developing a warning system according to the driver's situation.

Korean Patent No. 10-1711027 relates to a method and apparatus for providing a driver-customized service for a vehicle, and provides driver-customized services through environmental information such as information obtained through sensors provided in the vehicle, weather information, and driver's medical information .

Korean Patent No. 10-1449326 relates to an apparatus and method for calculating driving concentration and a vehicle collision warning system and method using the apparatus and method. The network communication unit obtains the acceleration of the own vehicle, the relative information A noise removing unit that removes noise from the acceleration of the own vehicle acquired by the measuring unit and the network communication unit and removes noise at a relative speed with the preceding vehicle measured by the relative information measuring unit, A correlation value calculating unit for calculating a correlation value on the basis of the acceleration of the preceding vehicle and a relative speed with respect to the preceding vehicle, and a driving concentration detector for detecting, as a driving concentration, a time point at which the maximum correlation value is calculated from the correlation values calculated by the correlation value calculating unit. Which is a feature of the present invention.

Korean Patent No. 10-1711027 (Feb. 22, 2017) Korean Patent No. 10-1449326 (Oct. 1, 2014)

An embodiment of the present invention is to provide an apparatus and method for determining a driver state that can effectively determine a driver's state by providing a personalized driver model.

An embodiment of the present invention is to provide an apparatus and method for determining a driver state that can prevent a driver's abnormal state from being determined as a driver's normal state.

An embodiment of the present invention provides an apparatus and method for determining a driver state that can provide a personalized driver model suitable for a driver by updating a state probability table and a state reference table through learning about driver state estimation.

Among the embodiments, the driver's condition determining device includes a peripheral dispersion measuring sensor that receives the driver's image in the running process of the vehicle and measures the driver's peripheral dispersion information, a biometric sensor that measures the driver's biometric information, A personalized driver model selection unit for selecting a personalized driver model when the driver is recognized, and a personalized driver model selection unit for selecting the personalized driver model based on the driver's perimeter And a driver driving state estimation unit for performing driver state estimation by reflecting a state probability value and a state reference value obtained based on the dispersion information, the driver's biometric information, or the index related to the vehicle state information.

In one embodiment, the driver state determination apparatus may further include a state probability table including the state probability value and the state reference value, respectively, through a reinforcement learning on the driver state estimation, and a personalized driver model update unit for updating the state reference table have.

In one embodiment, the ambient variance measurement sensor may measure the perimeter dispersion information of the driver based on at least one of an average size of a Percentage Closure of Eyes (PERCLOS), a face direction, or an eye blinking frequency.

In one embodiment, the biometric sensor may measure the driver's biometric information based on at least one of breathing rate, heart rate, skin electrical response, or skin temperature.

In one embodiment, the personalized driver model selection unit may select a specific personalized driver model corresponding to the driver image from among a plurality of personalized driver models.

In one embodiment, the driver driving state estimator may generate a normal distribution based on the state probability value to perform the driver state estimation.

In one embodiment, the driver driving state estimating unit may distinguish the driver normal state from the driver abnormal state based on the state reference value in the generated normal distribution.

In one embodiment, the driver's driving state estimator may calculate a steady state probability or an abnormal state probability in the driver's steady state of the normal distribution, and calculate a steady state probability or an abnormal state probability in the driver's abnormal state of the normal distribution .

Among the embodiments, the driver state determination method is performed in the driver state determination apparatus. The method includes the steps of: (a) a peripheral dispersion measuring sensor for receiving the driver image in the running process of the vehicle and measuring the driver's peripheral dispersion information, a biometric sensor for measuring driver's biometric information, and a vehicle measurement sensor for measuring vehicle status information (B) a driver recognizing step of recognizing a driver based on the driver image; (c) a personalized driver model selecting step of selecting a personalized driver model when the driver is recognized; (d) A driver's driving state estimation step of performing driver's state estimation by reflecting a state probability value and a state reference value obtained on the basis of the driver's peripheral dispersion information, the driver's biomedical information, or the vehicle state information on the model, and (e) Wherein the state probability value and the state reference value are stored in the storage unit That includes personalized driver model updating step of updating the probability state table and state standards table.

The disclosed technique may have the following effects. It should be understood, however, that the scope of the disclosed technology is not to be construed as limited thereby, as it does not imply that a particular embodiment should include all of the following effects or merely include the following effects.

The apparatus and method for determining a driver state according to an embodiment of the present invention can effectively determine the state of a driver by providing a personalized driver model.

The apparatus and method for determining driver's condition according to an embodiment of the present invention can prevent the driver's abnormal state from being determined as the driver's normal state.

The apparatus and method for determining a driver state according to an embodiment of the present invention can provide a personalized driver model suitable for a driver by updating the state probability table and the state reference table through learning about driver state estimation.

1 is a view for explaining an apparatus for determining a driver state according to an embodiment of the present invention.
FIG. 2 is a view for explaining a driver state determination method performed by the driver state determination apparatus shown in FIG. 1. FIG.
3 is a diagram illustrating an example of a normal distribution generated based on a status probability value and a status reference value according to an embodiment of the present invention.

The description of the present invention is merely an example for structural or functional explanation, and the scope of the present invention should not be construed as being limited by the embodiments described in the text. That is, the embodiments are to be construed as being variously embodied and having various forms, so that the scope of the present invention should be understood to include equivalents capable of realizing technical ideas. Also, the purpose or effect of the present invention should not be construed as limiting the scope of the present invention, since it does not mean that a specific embodiment should include all or only such effects.

Meanwhile, the meaning of the terms described in the present application should be understood as follows.

The terms "first "," second ", and the like are intended to distinguish one element from another, and the scope of the right should not be limited by these terms. For example, the first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component.

It is to be understood that when an element is referred to as being "connected" to another element, it may be directly connected to the other element, but there may be other elements in between. On the other hand, when an element is referred to as being "directly connected" to another element, it should be understood that there are no other elements in between. On the other hand, other expressions that describe the relationship between components, such as "between" and "between" or "neighboring to" and "directly adjacent to" should be interpreted as well.

It is to be understood that the singular " include " or "have" are to be construed as including the stated feature, number, step, operation, It is to be understood that the combination is intended to specify that it does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or combinations thereof.

In each step, the identification code (e.g., a, b, c, etc.) is used for convenience of explanation, the identification code does not describe the order of each step, Unless otherwise stated, it may occur differently from the stated order. That is, each step may occur in the same order as described, may be performed substantially concurrently, or may be performed in reverse order.

The present invention can be embodied as computer-readable code on a computer-readable recording medium, and the computer-readable recording medium includes all kinds of recording devices for storing data that can be read by a computer system . Examples of the computer-readable recording medium include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage, and the like. In addition, the computer-readable recording medium may be distributed over network-connected computer systems so that computer readable codes can be stored and executed in a distributed manner.

All terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs, unless otherwise defined. Commonly used predefined terms should be interpreted to be consistent with the meanings in the context of the related art and can not be interpreted as having ideal or overly formal meaning unless explicitly defined in the present application.

1 is a view for explaining an apparatus for determining a driver state according to an embodiment of the present invention.

1, the driver state determination apparatus 100 includes a driving environment sensing unit 110, a driver recognition unit 120, a personalized driver model selection unit 130, a driver driving state estimation unit 140, An update unit 150, and a control unit 160, and can be designed to transmit and receive electrical signals therebetween.

The driver's condition determining apparatus 100 may correspond to a computing apparatus that can learn a driving state of a driver in real time, dynamically update a personalized driver model, and reflect the driver's state in a driver's state, thereby providing a personalized driver's model. In one embodiment, the driver's condition determining apparatus 100 may be electrically connected to a vehicle or connected via a network, and may be implemented as a control unit of a vehicle included in the interior of the vehicle.

The traveling environment sensing unit 110 includes an ambient dispersion measuring sensor 112 for receiving the driver image in the running process of the vehicle and measuring the dispersion information about the driver, a living body measurement sensor 114 for measuring driver living body information, And may include a variety of other sensors. ≪ RTI ID = 0.0 >

The ambient dispersion measuring sensor 112 may correspond to an image sensor having a structure in which a shape of a subject comes in through a lens and the light is converted into an electric signal and stored in a memory when the image is sensed by the image sensor. The biometric sensor 114 may be embodied as a wearable device that includes a Galvanic Skin Response (GSR) sensor, a Photo-Plethysmography (PPG) sensor, an Electrocardiography (ECG) have.

The vehicle measurement sensor 116 may be a sensor capable of sensing internal information and external information of the vehicle and may be a sensor capable of sensing the driving of the actuators in the vehicle, Unit) to receive information related to driving the vehicle.

The travel environment sensing unit 110 can receive the driver image or the driver image from the ambient dispersion measurement sensor 112 and measure the driver's ambient dispersion information about the driver. The travel environment sensing unit 110 measures the driver peripheral dispersion information based on at least one of the number of blinking times of the driver's eyes, the face direction, and the average size of PERCLOS (Percentage Closure of Eyes) based on the driver's image or the driver's image And the index used to measure driver perimeter dispersion information about the driver may further include various other indicators.

The traveling environment sensing unit 110 can receive the living body signal from the living body measurement sensor 114 and measure the driver living body information. The traveling environment sensing unit 110 can measure driver's living body information based on at least one index of respiratory rate, heart rate, skin electric reaction, or skin temperature, and the index used to measure driver's living body information is And may further include various other indicators.

The traveling environment sensing unit 110 can receive vehicle interior / exterior information from the vehicle measurement sensor 116 and measure vehicle state information. The traveling environment sensing unit 110 can measure the vehicle state information based on at least one of the traveling speed, the acceleration, the steering angle, and the accelerator pedal, and the index (Index) And may further include various indicators.

The driver recognition unit 120 can recognize the driver based on the driver image. Here, the driver image may be generated through the ambient variance measurement sensor 112 and may include a driver face image corresponding to a particular driver.

The driver recognition unit 120 can perform driver registration on the driver based on the driver image. More specifically, the driver recognition unit 120 may acquire the driver image as the driver identifier (ID) in order to perform driver registration, and when the driver image for the driver is generated for the first time, The driver's registration of the driver can be performed based on the image. In one embodiment, when the driver registration for the driver is completed, the driver recognition unit 120 does not perform the driver registration process when the driver boarded the vehicle in the future.

The driver recognition unit 110 may store the driver registration information in a memory included in the vehicle. Here, the driver registration information may include at least one driver registration information generated after performing driver registration based on the driver image. For example, the driver registration information may include a vehicle model name, an administrative A vehicle number, a driver image, and the like.

The personalized driver model selection unit 130 can select a specific personalized driver model when the driver is recognized through the driver recognition unit 110. [ More specifically, the personalized driver model selection unit 130 can select a specific personalized driver model corresponding to the driver image from among the plurality of personalized driver models. In one embodiment, the personalized driver model selection unit 130 selects a personalized driver model corresponding to a driver image of a specific driver from {personalized driver model 1, personalized driver model 2, personalized driver model 3 ... personalized driver model n} Can be selected.

Driver 1 Driver image 1 Personalized Driver Model 1 Driver 2 Driver image 2 Personalized Driver Model 2 Driver 3 Driver image 3 Personalized Driver Model 3 Driver n Driver image n Personalized Driver Model n

For example, if the driver image received through the travel environment sensing unit 110 corresponds to the driver image 3, the personalized driver model selection unit 130 can select the personalized driver model 3 corresponding to the driver image.

The driver driving state estimating unit 140 can perform driver state estimation. More specifically, the driver's driving state estimating unit 140 reflects the state probability value and the state reference value obtained based on the driver's perimeter dispersion information, driver's biometric information, or vehicle state information on the personalized driver model to perform driver state estimation can do. For example, the indicators about the driver's perimeter dispersion information, the driver's biometric information, or the vehicle state information may include, but are not necessarily limited to, the following variables.

≪ Example of indicator about vehicle status information >

VD: longitudinal velocity variation

AxD: longitudinal acceleration variation

GM: Average size of accelerator pedal

GC: Average number of acceleration pedals

GD: Acceleration pedal change

StD: Steering angle variation

StRR: steering angle reversal rate

AyD: transverse acceleration variation

≪ Example of index concerning dispersion information about driver '

HCP: Percent facial orientation is frontal

BlC: Average number of eye blinks

PM: perclos average size

C3P: Percentage of conversations are 3 or more

≪ Example of indicator about driver ' s biometric information &

HrHP: High heart rate

HrvIP: Rate of increase in heart rate change

RrHP: Rate of breathing is high

RrvIP: Rate of increase in respiratory rate change

GSRHP: rate of high skin electrical response

GSRvIP: rate of increase in skin electrical response change

BtHP: rate of skin temperature is high

BtvIP: Rate of increase in skin temperature change

The driver driving state estimating unit 140 can acquire the state probability value and the state reference value on the basis of the indicator relating to the driver perimeter dispersion information, driver's biometric information, or vehicle state information in the personalized driver model. The state probability value and the state reference value may be set differently for a plurality of drivers, and each value may be changed by learning the ambient dispersion information, the driver's biometric information, or the vehicle state information in the driving process of the vehicle.

For example, the driver driving state estimating unit 140 may obtain a state reference table including a state probability table and a state reference value including a state probability value. In one embodiment, each of the state probability table and state reference table may be designed based on signal detection theory.

Table 2 Status probability table

<Table 3> Status reference table

The driver driving state estimating unit 140 may generate a normal distribution based on the state probability value to perform driver's driver state estimation. The driver's driving state estimating unit 140 can distinguish the driver's normal state from the driver's abnormal state based on the state reference value in the generated normal distribution. The driver abnormal condition may include, but is not limited to, driver drowsiness, driver distraction, or driver overload. Here, the driver's sleepiness refers to a state in which the driver has been deprived of the attention necessary for the first driving due to the physiological phenomenon of the driver who is not the intention of the driver, and the dispersion of the driver is the driver's intention, The driver's intentional overloading of the driver is caused by intentional 'outside environment of the vehicle' or 'interaction with the inside of the vehicle'. It refers to a state that has lost the attention necessary for the first run.

Hereinafter, a normal distribution generated on the basis of the status probability value and the status reference value according to an embodiment of the present invention will be described with reference to the drawing of FIG.

3, the driver driving state estimating unit 140 may generate a normal distribution 300 based on the state probability value and the normal distribution 300 may generate the driver normal state 310 And an operator abnormal state 320. [0050]

More specifically, the driver driving state estimating unit 140 can determine the driver state through a driver state estimation algorithm (Driver State Estimation Algorithm). The meanings of the variables shown in FIG. 3 are as follows.

Noise: Operator steady state data distribution

Signal + Noise: Operator abnormal condition data distribution

Criterion: status reference value

"no" region: an area judged to be the driver's normal state based on the reference value

"yes" region: an area that judges the driver to be in an abnormal state based on the reference value

Red area 322: If the driver state estimation algorithm determines that the driver is in an abnormal state,

Blue color area (324): When the driver state estimation algorithm judges that the driver is in an abnormal state,

In one embodiment, the driver driving state estimator 140 may calculate the probability 312 that the driver state is normal and the probability 314 that the driver state is abnormal in the driver steady state region 310 of the normal distribution 300 have. For example, the driver driving state estimating unit 140 may reflect the calculated four state probabilities in the state probability table. Accordingly, the driver's driving state estimating unit 140 can perform the driver's state estimation on the driver by selecting the corresponding probability from the normal distribution divided into the driver's normal state region 310 and the driver's abnormal state region 320.

1 will be described again.

The driver's running state estimation unit 140 can calculate four probabilities in the normal distribution. More specifically, the driver's running condition estimating unit 140 estimates the probability of a steady state in a driver's normal state of a normal distribution, a probability that an abnormal state occurs in a driver's normal state of a normal distribution, a normal state probability in a driver's abnormal state of a normal distribution, It is possible to calculate the probability of the abnormal state in the driver abnormal state of the distribution.

The personalized driver model updating unit 150 may update the state probability table and the state table including the state probability value and the state reference value through reinforcement learning about the driver state estimation. Here, reinforcement learning refers to learning in which an agent defined in an environment selects an action or sequence of actions that maximizes compensation among selectable behaviors by recognizing the current state.

In one embodiment, the personalized driver model update unit 150 may perform reinforcement learning on the driver state estimation in the driving process of the vehicle. For example, the personalized driver model updating unit 150 can recognize that the driver state is the driver abnormal state corresponding to 'driver sleepiness' through reinforcement learning, and reflects the recognized driver state result to the personalized driver model The personalized driver model can be updated.

The driver state determination unit (not shown) can determine the driver's state based on the driver state estimation result performed by the driver state estimation unit 140. In one embodiment, the driver state determiner can analyze the information measured through the plurality of sensors on the basis of the state probability value and the state reference value on the normal distribution, thereby determining which region corresponds to the normal distribution, The state can be determined.

The control unit 160 can control the overall operation of the driver condition determining apparatus 100 and includes a driving environment sensing unit 110, a driver recognition unit 120, a personalized driver model selection unit 130, (140) and the personalized driver model update unit (150).

FIG. 2 is a view for explaining a driver state determination method performed by the driver state determination apparatus shown in FIG. 1. FIG.

Here, FIG. 2 assumes that the driver's registration of the driver is completed for the convenience of explanation, and the personalized driver model is provided for each driver.

The driver recognizing unit 120 recognizes the driver based on the driver image received from the ambient dispersion measuring sensor 112 that photographs the driver when the driver boarded the vehicle (step S210). In one embodiment, the driver recognition unit 120 can select a personalized driver model corresponding to the driver image through the personalized driver model selection unit 130 when the driver's recognition is successful.

The personalized driver model selection unit 130 can select the personalized driver model when the driver is recognized (step S220). More specifically, the personalized driver model selection unit 130 can select a specific personalized driver model corresponding to the driver image from among the plurality of personalized driver models.

The traveling environment sensing unit 110 may receive the driver image from the ambient dispersion measuring sensor 112 in the traveling process of the vehicle and measure the driver's environment dispersion information and measure the driver's living body information through the living body measurement sensor 114 And can measure the vehicle status information through the vehicle measurement sensor 116 (step S230). In one embodiment, the travel environment sensing unit 110 may reflect the information measured by the plurality of sensors in the driving process of the vehicle to the driver state estimation algorithm.

The driver driving state estimating unit 140 may perform driver state estimation on the selected personalized driver model by reflecting the state probability value and the state reference value acquired based on the driver perimeter dispersion information, the driver's living body information, or the vehicle state information (Step S240). The personalized driver model updating unit 150 may update the state probability table and the state reference table each including the state probability value and the state reference value through reinforcement learning about the driver state estimation (step S250).

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as set forth in the following claims And changes may be made without departing from the spirit and scope of the invention.

100: Driver status judging device
110: traveling environment sensing unit 112: ambient dispersion measuring sensor
114: Biometric sensor 116: Vehicle measurement sensor
120: driver recognition unit 130: personalized driver model selection unit
140: driver driving state estimating unit 150: personalized driver model updating unit
160:

Claims (9)

A traveling environment sensing unit including a peripheral dispersion measuring sensor for receiving the driver image in the traveling process of the vehicle and measuring dispersion information about the driver, a living body measuring sensor for measuring driver's living body information, and a vehicle measuring sensor for measuring vehicle state information;
A driver recognition unit for recognizing a driver based on the driver image;
A personalized driver model selection unit for selecting a personalized driver model when the driver is recognized; And
A driver driving state estimating unit for estimating driver state by reflecting a state probability value and a state reference value obtained on the basis of the driver perimeter dispersion information, the driver's biometric information, or the vehicle state information on the personalized driver model, State determination device.
The method according to claim 1,
Further comprising a personalization driver model updating unit for updating a state probability table and a state reference table each including the state probability value and the state reference value through reinforcement learning regarding the driver state estimation.
The apparatus as claimed in claim 1, wherein the ambient dispersion measuring sensor
Wherein the driver perimeter dispersion information is measured based on at least one of an average size, a face direction, and an eye blinking frequency of PERCLOS (Percentage Closure of Eyes).
The biometric sensor according to claim 1, wherein the biometric sensor
Wherein the driver's biometric information is measured based on at least one of respiratory rate, heart rate, skin electrical response, or skin temperature.
The method of claim 1, wherein the personalized driver model selection unit
And selects a specific personalized driver model corresponding to the driver image from among the plurality of personalized driver models.
The driving force control apparatus according to claim 1,
And generates a normal distribution based on the status probability value to perform the driver state estimation.
7. The apparatus of claim 6, wherein the driver's driving state estimating unit
And discriminates the driver's normal state from the driver's abnormal state based on the state reference value in the generated normal distribution.
8. The apparatus of claim 7, wherein the driver's driving state estimating unit
And calculates a steady state probability or an abnormal state probability in the driver normal state of the normal distribution and calculates a steady state probability or an abnormal state probability in the driver abnormal state of the normal distribution.
In a driver state judging method performed by a driver state judging device,
(a) a traveling environment sensor including a peripheral dispersion measuring sensor for receiving the driver's image in the traveling process of the vehicle and measuring the dispersion information about the driver, a living body measuring sensor for measuring driver's living body information, and a vehicle measuring sensor for measuring the vehicle condition information step;
(b) a driver recognizing step of recognizing a driver based on the driver image;
(c) a personalized driver model selection step of selecting a personalized driver model when the driver is recognized;
(d) a driver driving state estimating step of performing driver state estimation by reflecting the state probability value and the state reference value obtained on the basis of the driver perimeter dispersion information, the driver's biometric information, or the vehicle state information on the personalized driver model, ; And
(e) a personalized driver model updating step of updating a state probability table and a state reference table each including the state probability value and the state reference value through reinforcement learning on the driver state estimation.

Images