KR102479049B1 - The apparatus and method for Driver Status Recognition based on Driving Status Decision Information - Google Patents

Abstract

The present invention relates to an apparatus for recognizing a driver's condition based on driving situation determination information, comprising: an image acquisition unit that obtains driver's face condition information from an image capture device and driving behavior information of a driver from a driver's driving information acquisition device; a driving situation recognition unit that obtains driving situation recognition information of a surrounding vehicle from an external interference recognizing device; a driving concentration determining unit that determines a driver's driving concentration by comparing obtained information such as the driver's face state information, driving situation recognition information of surrounding vehicles, and driver's driving behavior information with learned face vector information; and a driving map determination unit configured to determine a driver driving map corresponding to the determined driver driving concentration.

Description

The apparatus and method for Driver Status Recognition based on Driving Status Decision Information}

The present invention relates to an apparatus and method for recognizing a driver's state based on driving situation determination information, and more particularly, to an apparatus and method for recognizing a driver's state based on fusion information usable in a driving assistance system or an autonomous driving system.

Recently, before the complete growth of autonomous driving systems, safe driving induction technology that applies driver's state recognition technology is being applied for safe driving.

In particular, technology for driving drowsiness, distraction, and unresponsiveness, as well as technology for inducing safe driving by predicting the driver's psychology with facial expressions and patterns and providing warnings or feedback while driving, are developing.

In addition, a technology for determining the behavior of the vehicle by identifying when the vehicle is difficult to drive or when the driver is unable to drive in connection with the autonomous driving system is also being studied.

However, for the commercialization of autonomously driven technology based on the judgment of the system, an advanced recognition rate and safety devices are required. However, at this point in mass production, it is difficult to apply the technology due to problems related to cost and various situations.

In addition, a conventionally commercialized driver assistance system judges a driver driving without stopping for a long time as a driving load, and, like BSD (Blind Spot Detection), the driver does not turn on the turn signal in a situation where the vehicle on the side and rear is in a blind spot. It is a level that generates a warning sound if you try to enter.

In this way, conventionally commercialized driver assistance systems have developed driver state recognition functions such as drowsiness recognition, facial expression recognition, and distraction recognition, but are used only as auxiliary means, making it difficult to provide wide reliability.

In this way, the method of monitoring the driver's condition before fully autonomous driving is commercialized is required to be advanced at the level of the auxiliary system required for safe driving, focusing on interaction at the point of approaching complete autonomous driving. It is expected that this will be a time when active communication between vehicles, drivers and content takes place.

Existing driver state recognition methods allow the determination of the degree of eyes closed, gaze dispersion, braking while driving, acceleration, direction indicators, and steering presence/absence information within previously defined categories.

This includes problems with the recognition rate for judgment other than a few cases defined in advance for the situation or the limitation of the scope for judgment and the occurrence of errors.

The present invention has been made to solve the conventional problems, and an object of the present invention is to advance the driver state recognition technology, which was determined based on only the closed state of the driver's eyes or when the driver did not look forward. An apparatus and method for recognizing a driver's state based on driving situation determination information that recognizes and determines the driver's state based on eye-closing patterns, vehicle information, and driving environment determination information are provided.

Another object of the present invention is to provide an apparatus and method for recognizing a driver's state based on driving situation determination information that can analyze and determine driver's drowsiness and distraction information in real time by fusing vehicle information and surrounding vehicle information.

Another object of the present invention is information obtained by analyzing real-time information on the driver's behavior and face while driving, the influence of the surrounding environment and external light, the location and movement patterns of moving or static obstacles including surrounding vehicles, and risk factors. Based on this, we intend to provide driver-vehicle decision information on safe driving or autonomous driving by analyzing the degree of concentration required of the driver when the current vehicle is driving.

The object of the present invention is not limited to the object mentioned above, and other objects not mentioned will be clearly understood by those skilled in the art from the description below.

In order to achieve the above object, an apparatus for recognizing a driver's condition based on driving situation determination information according to an embodiment of the present invention acquires driver's face condition information from an image capture device and driver's driving behavior information from a driver's driving information acquisition device. acquisition unit; a driving situation recognition unit that obtains driving situation recognition information of a surrounding vehicle from an external interference recognizing device; a driving concentration determining unit that determines a driver's driving concentration by comparing obtained information such as the driver's face state information, driving situation recognition information of surrounding vehicles, and driver's driving behavior information with learned facial vector information; and a driving map determination unit configured to determine a driver driving map corresponding to the determined driver driving concentration.

Here, the driving concentration determination unit determines whether the driver is drowsy or stares straight ahead through the information on the driver's facial condition, and if it is determined that the driver gazes straight ahead, it is preferable to determine a situation according to the degree of danger and the manipulation level. .

and a point selection unit selecting two reference points for setting a reference area in the face image in order to normalize the face image acquired through the camera; and a normalizer for setting a rectangular reference area using the coordinates of the selected reference point and the distance and ratio between the two reference points.

In addition, the point selector determines whether a pupil reference point is selected at the center of the pupil in order to select a pupil reference region in the face image, and when confirming that the reference point is located at the center of the pupil, the size is adjusted from the pupil reference point. A reference guide circle is provided to the user, and when setting selection is completed, the corresponding reference guide circle is set as a pupil reference area.

Further, the point selection unit preferably processes pixels of the remaining region except for the selected pupil reference region as a '0' value.

Meanwhile, according to an embodiment of the present invention, a vector information acquisition unit for acquiring face vector information of a driver including face direction reference gaze information, whether eyes are closed, and a gaze state using vector information between set rectangular reference regions; and a facial state information determining unit that obtains driver connection state information including a driver's operation state of an additional function and determines driver's face state information using the driver connection state information based on the obtained driver's face vector information. .

A method for recognizing a driver's condition based on driving situation determination information according to an embodiment of the present invention includes obtaining driver's face condition information from an image capture device and driver's driving behavior information from a driver's driving information acquisition device; obtaining driving situation recognition information of surrounding vehicles from an external interference detection device; determining a driver's driving concentration by comparing obtained information such as the driver's face condition information, driving situation recognition information of surrounding vehicles, and driver's driving behavior information with learned face vector information; and determining a driver driving map to correspond to the determined driver driving concentration.

The determining of the driver's driving concentration employed in an embodiment of the present invention may include determining whether the driver is drowsy and looking straight ahead through information on the driver's facial condition; and determining a situation according to a degree of danger and an operation level when it is determined that the driver is looking straight ahead.

Meanwhile, in an embodiment of the present invention, in order to normalize a face image acquired through a camera, a reference point selection step of selecting two reference points for setting a reference region in a face image; and a warping and normalization step of setting a rectangular reference area using the coordinates of the selected reference point and the distance and ratio between the two reference points.

The reference point selection step may include selecting a pupil reference point at the center of the pupil to select a pupil reference region in the face image; providing a reference guide circle sized from the pupil reference point; and selecting and normalizing a pupil reference region through the reference guide circle.

Here, in the step of selecting and normalizing the pupil reference region, pixels in the remaining region except for the selected pupil reference region are treated as '0' values.

In addition, according to an embodiment of the present invention, obtaining face vector information of a driver including face direction reference gaze information, whether eyes are closed, and a gaze state using vector information between set rectangular reference regions; obtaining driver connection state information including a driver's operation state of an additional function; and determining facial state information of the driver using driver connection state information based on the obtained driver's face vector information.

According to an embodiment of the present invention, the reference area can be automatically set simply by simply marking a required part of the face image, so that when the driver's normal/abnormal judgment is used as data for analysis in the future, , there is an advantage that the reference area can be normalized and collected.

1 is a functional block diagram illustrating an apparatus for recognizing a driver's condition based on driving situation determination information according to an embodiment of the present invention.
2 and 3 are reference views illustrating selection of a reference point to detect a facial area in one embodiment of the present invention.
4 is a reference diagram for explaining a process for selecting a pupil area in one embodiment of the present invention;
5 is a reference diagram for explaining a process for selecting a region of a face part in one embodiment of the present invention;
6A to 6D are reference diagrams for explaining a driver's frontal recognition criterion according to an embodiment of the present invention.
7A to 7D are reference diagrams for explaining recognition criteria according to driver's eye conditions;
8 is a flowchart illustrating a method of providing recognition data for recognizing a driver's condition based on driving situation determination information according to an embodiment of the present invention.
9 is a flowchart illustrating a method for recognizing a driver's condition based on driving situation determination information according to an embodiment of the present invention.

Advantages and features of the present invention, and methods of achieving them, will become clear with reference to the detailed description of the following embodiments taken in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but will be implemented in various different forms, only these embodiments make the disclosure of the present invention complete, and common knowledge in the art to which the present invention belongs. It is provided to fully inform the holder of the scope of the invention, and the present invention is only defined by the scope of the claims. Meanwhile, terms used in this specification are for describing the embodiments and are not intended to limit the present invention. In this specification, singular forms also include plural forms unless specifically stated otherwise in a phrase. As used herein, "comprises" and/or "comprising" means that a stated component, step, operation, and/or element is the presence of one or more other components, steps, operations, and/or elements. or do not rule out additions.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. 1 shows an apparatus for recognizing a driver's condition based on driving situation determination information according to an embodiment of the present invention, which includes a point selection unit 110, a normalization unit 120, and a vector information acquisition unit 130 to register face vector data information. ), a face state information determination unit 140 and a learning information storage unit 150.

In order to normalize the face image acquired through the camera, the point selector 110 selects the corresponding position as a reference point when two reference points for selecting a reference area (area) are input from the user. For example, As shown in FIG. 2, the reference point is preferably positioned on both sides of the tail of the eye, both sides of the tail of the mouth, and both sides of the nose, but is not limited thereto.

Subsequently, as shown in FIG. 3, the normalization unit 120 uses the coordinates of the selected reference point SP and the distance D between the two reference points and the ratio through [Equation 1] below to determine the square angle. A reference area SA is set for each part of the shaped face.

Thereafter, the normalization unit 120 goes through a process of learning the normalized data of the eyes, the tail of the eyes, the pupil, the closed eyes, the mouth, the tail of the mouth, and the nose and updating the learning information storage unit 150 .

For example, the normalization unit 120 normalizes face data, eye data, closed eye data, pupil data, nose data, and mouth data in the forward 9 directions, and then the face, eyes (left and right divisions), nose, and pupils ( Classification of left and right), closed eyes (classification of left and right), both eye tails and mouth corners are learned and stored in the learning information storage unit 150 .

Here, Imagenormailize is the reference area, D is the distance to the point, x and y are the coordinate values of the reference point, W is the width of the reference area, and H is the height of the reference area.

According to such an embodiment of the present invention, the user can automatically set the reference area (SA) by simply taking a reference point (SP) at a necessary part of the face image, so that the driver's normal/abnormal judgment is analyzed later. When used as data for , there is an advantage in that the reference area can be normalized and collected.

Meanwhile, as shown in FIG. 4 , the point selection unit 110 employed in an embodiment of the present invention determines whether a pupil reference point CP is selected at the center of the pupil, and the pupil reference point is selected at the center of the pupil. , it is determined that the pupil reference region is selected from the face image, and a reference guide circle GC whose size is adjusted from the pupil reference point is provided to the user. Here, it is preferable that the user adjusts the size of the reference guide circle using a mouse wheel or direction buttons.

Then, when the user completes the selection, the point selector 110 selects and normalizes the pupil reference area SA through the reference guide circle GC. Here, it is preferable that the point selection unit 110 processes pixels of the remaining area except for the selected pupil reference area as a '0' value.

Therefore, the point selector employed in one embodiment of the present invention has the advantage of easily normalizing the pupil in the face image.

In order to more accurately and quickly detect eye, nose, and mouth-related parts within the face region, the point selection unit 110 employed in an embodiment of the present invention, as shown in FIG. The nose candidate region and the mouth candidate region are first detected, and then the eye region is detected. Then, as shown in FIG. 5C, the eye designates an iris detection candidate region, and as shown in FIG. 5D, the part of the face is detected with closed eyes.

In addition, the face vector information acquisition unit 130 uses the vector information between the rectangular reference areas SA set in the face image to determine the driver's face vector including the face direction, face direction reference gaze information, whether the eyes are closed, and the gaze state. Acquire information.

For example, in the face vector information acquisition unit 130 , face vector information of a driver gazing forward is stored in the learning information storage unit 150 . At this time, the stored face vector information of the driver is calculated based on distribution and probability between face parts. Here, the relationship refers to the relationship between the size and relative position of both eyes, nose, and eyes.

In addition, as shown in FIGS. 6A to 6D , the face vector information acquisition unit 130 obtains normalization information about the position of the pupil and the relative position and size of the eyes, nose, and mouth based on the reference points selected at the tails of both eyes. A probability distribution based on this is calculated and stored in the learning information storage unit 150 .

For example, as shown in FIG. 6A , the face vector information acquisition unit 130 detects the front through the eye reference areas and the nose and mouth reference areas SA when the face is not gazing at the front. Although it cannot be regarded as gazing, it is determined that the driver is gazing at the front as the pupil (CP) is gazing at the front and stored, and FIGS. 6B and 6C show reference points (SP) of the eyes, nose and mouth and Since (CP) is not all gazing at the front, it is determined and stored as a non-frontal gazing state, and FIG. It is stored and learned based on frontal gaze.

And, in the method of determining whether the driver's eyes are closed, the face vector information acquisition unit 130, as shown in FIGS. If it cannot be detected, it is determined that all eyes are closed, and FIG. 7D determines that the driver's eyes are open when both the reference point (SP) and pupil (CP) are detected and stored.

In addition, the face vector information acquisition unit 130, as shown in FIG. 7A , uses the reference areas SA of the eyes and the reference areas SA of the nose and mouth, even though the face is gazing at the front, the pupil CP (Lv1), a state in which the pupil (CP) cannot be detected in a state in which the head is slightly tilted through the reference areas (SA) of the eyes and the reference areas (SA) of the nose and mouth. (Lv2), the face based on the state (Lv3) in which the pupil (CP) cannot be detected in a state in which the head is lowered through the reference regions (SA) of the eyes and the reference regions (SA) of the nose and mouth. The state vector is stored in the learning information storage unit 150 and learned.

Meanwhile, the face vector information acquisition unit 130 analyzes the drowsiness state level and the attentional state level constituting the face vector information to recognize the face state based on the face information by referring to [Equation 2]. .

Here, Statusdefinition(S) is an eyes-closed state determination function, Statuseyeclosed(W) is an eyes-closed state normalization parameter weight (W), and fps(t) is an eyes-closed state time weight parameter according to execution speed.

Meanwhile, the function Fn(Lv) for determining the level of the closed eyes state can be obtained through [Equation 3] below.

는 얼굴 부분 정보의 절대 위치, 크기, 상대 위치 및 거리 정보를 정규화한 벡터 값이다. Here, Fn(Lv) is a function for determining the state level with eyes closed,

is a vector value obtained by normalizing the absolute position, size, relative position, and distance information of face part information.

Meanwhile, the face vector information acquisition unit 130 may first perform a correction or normalization operation on the initial face information before acquiring the driver's face vector information. For example, the pupil (double-sided) height/width difference may be normalized, the nose may be normalized, the eye blink cycle in a normal state may be normalized, or the normalization may be performed by determining whether or not glasses are worn.

Also, the face vector information acquisition unit 130 may normalize the eye-tail width normalization ratio (A), the mouth-tail width normalization ratio (B), and the position ratio between the nose and A and B.

On the other hand, the facial state information determining unit 140 includes driver manipulation information while driving including the driver's operation status of additional functions from the vehicle external interference recognition device 160 and external interference situation information while driving, which is warning information provided from surrounding vehicles. Acquire

Here, driver manipulation information while driving is information capable of determining whether a smartphone is operated, navigation is operated, or a device inside the vehicle is used, and external interference situation information while driving is information provided from surrounding vehicles, for example, surrounding vehicles. These include turn signal information, high beam information, side vehicle driving pattern information, distance information from the vehicle in front, lane keeping information of surrounding vehicles, BSD (Blind Spot Detection) information, signal waiting and entry information, and vehicle speed information.

Subsequently, the facial condition information determination unit 140 serves to determine the driver's face condition information using the driver connection state information based on the obtained driver's face vector information.

That is, the facial state information determining unit 140 determines the case where driver manipulation information while driving and external interference situation information while driving recognizing an external interference situation are received even if the driver's face condition information is normal. There is an advantage in that the driver's pattern can be learned by labeling and classifying the driver's driving as "normal or abnormal".

Meanwhile, the apparatus for detecting external interference in a vehicle preferably uses a camera, but is not limited thereto, and a microphone capable of recognizing the sound of a horn of a nearby vehicle may be additionally used.

According to such an embodiment of the present invention, there is an effect of learning normal/abnormal driving by recognizing the driver's driving manipulation and warning messages from other vehicles, and driving guidance is possible and updated in real time to improve the driver's It has the advantage of being able to cope with the autonomous driving situation judgment algorithm by dividing it into normal / abnormal.

An apparatus for recognizing a driver's state based on driving situation determination information according to an embodiment of the present invention includes an image acquiring unit 210, a driving situation recognizing unit 220, a driving concentration determining unit 230, and a driving map determining unit 240. It is done by

The image acquisition unit 210 serves to acquire driver's face condition information from an image capture device such as a camera and driver's driving behavior information from a driver's driving information acquisition device.

Also, the driving situation recognition unit 220 serves to obtain driving situation recognition information of surrounding vehicles from an apparatus for recognizing external vehicle interference.

In addition, the driving concentration determining unit 230 compares acquired information such as the driver's face condition information, driving situation recognition information of surrounding vehicles, and driver's driving behavior information with the learned face vector information to determine the driver's driving concentration. do.

And, the driving map determining unit 240 serves to determine a driver driving map to correspond to the determined driver driving concentration.

According to an embodiment of the present invention, a safe driving method is not provided by simply determining whether the driver's eyes are closed or whether the driver's gaze is in a dispersed state, but by using driver connection state information including the information manipulation state of surrounding vehicles. Among the states in which the driver is not looking at the front, a state in which the driver is normally driving can be detected, and in a state in which the driver is looking at the front, a state in which the driver is driving abnormally can be detected.

Meanwhile, the driving concentration determining unit 230 employed in an embodiment of the present invention determines whether the driver is drowsy and gazes forward through the driver's face condition information, and if it is determined that the driver gazes forward, the degree of danger and judge the situation according to the operation level.

Hereinafter, a method of obtaining, learning, and storing data to be compared in the method for recognizing a driver's state based on driving situation determination information according to the present invention will be described with reference to FIG. 8 .

First, the driver state recognition apparatus selects two reference points for setting a reference region in the face image in order to normalize the face image acquired through the camera (S110). Here, the eye designates a candidate for detection of the tail area based on the left/right tail end of the eye area, the nose places a candidate area in the center of the face, and the mouth designates and detects a candidate area for detection at the lower part of the face, As for the mouth tail region, it is preferable to detect the mouth tail based on the central ends of both sides of the detected mouth, similar to the eye tail candidate region.

Subsequently, the driver state recognition apparatus performs warping and normalization to set a rectangular reference area using the coordinates of the selected reference point and the distance and ratio between the two reference points (S120).

Hereinafter, a method of processing pupil data among data processing methods employed in an embodiment of the present invention will be described.

First, when a pupil reference point is selected at the center of the pupil in order to select a pupil reference region in the face image, the driver state recognition apparatus provides a reference guide circle whose size is adjusted from the pupil reference point.

Then, when the user completes the selection, the driver state recognition apparatus selects and normalizes the pupil reference area through the reference guide circle. Here, it is preferable that the point selection unit 110 processes pixels of the remaining area except for the selected pupil reference area as a '0' value.

Thereafter, the driver's condition recognition apparatus obtains the driver's face vector information including face direction reference gaze information, whether eyes are closed, and gaze state by using the vector information between the set rectangular reference regions (S130).

Next, the driver state recognition device acquires driver manipulation information while driving including the driver's manipulation states of the additional functions (S140).

Then, the driver state recognition apparatus obtains the driver connection state information including the manipulation state, and determines the driver's face state information using the driver connection state information (S150).

On the other hand, the step of determining face state information (S150) includes driver manipulation information while driving including the driver's operation state of additional functions from the vehicle external interference recognition device 160 and external interference while driving, which is warning information provided from surrounding vehicles. Get situational information. Here, driver manipulation information while driving is information capable of determining whether a smartphone is operated, navigation is operated, or a device inside the vehicle is used, and external interference situation information while driving is information provided from surrounding vehicles, for example, surrounding vehicles. These include turn signal information, high beam information, side vehicle driving pattern information, distance information from the vehicle in front, lane keeping information of surrounding vehicles, BSD (Blind Spot Detection) information, signal waiting and entry information, and vehicle speed information.

Hereinafter, a driver state recognition method based on driving situation determination information according to an embodiment of the present invention will be described with reference to FIG. 9 .

First, the method for recognizing a driver's state based on driving situation determination information according to an embodiment of the present invention is preferably performed by a driver's state recognition device.

First, the driver state recognition device acquires the driver's face state information from the image acquisition device and the driver's driving behavior information from the driver's driving information acquisition device (S210).

Subsequently, the driver state recognition device acquires driving situation recognition information of surrounding vehicles from the vehicle external interference recognition device (S220).

Thereafter, the driver state recognition device compares acquired information such as the driver's face state information, driving situation recognition information of surrounding vehicles, and driver's driving behavior information with the learned face vector information to determine the driver's driving concentration (S230).

Subsequently, the driver state recognition apparatus determines a driver driving map to correspond to the determined driver driving concentration (S240).

Meanwhile, in the step of determining the driver's driving concentration (S230), it is determined whether the driver is drowsy or stares straight ahead through information on the driver's facial condition (S231).

If it is determined in the determination step (S231) that the driver is looking straight ahead (YES), the driver condition recognition device determines the situation according to the degree of danger and the manipulation level (S232).

In the above, the configuration of the present invention has been described in detail with reference to the accompanying drawings, but this is only an example, and various modifications and changes within the scope of the technical idea of the present invention to those skilled in the art to which the present invention belongs Of course this is possible. Therefore, the scope of protection of the present invention should not be limited to the above-described embodiments, but should be defined by the description of the claims below.

110: point selection unit 120: normalization unit
130: vector information acquisition unit 140: face state information determination unit
150: learning information storage unit
210: image acquisition unit 220: driving situation recognition unit
230: driving concentration determining unit 240: driving guidance determining unit

Claims (16)

an image acquisition unit that acquires driver's face condition information from the image acquisition device and driver's driving behavior information from the driver's driving information acquisition device;
a driving situation recognition unit that obtains driving situation recognition information of a surrounding vehicle from an external interference recognizing device;
a driving concentration determining unit that determines a driver's driving concentration by comparing obtained information such as the driver's face state information, driving situation recognition information of surrounding vehicles, and driver's driving behavior information with learned face vector information; and
A driving map determination unit configured to determine a driver driving map to correspond to the determined driver driving concentration;
a point selector for selecting two reference points for setting a reference area in the face image in order to normalize the face image acquired through the camera; and
A normalization unit for setting a rectangular reference area using the coordinates of the selected reference point and the distance and ratio between the two reference points;
The point selector,
In order to select the pupil reference area in the face image, if it is determined whether a pupil reference point is selected at the center of the pupil and it is confirmed that the reference point is located at the center of the pupil, a reference guide circle whose size is adjusted from the pupil reference point is provided to the user. and setting the corresponding reference guide circle as a pupil reference area when setting selection is completed.
According to claim 1,
The driving concentration determining unit,
Driver state recognition based on driving situation determination information to determine whether the driver is drowsy and gazes forward through the driver's facial state information, and if it is determined that the driver gazes forward, the situation is determined according to the degree of danger and operation level. Device.
delete delete According to claim 1,
The point selector,
An apparatus for recognizing a driver's condition based on driving situation determination information, characterized in that pixels in an area other than the selected pupil reference area are treated as '0' values.
According to claim 1,
a vector information acquisition unit that obtains face vector information of a driver including face direction reference eye gaze information, whether eyes are closed, and a gaze state by using vector information between set rectangular reference regions; and
A face condition information determining unit that obtains driver manipulation information during driving including the driver's operation state of the additional function and determines the driver's face condition information using the driver connection state information based on the acquired driver's face vector information. Driver condition recognition device based on driving situation judgment information.
According to claim 6,
The face vector information,
A device for recognizing a driver's condition based on driving situation determination information calculated by a relationship-based distribution and probability between face parts.
According to claim 7,
The above relationship
A driver's condition recognition device based on driving situation judgment information, which is the relationship between the size and relative position of both eyes, nose and eyes.
obtaining driver's face condition information from an image capture device and driver's driving behavior information from a driver's driving information acquisition device;
obtaining driving situation recognition information of surrounding vehicles from an external interference detection device;
determining a driver's driving concentration by comparing obtained information such as the driver's face condition information, driving situation recognition information of surrounding vehicles, and driver's driving behavior information with learned face vector information; and
Determining a driver driving map to correspond to the determined driver driving concentration;
A reference point selection step of selecting two reference points for setting a reference area in the face image to normalize the face image acquired through the camera; and
A warping and normalization step of setting a rectangular reference area using the coordinates of the selected reference point and the distance and ratio between the two reference points;
selecting a pupil reference point at the center of the pupil to select a pupil reference region in the face image;
providing a reference guide circle sized from the pupil reference point; and
and selecting and normalizing a pupil reference area through the reference guide circle.
According to claim 9,
The step of determining the driver's driving concentration,
determining whether the driver is drowsy and looking straight ahead through the facial condition information of the driver; and
A method for recognizing a driver's condition based on driving situation determination information comprising: determining a situation according to a degree of danger and an operation level when it is determined that the driver is looking straight ahead.
delete delete According to claim 9,
The step of selecting and normalizing the pupil reference region,
A method for recognizing a driver's state based on driving situation determination information, characterized in that pixels in an area other than the selected pupil reference area are treated as a '0' value.
According to claim 9,
obtaining face vector information of a driver including face direction reference gaze information, whether eyes are closed, and a gaze state by using vector information between set rectangular reference regions;
obtaining driver manipulation information while driving including a driver's manipulation state of an additional function; and
and determining facial state information of the driver using driver connection state information based on the obtained driver's face vector information.
According to claim 14,
The face vector information,
A method for recognizing a driver's condition based on driving situation determination information, which is calculated by distribution and probability based on relationships between face parts.
According to claim 15,
The above relationship
Driver state recognition method based on driving situation judgment information, which is the relationship between the size and relative position of both eyes, nose and eyes.

Images