JP7154959B2 - Apparatus and method for recognizing driver's state based on driving situation judgment information - Google Patents

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an apparatus and method for recognizing a driver's state based on driving situation determination information, and more particularly, to an apparatus and method for recognizing a driver's state based on fusion information that can be used in a driving assistance system or an autonomous driving system. .

Recently, before the full development of autonomous driving systems, safe driving guidance technology that applies driver's state recognition technology is being applied for safe driving.

In particular, technology has been developed to guide safe driving by predicting the driver's psychology based on techniques, facial expressions, patterns, etc., and providing warnings and feedback while driving. ing.

In addition, research is also being conducted on a technology that is linked to an autonomous driving system to determine the behavior of the vehicle when the vehicle is difficult to drive or when the driver is unable to drive the vehicle.

However, in order to commercialize the technology that the system judges and drives autonomously, advanced recognition rate and safety device are required, but at the moment, there are problems due to cost and various situations in mass production Therefore, it is difficult to apply the technology.

In addition, the conventional commercial driver assistance system judges that when the driver does not stop for a long time, it is a driving load, and the vehicle behind the side is in the blind spot like BSD (Blind Spot Detection). This is the level at which a warning sound is generated when the driver attempts to enter without turning on the direction indicator.

In this way, the driver assistance systems that have been commercialized in the past have developed functions for recognizing the driver's state, such as drowsiness recognition, facial expression recognition, and attention dispersion recognition, but they were only used as an auxiliary means. , is a difficult situation to provide broad confidence.

In this way, before fully autonomous driving is commercialized, the method of monitoring the driver's condition must be advanced at the auxiliary system level necessary for safe driving. Focusing on interaction, fully autonomous driving is expected to enter a period of active communication between the vehicle, driver, and content.

The existing method of recognizing the driver's state makes it possible to judge the extent to which the driver's eyes are closed, the divergence of the line of sight, the braking, the acceleration, the direction indicator, and the presence or absence of steering information within the existing defined categories.

This includes the problem of the recognition rate for judgments other than some predefined cases for situations, or the limitation of the range for judgments and the occurrence of errors.

The present invention has been devised to solve the conventional problems, and the purpose of the present invention is to determine whether the driver does not gaze forward or only based on the closed eyes of the driver. This is an advanced driver state recognition technology that recognizes and judges the driver's state based on the state of the driver looking ahead, the pattern of closed eyes, vehicle information, and judgment information on the driving environment. An apparatus and method for recognizing a driver's state based on judgment information are provided.

Another object of the present invention is to provide an apparatus and method for recognizing a driver's state based on driving situation determination information, which can analyze in real time by integrating information on the driver's doze and attention divergence with vehicle information and surrounding vehicle information. offer.

It is another object of the present invention to analyze information about the driver's behavior and face while driving in real time, to determine the effects on the surrounding environment and external light, and the position and movement of moving or static obstacles including surrounding vehicles. Based on the information obtained by analyzing the patterns and risk factors, it analyzes the degree of concentration required of the driver when the vehicle is currently running, and provides driver-vehicle judgment information for safe driving or autonomous driving.

The objects of the present invention are not limited to those mentioned above, and other objects not mentioned will be clearly understood by those skilled in the art from the following description.

According to an embodiment of the present invention, there is provided an apparatus for recognizing a driver's state based on driving situation determination information to achieve the above object. a video acquisition unit that acquires the driving action information of the above; a driving situation recognition unit that acquires the driving situation recognition information of surrounding vehicles from the vehicle external interference recognition device; the face state information of the driver, the driving situation recognition information of the surrounding vehicles, a driving concentration determination unit that determines a driver's driving concentration by comparing acquired information such as driver's driving behavior information with learned facial vector information; and a driving guidance determination unit that correspondingly determines a driving guidance for the driver.

Here, the driving concentration determining unit determines whether the driver is dozing off or staring straight ahead based on the facial state information of the driver, and determines that the driver is staring straight ahead. For example, it is desirable to judge the situation according to the degree of danger and the operation level.

a point selection unit for selecting two reference points for setting a reference area from the face image in order to normalize the face image acquired through the camera; It includes a normalizer that uses the distance and ratio between two reference points to set a rectangular reference region.

In addition, the point selection unit determines whether a pupil reference point is selected at the center of the pupil in order to select the pupil reference area from the face image, and confirms that the reference point is positioned at the center of the pupil. and providing a user with a reference guide circle whose size is adjusted from the pupil reference point, and setting the corresponding reference guide circle as the pupil reference area when setting selection is completed.

Also, the point selection unit may process the pixels of the remaining regions other than the selected pupil reference region to have a value of '0'.

Meanwhile, according to an embodiment of the present invention, the driver's face vector information including face direction reference gaze information, whether the eyes are closed, and gaze state is obtained using vector information between the set rectangular reference areas. and driver connection state information including a driver's additional function operation state, and driving using the driver connection state information based on the acquired driver's face vector information. and a face state information determination unit for determining face state information of the person.

A method for recognizing a driver's state based on driving situation determination information according to an embodiment of the present invention includes a step of acquiring facial state information of a driver from an image acquisition device and driving behavior information of a driver from a device for acquiring driving information of a driver. a step of acquiring the driving situation recognition information of surrounding vehicles from the vehicle external interference recognition device; comparing the obtained face vector information to determine the driving concentration level of the driver; and determining driving guidance for the driver corresponding to the determined driving concentration level of the driver.

The step of determining the driving concentration level of the driver employed in an embodiment of the present invention includes determining whether the driver is dozing off and staring straight ahead through the facial state information of the driver; and determining a situation according to the degree of danger and the operation level if it is determined that the driver is staring straight ahead.

On the other hand, in one embodiment of the present invention, in order to normalize the face image acquired through the camera, a reference point selection step of selecting two reference points for setting a reference region from the face image; a warping and normalizing step of setting a rectangular reference region using the coordinates of the selected reference points and the distance and ratio between the two reference points.

The step of selecting a reference point includes selecting a pupil reference point at the center of the pupil to select a pupil reference region from the face image, and determining a reference guide circle whose size is adjusted from the pupil reference point. and selecting and normalizing a pupil reference region through the reference guide circle.

Here, in the step of selecting and normalizing the pupil reference region, the pixels of the remaining regions other than the selected pupil reference region are treated as '0' values.

Furthermore, in one embodiment of the present invention, the vector information between the set rectangular reference areas is used to obtain the driver's face vector information including direction reference line-of-sight information, whether or not the eyes are closed, and line-of-sight state. acquiring driver connection state information including the driver's additional function operating state; and using the driver connection state information based on the acquired driver face vector information, determining facial state information of the person.

According to an embodiment of the present invention, the user can automatically set the reference area by simply pointing the desired part of the face image, thereby analyzing the driver's judgment of normality/abnormality in the future. There is an advantage that the reference area can be normalized and collected when used as data for

1 is a functional block diagram for explaining a driver state recognition device based on driving situation determination information according to an embodiment of the present invention; FIG. FIG. 4 is a reference diagram showing how reference points are selected to detect a facial region in one embodiment of the present invention; FIG. 4 is a reference diagram showing how reference points are selected to detect a facial region in one embodiment of the present invention; FIG. 4 is a reference diagram for explaining a process for selecting a pupillary region in one embodiment of the present invention; FIG. 4 is a reference diagram for explaining a process for selecting a facial region in one embodiment of the present invention; (a) to (d) are reference diagrams for explaining the driver's front recognition criteria in one embodiment of the present invention. (a) to (d) are reference diagrams for explaining recognition criteria according to the condition of the driver's eyes. 4 is a flow chart for explaining a recognition data providing method for recognizing a driver's state on the driving situation determination information base according to an embodiment of the present invention; 4 is a flowchart for explaining a method for recognizing a driver's state based on driving situation determination information in an embodiment of the present invention;

Modes for Carrying Out the Invention

Advantages and features of the present invention, as well as the manner in which they are achieved, will become apparent with reference to the embodiments described in detail below in conjunction with the accompanying drawings. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments disclosed below, merely for the purpose of providing a complete and complete disclosure of the invention. It is provided to fully convey the scope of the invention to those skilled in the art, and the invention is defined only by the scope of the claims. On the other hand, the terminology used herein is for the purpose of describing the embodiments and is not intended to limit the present invention. In this specification, singular forms also include plural forms unless the phrase specifically states otherwise. As used herein, "comprises, comprises" means that a stated component, step, operation, and/or element does not include the presence or presence of a plurality of other components, steps, operations, and/or elements. Do not exclude additions.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. FIG. 1 shows a driving situation determination information-based driver state recognition apparatus according to an embodiment of the present invention, which includes a point selection unit 110, a normalization unit 120, and a vector information acquisition unit so as to register facial vector data information. 130 , a face state information determination unit 140 and a learning information storage unit 150 .

When the user inputs two reference points for selecting a reference area from the face image, the point selection unit 110 sets the corresponding position as the reference point in order to normalize the face image acquired through the camera. For example, as shown in Fig. 2, the reference points are preferably located on both sides of the corners of the eyes, both sides of the mouth, and both sides of the nostrils, but are not limited to this.

Next, as shown in FIG. 3, the normalization unit 120 uses the coordinates of the selected reference point (SP), the distance (D) between the two reference points, and the ratio according to [Formula 1] below. Then, a reference area (SA) is set for each part of the square face.

Thereafter, the normalization unit 120 learns the normalized data of the eyes, corners of the eyes, pupils, closed eyes, mouth, mouth, and nose, and updates the learning information storage unit 150. .

ここで、Ｉｍａｇｅｎｏｒｍａｉｌｉｚｅは基準領域であり、Ｄはポイントとの距離であり、ｘ、ｙは、基準ポイントの座標値であり、Ｗは基準領域の広さの値であり、Ｈは基準領域の高さの値である。 For example, the normalization unit 120 normalizes the face data, the eye data, the closed eye data, the pupil data, the nose data, and the mouth data from the front nine directions, and then normalizes the face, eyes (left, right). ), nose, pupils (left and right), closed eyes (left and right), corners of both eyes, and mouth are learned and stored in the learning information storage unit 150 .

Here, Imagenormalize is the reference area, D is the distance from the point, x, y are the coordinate values of the reference point, W is the width value of the reference area, and H is the height of the reference area. is the value of

According to one embodiment of the present invention, the user can automatically set a reference area (SA) by simply marking a reference point (SP) on a required portion of the face image, thereby enabling the driver's normal/ When used as data for analyzing judgment of abnormality, there is an advantage that the reference area can be normalized and collected.

Meanwhile, the point selection unit 110 employed in an embodiment of the present invention determines whether a pupil reference point (CP) is selected at the center of the pupil, as shown in FIG. Once the pupil reference point is selected, it is determined to select the pupil reference area from the face image and provides the user with a reference guide circle (GC) whose size is adjusted from the pupil reference point. Here, it is preferable that the user adjusts the size of the reference guide circle using a mouse wheel or direction buttons.

Thereafter, when the user completes the selection, the point selection unit 110 selects and normalizes the pupil reference area (SA) through the reference guide circle (GC). Here, it is preferable that the point selection unit 110 process the pixels of the remaining regions other than the selected pupil reference region as '0' values.

Therefore, the point selector employed in one embodiment of the present invention has the advantage that the pupil can be easily normalized from the facial image.

The point selection unit 110 employed in one embodiment of the present invention may extract points from the face image, as shown in FIG. After first detecting the eye candidate region, the nose candidate region, and the mouth candidate region, the eye region is detected. Thereafter, as shown in FIG. 5c, an iris detection candidate area is specified for the eyes, and closed eyes are detected from the face portion as shown in FIG. 5d.

Then, the face vector information acquisition unit 130 uses the vector information between the rectangular reference areas (SA) set from the face image to determine the direction of the face, the direction reference line of sight information of the face, whether the eyes are closed, and the line of sight. Acquire driver face vector information, including state.

For example, the face vector information acquisition unit 130 stores the face vector information of the driver staring at the front in the learning information storage unit 150 . At this time, the stored face vector information of the driver is calculated according to the relation-based distribution and probability between the face parts. The relationship here is the relationship to the size and relative position of the eyes, nose, and eyes.

6a to 6d, the face vector information acquisition unit 130 obtains normalization information for the relative positions and sizes of the pupil position, eyes, nose and mouth based on the reference points selected for both corners of the eyes. is calculated and stored in the learning information storage unit 150 .

For example, as shown in FIG. 6a, the face vector information acquisition unit 130 determines that the face is not gazing forward through the reference area of the eyes and the reference area (SA) of the nose and mouth. Although it is not seen, it is determined that the driver is staring at the front because the pupil (CP) is staring at the front, and stored. (SP) and pupils (CP) are not staring at the front, it is determined that they are not staring at the front and stored. Since both (CP) and (CP) are gazing at the front, they are stored and learned based on the fact that they are gazing at the front.

7a to 7c, the facial vector information acquisition unit 130 determines whether the driver's eyes are closed or not. However, since the pupil (CP) cannot be detected, it is determined that both eyes are closed. is determined to be in a state of opening its eyes and saved.

Further, as shown in FIG. 7A, the face vector information acquisition unit 130 detects pupils (CP) when the face is staring forward through the reference area (SA) of the eyes and the reference area (SA) of the nose and mouth. Unable to detect (Lv. 1), pupil (CP) cannot be detected ( Lv.2), the state where the pupil (CP) cannot be detected while confirming that the head has been dropped through the reference area (SA) of the eyes and the reference area (SA) of the nose and mouth (Lv.3) is stored in the learning information storage unit 150 and learned.

ここで、Ｓｔａｔｕｓｄｅｆｉｎｉｔｉｏｎ（Ｓ）は、目を閉じた状態を判断する関数であり、Ｓｔａｔｕｓｅｙｅｃｌｏｓｅｄ（Ｗ）は、目を閉じた状態の正規化パラメータ加重値（Ｗ）であり、ｆｐｓ（ｔ）は、実行速度による目を閉じた状態の時間加重値パラメータである。 On the other hand, the face vector information acquisition unit 130 analyzes the dozing state level and the attention distribution state level that constitute the face vector information, and determines the face state based on the face information with reference to [Equation 2]. can recognize.

where StatusDefinition (S) is a function that determines the eye-closed state, Statuseyeclosed (W) is the normalized parameter weight value (W) for the eye-closed state, and fps (t) is It is a time weighted value parameter with eyes closed according to execution speed.

ここで、Ｆｎ（Ｌｖ）は目を閉じた状態のレベルを決定する関数であり、

は顔部分情報の絶対位置、サイズ、相対位置及び距離情報を正規化したベクトル値である。 On the other hand, the level determination function (Fn(Lv)) with eyes closed is obtained by the following [Mathematical Formula 3].

Here, Fn (Lv) is a function that determines the level with the eyes closed,

is a vector value obtained by normalizing the absolute position, size, relative position and distance information of face part information.

Meanwhile, the face vector information acquisition unit 130 may first correct or normalize initial face information before acquiring the driver's face vector information. For example, normalize the height/width difference of pupils (both sides), normalize around the nose, normalize eye blink cycle in normal state, normalize by determining whether glasses are worn or not. can do.

The face vector information acquisition unit 130 can also normalize the normalization ratio (A) of the width of the corner of the eye, the normalization ratio (B) of the width of the mouth, and the position ratio between A and B of the nose. .

On the other hand, the facial state information determining unit 140 determines whether the driver's operation information during driving including the additional function operation state of the driver from the vehicle external interference recognition device 160 and the external interference during driving, which is warning information provided from the surrounding vehicles, is detected. Get status information.

Here, the driver operation information while driving is information that can determine whether the smartphone is operating or not, whether the navigation is operating or not, and whether the vehicle's internal devices are being used, and the external interference situation information while driving is provided by surrounding vehicles. Information, for example, direction indicator information of surrounding vehicles, high beam information, driving pattern information of side vehicles, distance information with the preceding vehicle, lane keeping information of surrounding vehicles, BSD (Blind Spot Detection) information, signal waiting and approaching information and vehicle speed information.

Next, the face state information determination unit 140 determines face state information of the driver by using connection state information of the driver based on the acquired face vector information of the driver.

In other words, even if the face state information of the driver is normal, the face state information determining unit 140 can recognize the driver operation information used by the driver while driving and the vehicle external interference situation. By labeling and classifying 'whether the driver's driving is normal or abnormal' for the case where the situation information is received, there is an advantage that the driver's pattern can be learned.

Meanwhile, the vehicle external interference recognition device preferably uses a camera, but is not limited to this, and may further use a microphone capable of recognizing the horn sound of surrounding vehicles.

In addition, according to an embodiment of the present invention, it is possible to learn normal/abnormal driving by recognizing the driver's driving operation and warning messages transmitted by other vehicles. There is an advantage that the driver's normal/abnormality can be classified and dealt with by the autonomous driving situation determination algorithm.

An apparatus for recognizing a driver's state based on driving situation determination information according to an embodiment of the present invention includes an image acquiring unit 210, a driving situation recognizing unit 220, a driving concentration determining unit 230, and a driving instruction determining unit 240. FIG.

The image acquisition unit 210 acquires face state information of the driver from an image acquisition device such as a camera and driving behavior information of the driver from the driving information acquisition device of the driver.

The driving situation recognition unit 220 acquires driving situation recognition information of surrounding vehicles from the vehicle external interference recognition device.

In addition, the driving concentration determination unit 230 compares acquired information such as the driver's facial state information, the driving situation recognition information of surrounding vehicles, and the driver's driving behavior information with the learned facial vector information to drive the vehicle. It plays a role in determining the driving concentration of the driver.

Further, the driving instruction determination unit 240 serves to determine the driving instruction for the driver corresponding to the determined degree of driving concentration of the driver.

According to an embodiment of the present invention, it is possible to provide a safe driving method by simply judging whether the driver has his/her eyes closed or whether the line of sight is dispersed. Using the driver connection state information including the operation state, it is possible to detect the state in which the driver is driving normally without looking at the front, and the state in which the driver is looking at the front. , it is possible to detect a state in which the driver is driving abnormally.

On the other hand, the driving concentration determining unit 230 employed in an embodiment of the present invention determines whether the driver is dozing off or staring straight ahead based on the facial state information of the driver. When it is determined that the user is staring at the target, the situation is determined according to the degree of danger and the operation level.

Hereinafter, a method of acquiring, learning, and storing data to be compared in the driver state recognition method based on driving situation determination information according to the present invention will be described with reference to FIG.

First, the driver's condition recognition device selects two reference points for setting a reference area from the face image in order to normalize the face image acquired through the camera (S110). Here, for the eyes, candidates for the outer corner of the eye area are specified based on the corners of the left and right corners of the eye area, for the nose, the candidate area is placed at the center of the face, and for the mouth, the detection candidate area is selected from the lower part of the face. It is desirable to specify and detect the mouth area, and to detect the mouth area based on both central ends of the detected mouth in the same manner as the eye corner candidate area.

Next, the device for recognizing driver's state performs warping and normalization to set a rectangular reference area using the coordinates of the selected reference points and the distance and ratio between the two reference points (S120).

A method for processing pupil data among the data processing methods employed in an embodiment of the present invention will be described below.

First, when a pupil reference point is selected at the center of the pupil in order to select the pupil reference area from the face image, the driver state recognition apparatus provides a reference guide circle whose size is adjusted from the pupil reference point. be.

Thereafter, when the user completes the selection, the device for recognizing driver's condition selects and normalizes the pupil reference area through the reference guide circle. Here, it is preferable that the point selection unit 110 process the pixels of the remaining regions other than the selected pupil reference region as '0' values.

After that, the device for recognizing driver's state obtains face vector information of the driver, including face direction reference gaze information, whether the eyes are closed, and gaze state, using the vector information between the set rectangular reference areas. (S130).

Next, the device for recognizing the driver's state acquires the driver's operation information during driving including the additional function operation state of the driver (S140).

Then, the device for recognizing driver's state acquires driver's connection state information including the operation state, and determines face state information of the driver using the driver's connection state information (S150).

On the other hand, the step of determining face state information (S150) includes driver operation information during driving including the driver's additional function operation state from vehicle external interference recognition device 160 and warning information provided from surrounding vehicles. obtain the external interference situation information of Here, the driver's operation information while driving is information that can determine whether the smartphone is operating, whether the navigation is operating or not, and whether the vehicle's internal devices are being used, etc. External interference status information while driving is provided by surrounding vehicles. information such as turn indicator information of surrounding vehicles, high beam information, driving pattern information of side vehicles, distance information with the preceding vehicle, lane keeping information of surrounding vehicles, BSD (Blind Spot Detection) information, signal waiting and entry information and vehicle speed information.

Hereinafter, a driver state recognition method based on driving situation determination information according to an embodiment of the present invention will be described with reference to FIG.

First, it is preferable that the driver state recognition method based on driving situation determination information according to an embodiment of the present invention is performed by a driver state recognition device.

First, the device for recognizing driver's state acquires face state information of the driver from the image acquisition device and driving action information of the driver from the driving information acquisition device (S210).

Next, the driver state recognition device acquires driving situation recognition information of surrounding vehicles from an interference recognition device outside the vehicle (S220).

After that, the driver state recognition device compares the acquired information such as the face state information of the driver, the driving situation recognition information of the surrounding vehicles, and the driving behavior information of the driver with the learned vector information of the driver's face. A driving concentration degree is determined (S230).

Next, the device for recognizing driver's condition determines a driving guidance for the driver corresponding to the determined degree of driving concentration of the driver (S240).

On the other hand, the step of determining the driving concentration level of the driver (S230) determines whether the driver is asleep or staring straight ahead based on the driver's facial state information (S231).

In the determination step (S231), if it is determined that the driver is staring at the front (YES), the driver state recognition device determines the situation according to the degree of danger and the operation level (S232).

Although the configuration of the present invention has been described in detail with reference to the accompanying drawings, this is merely an example, and any person having ordinary knowledge in the technical field to which the present invention pertains will be able to understand the present invention. It goes without saying that various modifications and changes are possible within the scope of technical ideas. Therefore, the scope of protection of the present invention should not be limited to the above-described embodiments, but should be determined by the following claims.

110: Point selection unit 120: Normalization unit 130: Face vector information acquisition unit 140: Face state information determination unit 150: Learning information storage unit 210: Image acquisition unit 220: Driving situation recognition unit 230: Driving concentration level determination unit 240: Driving guidance determination part

Claims (16)

an image acquisition unit for acquiring facial state information of the driver from the image acquisition device and driving action information of the driver from the driving information acquisition device;
a driving situation recognition unit that acquires driving situation recognition information of surrounding vehicles from the vehicle external interference recognition device;
A driving concentration level for determining a driver's driving concentration level by comparing acquired information such as the driver's facial state information, driving situation recognition information of surrounding vehicles, and driver's driving behavior information with learned face vector information. A driver state recognition apparatus based on driving situation determination information, comprising: a determining unit; and a driving instruction determining unit that determines driving instruction for the driver corresponding to the determined degree of driving concentration of the driver. The driving concentration degree determination unit
It is determined whether the driver is asleep or stares at the front through the face state information of the driver, and if it is determined that the driver is staring at the front, the situation is determined according to the degree of risk and the operation level. 2. The device for recognizing driver's state based on driving situation judgment information according to claim 1. a point selection unit that selects two reference points for setting a reference area from the face image in order to normalize the face image acquired through the camera; and the coordinates of the selected reference point and the two references. 2. The device for recognizing driver's state based on driving situation determination information as set forth in claim 1, further comprising a normalizer for setting a rectangular reference area using distances and ratios between points. The point selection unit
determining whether a pupil reference point is selected at the center of the pupil to select the pupil reference region from the face image; 4. The apparatus for recognizing driver's state based on driving situation judgment information as claimed in claim 3, wherein a reference guide circle for adjusting is provided to the user, and when the setting selection is completed, the reference guide circle is set as a pupil reference area. The point selection unit
[5] The device for recognizing a driver's state based on driving situation determination information as set forth in claim 4, wherein the pixels in the remaining area except for the selected pupil reference area are processed to have a value of '0'. a vector information acquisition unit that acquires face vector information of a driver including face direction reference line of sight information, whether or not the eyes are closed, and line of sight state, using vector information between the set rectangular reference areas; Obtaining driver operation information during driving including the additional function operation state of the driver, and determining the face state information of the driver by using the driver connection state information based on the obtained face vector information of the driver. 4. The device for recognizing driver's state based on driving situation judgment information as claimed in claim 3, comprising a face state information determining unit. The face vector information is
7. The apparatus for recognizing driver's state based on driving situation judgment information as claimed in claim 6, which is calculated based on relationship-based distribution and probability between facial parts. The relationship is
8. The device for recognizing driver's state based on driving situation judgment information as claimed in claim 7, wherein it is a relationship with respect to sizes and relative positions of both eyes, nose and eyes. a step of acquiring face state information of the driver from the image acquisition device and driving behavior information of the driver from the driving information acquisition device of the driver;
a step of acquiring driving situation recognition information of surrounding vehicles from the vehicle external interference recognition device;
determining the driving concentration level of the driver by comparing acquired information such as the facial state information of the driver, driving situation recognition information of surrounding vehicles, and driving action information of the driver with the learned facial vector information; and determining a driving guidance for the driver corresponding to the determined driving concentration level of the driver. The step of determining the driving concentration level of the driver includes:
a step of determining whether the driver is asleep or staring at the front based on the face state information of the driver; The driver state recognition method based on driving situation determination information according to claim 9, comprising the step of determining a reference point selection step of selecting two reference points for setting a reference region from the face image to normalize the face image acquired through the camera; The driver state of the driving situation judgment information base according to claim 9, comprising a data processing means including a warp and normalization step of setting a rectangular reference area using the distance and ratio between reference points. recognition method. a pupil reference point is selected at the center of the pupil to select a pupil reference region from the face image;
12. The driving situation determination according to claim 11, comprising providing a reference guide circle whose size is adjusted from the pupil reference point; and selecting and normalizing a pupil reference area through the reference guide circle. Information-based driver state recognition method. Selecting and normalizing the pupil reference region comprises:
[Claim 13] The method of claim 12, wherein the pixels of the remaining area except the selected pupil reference area are treated as '0' value. obtaining face vector information of the driver including direction reference gaze information of the face, whether the eyes are closed and gaze state using vector information between the set rectangular reference areas;
a step of obtaining driver operation information during driving including a driver's additional function operation state; and a driver's face state using driver connection state information based on the obtained driver's face vector information. The method for recognizing a driver's state based on driving situation judgment information as claimed in claim 11, comprising the step of determining information. The face vector information is
15. The method of claim 14, wherein the driving situation determination information-based driver state recognition method is calculated based on the relationship-based distribution and probability between facial parts. The relationship is
16. The method for recognizing a driver's state based on driving situation determination information as claimed in claim 15, wherein the relationship is the size and relative position of both eyes, nose, and eyes.

Images