KR101360412B1 - A system for monitoring loose/droway driver using head moving information - Google Patents

Abstract

This document is a system that helps to drive safely by obtaining images using infrared cameras and analyzing driver's head movements to determine the driver's distraction or drowsiness and notify the driver. In particular, when the eyes are not detected, such as a driver wearing sunglasses or glasses, the driver's head movement information is obtained using the position information of the mouth and nose rather than the eyes, which is why the existing distraction / drowsiness driving monitoring systems are not used. It is a safe driving system that can be applied to a wide range of drivers.

Head Movement, Distraction / Drowsiness Judgment, AAM

Description

Distraction / drowsiness driving monitoring system using head movement information {A SYSTEM FOR MONITORING LOOSE / DROWAY DRIVER USING HEAD MOVING INFORMATION}

This document relates to a distraction / drowsiness driving monitoring system, and more particularly, to a distraction / drowsiness driving monitoring system using head movement information.

For safe driving of vehicles, various methods for coping with distracted driving or drowsy driving, which are the main causes of car accidents, have been proposed. Most methods of determining whether drowsy driving is a method of determining whether drowsiness is based on the shape of a driver's eyes (Patents 1, 2, 3). However, since the existing drowsy driving detection technology relies heavily on the eye image, it cannot be applied to a driver wearing glasses or light sunglasses, for example, when the eyes cannot be detected.

For example, the following problems occur when wearing glasses or light sunglasses. First, the existing eye detection methods rely heavily on the image information of the eye area. When wearing visual aids such as glasses or sunglasses, the image around the eye including the eye is changed, which is a serious mistake in the conventional eye detection method. Generates. In order to solve the above problem, it is necessary to consider visual aids in eye detection, but there is no technique for detecting and processing visual aids having various colors, shapes, and sizes.

Second, when wearing a visual aid such as glasses or sunglasses, the external reflection image is drawn on the starting aid due to the characteristics of the lens material. At night, there is a problem that the light source from the headlights of the car coming from the front or the road lighting is reflected on the visual aid. In addition, during the day, the light from the sun is reflected by surrounding objects such as buildings, roads, and mountains, and thus the appearance of the outside of the car is displayed on the starting aid. In this case, it is impossible to find eyelids or pupils in the image of the eye area. Therefore, the drowsiness detection technology through eye image has a limitation that is widely used in the situation where many drivers are wearing glasses or light sunglasses.

Distracted driving of drivers is also important to reduce accidents. However, at present, many indirect methods have been proposed to reduce or eliminate obstacles causing mental distraction in advance, rather than directly detecting distracted driving of drivers (Patents 4 and 5). Although the above methods can be preventive measures to reduce distracted driving, there is a problem that even if the driver distracts during actual driving, no response or effect can be exerted.

Patent 1) Publication No. 10-2006-0022935 (eye image based drowsiness detection method and apparatus)

Patent 2) Registration No. 20-0392762-0000 (Near infrared image based drowsiness detection device)

Patent 3) Registration No. 10-0766592-0000 (Iris reflection image based drowsiness detection device and method)

Patent 4) Publication No. 10-1999-027440 (Long time driving warning device)

Patent 5) Publication No. 10-1999-0041852 (room pressure compensation sunroof system)

This document aims to provide a system to reduce car accidents by detecting driver distractions and drowsy driving, which are the main causes of car accidents, through driver's head movements and alerting the driver.

The distraction / drowsiness driving monitoring system as one means for solving the above-described problems includes a face detection unit extracting a face region from a input image as a face image, and an eye detection determination unit determining whether eyes can be found from the face image.

In addition, in consideration of the relative position between the parts constituting the face in the face image, depending on whether the eye is detected, if the eye is detected, the head angle and the position is extracted by the first measuring method, and if not detected, the second position It includes a head angle and position measuring unit for extracting the head angle and position as a measuring method.

In addition, the head motion tracking unit for tracking the head movement using the cylinder and ellipse model set based on the extracted head angle and position information, and generates the head movement information, and the distraction / using the head movement information It includes a distraction / drowsy driving determination unit for determining whether or not drowsy driving.

According to the first measurement method, the head angle and position information is obtained through an AAM (Active Appearance Model) -based face precision detection unit, and the AAM-based face precision detection unit is configured using the AAM algorithm based on a face model prepared in advance. In the face region, the face model is transformed into a shape that matches the actual face, and the head angle and position information of the head can be obtained using the fitted face model information.

According to the second measurement method, a left and right tilt angle measuring unit using the face image, a face normalization module for normalizing a face based on the mouth and nose of the face image, and based on the left and right tilt angle information and the face normalization information The angle and position information of the head can be obtained through the left and right rotation angle measuring unit using the face pose measurement.

The left and right tilt angle measuring unit and the left and right rotation angle measuring unit may obtain a left and right rotation angle of the head using a linear model based on a predetermined angle.

The head motion tracking unit may set initial values of a cylinder and an ellipse model using the head angle and position information, and provide head motion information in real time while tracking a face through a face image.

The distraction / drowsy driving determination unit may simultaneously perform distraction driving determination and drowsy driving determination.

The distraction driving determination may be determined as distraction driving when the left and right rotation angle information exceeds a preset allowable value and the time exceeds the preset allowable time.

The drowsy driving determination may be determined to be drowsy driving when the vertical rotation angle information exceeds a preset allowable value and the time exceeds the preset allowable time or the number of times exceeds the preset allowable number.

According to the present invention, it is possible to determine whether the driver is distracted driving and drowsy driving to inform the driver of this, thereby reducing an automobile accident.

 In particular, it is possible to determine whether distracted or drowsy driving is performed for all drivers regardless of whether they wear visual aids such as glasses or pale sunglasses that are difficult to detect.

In addition, since the driver determines the drowsy driving judgment and distracted driving together, there is no need to separately install the device for determining the drowsy driving and the device for determining the distracted driving, and to determine whether the driver is distracted or drowsy with only one system. It is effective.

In addition, the technology for identifying the driver's appearance or behavior pattern for safe driving can be used to detect distracted / drowsy driving, as well as to identify the driver's gesture in the vehicle and to communicate the command between the driver and the vehicle. There is an effect that can be used as.

Hereinafter, exemplary embodiments of the present disclosure will be described in detail with respect to a distracted / drowsy driving detection system for detecting a distracted driving and a drowsy driving state of a driver using head movement information with reference to the accompanying drawings.

1 illustrates a block diagram of a driver's distraction / drowsiness driving detection system according to an embodiment of the present invention, and FIG. 2 illustrates an operation of a distraction / drowsiness driving detection system using driver's head movement information according to the present embodiment. .

According to the distraction / drowsiness driving detection system of the present embodiment, in consideration of the relative position between the parts constituting the face in the face image through the input image, when the driver's eyes can be detected and the eyes cannot be detected. It is characterized by measuring the angle and position of the head in a separate method for each. Then, the initial values of the cylinder and the ellipse model are set based on the measured head angles and positions, and the head movement information is generated by tracking the movement of the driver's head using the head angle information and the head movement information. Distinguish / drowsiness is discriminated.

The driver distraction / drowsiness detection system of the present embodiment includes an image acquisition unit 10, a face detection unit 20, an eye detection determination unit 30, a head angle and position measurement unit 40, a head motion tracking unit 50, and Distraction / drowsiness operation determination unit 60 is included.

In operation of the driver distraction / drowsiness detection system, first, in step S110, the image acquisition unit 10 acquires an input image from the outside. The image acquisition unit 10 may acquire an input image by an image input sensor or an input image transmitted by a wired or wireless connection from the outside.

For example, when an input image is acquired by an image input sensor, an input image may be obtained by converting an image signal of a subject incident through a predetermined lens into an electrical signal. Here, the image input sensor may include a charge coupled device (CCD), a CMOS, and other image acquisition means known in the art. In addition, an analog / digital converter for converting the electrical signal obtained by the image input sensor into a digital signal and a digital signal processor (DSP) for receiving a digital signal converted by the analog / digital converter and processing the image signal It is possible to obtain a predetermined input image by, for example.

As another example, the input image may be stored in a storage medium or transmitted through a wired or wireless network to be provided to a driver's distraction / drowsiness driving detection system according to an embodiment of the present invention. The image acquisition unit 10 may convert the obtained input image into a single channel image. For example, the input image may be changed to a gray scale. Alternatively, when the input image is a multi-channel image of the 'RGB' channel, it may be changed to one channel value. Therefore, by converting the input image into an intensity value in one channel, the brightness distribution of the input image can be easily represented.

In operation S120, the face detection unit 20 detects a face region as a face image from the input image. The face detector 20 may extract an eye, a nose, a mouth, etc. which are specific components in the face after detecting the approximate face from the input image, and extract the face region based on the face image.

For example, if the position of two eyes is detected, the distance between the two eyes can be obtained. The face detector 20 may extract a face region from the input image as a face image based on the distance between the two eyes, thereby reducing the influence of the background of the input image or the change of the human hair style.

In addition, the face detector 20 may normalize the size of the face region using the extracted face region information. By normalizing the size of the face region, unique features such as distance between two eyes and distance between eyes and nose in the face region can be calculated at the same scale level.

In operation S130, the eye detection determiner 30 determines whether the eye is detected in the eye region of the face. In the face area, eye detection can be determined by comparing a previously prepared eye model centering on the expected eye area and comparing the similarity with the best matching area.

Although the area estimated as the eye is detected, if the degree of similarity is very low, the eye is detected as a part other than the eye in the detected area, and the eye is not found. On the other hand, when the degree of similarity is above a predetermined reference value, it is determined that the eye is detected.

In the head angle and position measurement unit 40, the relationship between the two eyes, the nose, the mouth, and the like, in consideration of each component of the face, that is, the two eyes, the nose, the mouth, and the like as the judgment elements, is measured. That is, it is characterized by measuring the head angle and position in consideration of the relative position.

In particular, according to the present embodiment, when the eyes are detected in the face image, when the eyes are detected, the head angle and the position are extracted by the first measuring method of step S140, and when the eyes are not detected, the steps S150, S160 and The angle and position of the driver's head are measured by the second measuring method of step S170.

Here, the AAM-based face precision detection method may be used as the first measurement method. In this case, the distraction / drowsiness detection system according to the present embodiment further includes an AAM-based face precision detection unit.

In step S140, the face region detected by the face detector 20 in the image using the AAM-based face model through the AAM-based face precision detection unit, reconstructs the correct face region, and the tilted head angle information of the head and each part Obtain the location information of.

The AAM algorithm is a technique commonly used for face fitting operations (matching points or lines at the closest positions), and includes a shape model containing position information of each point of the face and a surface containing image information between each face portion ( By using the texture model, the most similar face areas are found in the image, so precise face fitting is possible. When the face fitting operation is performed, the matched position information is used to provide the tilt, rotation angle, and position information of the main part of the human head.

On the other hand, in the second measurement method, the rough left and right head tilt angles are measured through face detection. In this case, since eye detection information is not available, normalization information is generated by normalizing the face based on the nose and mouth, and the left and right head rotation angles are measured through face pose measurement based on the left and right head tilt angle and normalization information. You can use the method of inferring the detailed angles. In this case, the distraction / drowsiness detection system according to the present embodiment further includes a left and right hill tilt angle measuring unit, a face normalization module, and a left and right hill rotation angle measuring unit.

In step S150, the left and right tilt angle information of the head obtained by the face detection unit 20 through the left and right hill tilt angle measuring unit and a linear model based on, for example, 0 degrees, 30 degrees, and 60 degrees are used. We work to infer the detailed angles.

In operation S160, the face is normalized based on the nose and the mouth through the face normalization module. If both eyes cannot be detected in the normalization based on the existing eyes, the normalization is performed by using the tip of the nose and the ends of the mouth for coordinates for normalization. In this case, the normalization method performs normalization using an affine transform as in the case of the two eyes.

Then, the detailed left and right rotation angle information of the head generated by using the face pose measurement through the left and right hill rotation angle measurement unit in step S170 and the detailed angle using a linear model based on, for example, 0 degrees and 25 degrees. Inferring the work.

In step S180, the head movement tracking unit 50 initializes the cylinder and ellipse model using the head angle information and the position information obtained from the head angle and the position measuring unit 40, and then moves to the initialized cylinder and ellipse model. It compares the changed image and detects and tracks the movement.

Here, the head angle information obtained from the head angle and the position measuring unit 40 may include left and right tilt angles, vertical movement angles, left and right rotation angles, and the like. When the angles cannot be measured through the first and second measurement methods, the cylinder and the ellipse model can be initialized by setting the value to '0', that is, the front side.

That is, in the distraction / drowsiness detection system according to the present embodiment, when the eye can be found in the face image of the person in the image input acquisition unit 10, the AAM-based face precision detection unit, and the left and right hill angles when the eye cannot be found. Estimate the human head movement obtained from the measurement unit and the face normalization module.

Then, in step S190 through the distraction / drowsy driving determination unit 60 by analyzing the head movement pattern obtained from the head movement tracking unit 50 using the cylinder and the elliptic model, it is determined whether the driver distracted driving and drowsy driving Notify the driver later. The distraction / drowsy driving determination unit 60 includes a distraction driving determination portion and a drowsy driving determination portion.

3 is a view for explaining an operation of detecting a distracted driving of a driver using the head movement information of the driver generated according to an embodiment of the present invention.

According to the driver's distraction driving detection system shown in FIG. 3, the head movement tracking unit 50 continuously tracks the head movement of a person in step S210, and provides left and right movement angles and up and down movement angle information of the head for each hour. .

In operation S220, it is determined whether the current left and right moving angle, in particular, the left and right rotation angle, exceeds a predetermined allowable value using the left and right angle information provided by the head motion tracking unit 50.

As a result of the determination in step S220, when the left and right movement angle of the current head exceeds a predetermined allowable value, the process proceeds to step S230 to determine whether the left and right angle allowable value exceeding state is maintained for a predetermined time or more. This can be seen as determining whether the driver is continuously looking at a place that is not in front for a predetermined time.

As a result of the determination in step S230, when the left and right angle allowance exceeding state continues for a predetermined time or more, the process proceeds to step S240 to determine that the driver is distracted driving, and may take action. For example, a warning can be given to the driver, such as a voice or a screen, which can be thought of as flashing a warning light, a voice warning, or a display indication.

However, when the determination result of the step S220 and the step S230 does not exceed the predetermined allowable angle of the head or the left and right angle allowance exceeded for a predetermined time or more, proceeds to step S210 again without any further action and the above-described process Can be performed repeatedly.

4 is a diagram for describing an operation of detecting a drowsy driving of a driver by using head movement information of a driver generated according to an exemplary embodiment of the present invention.

According to the driver's drowsy driving detection system shown in FIG. 4, the head movement tracking unit 50 continuously tracks the head movement of the person in step S310, and provides information about the left and right movement angles and the up and down movement angles of the head for each hour. .

In operation S320, it is determined whether the current vertical movement angle of the head exceeds a predetermined allowable value by using the vertical angle information provided by the head motion tracking unit 50.

As a result of the determination in step S320, if the current vertical movement angle of the head exceeds a predetermined allowable value, the process proceeds to step S330 to determine whether the upper and lower angle allowable value exceeding state continues for a predetermined time or more. This can be seen as determining whether the driver is continuously looking at a place that is not in front for a predetermined time. Alternatively, in step S330, the number of head movements of the head exceeding the allowable upper and lower angles, in particular, the upper and lower angle allowances, may be measured for a predetermined time to determine whether the preset allowable number is exceeded.

As a result of the determination in step S330, when the state in which the upper and lower angle allowable value exceeds the predetermined time or the number of the upper and lower movements exceeds the allowable number, the process proceeds to step S340 and determines that the driver is drowsy driving, and may take action. For example, a warning can be given to the driver, such as a voice or a screen, which can be thought of as flashing a warning light, a voice warning, or a display indication.

However, as a result of the determination of steps S320 and S330, if the vertical movement angle of the li does not exceed the predetermined allowable value or the state of exceeding the upper and lower angle allowable value does not last for a predetermined time, the process described above is repeated again without further action and the above-described process is repeated. It can be done with

The technique of identifying the driver's appearance or behavior pattern for safe driving, such as judging the movement of the head described above, can be used to detect distracted / drowsy driving as well as to grasp the gesture of the driver in the vehicle. It could also be used as a command delivery interface technology between the driver and the car.

 In the above, the present invention has been described with reference to the drawings for a block diagram or a processing flowchart for explaining a face recognition system and method according to embodiments of the present invention. It will be appreciated that the combinations of blocks and flowchart illustrations in the process flow diagrams may be performed by computer program instructions. Since these computer program instructions may be mounted on a processor of a general purpose computer, special purpose computer, or other programmable data processing equipment, those instructions executed through the processor of the computer or other programmable data processing equipment may be described in flow chart block (s). It creates a means to perform the functions.

These computer program instructions may also be stored in a computer usable or computer readable memory capable of directing a computer or other programmable data processing apparatus to implement the functionality in a particular manner so that the computer usable or computer readable memory The instructions stored in the block diagram (s) are also capable of producing manufacturing items containing instruction means for performing the functions described in the flowchart block (s).

Computer program instructions may also be stored on a computer or other programmable data processing equipment so that a series of operating steps may be performed on a computer or other programmable data processing equipment to create a computer- It is also possible for the instructions to perform the processing equipment to provide steps for executing the functions described in the flowchart block (s).

In addition, each block may represent a portion of a module, segment, or code that includes one or more executable instructions for executing a specified logical function (s). It should also be noted that in some alternative implementations, the functions mentioned in the blocks may occur out of order. For example, two blocks shown in succession may actually be executed substantially concurrently, or the blocks may sometimes be performed in reverse order according to the corresponding function.

The term '~' or '~' as used in this embodiment refers to software or a hardware component such as an FPGA or an ASIC, and the '~' or '~ module' performs certain roles. However, '~' or '~' is not meant to be limited to software or hardware. The '~' or '~' module may be configured to reside in an addressable storage medium or may be configured to play one or more processors.

Thus, as an example, 'part' or '~ module' means components such as software components, object-oriented software components, class components, and task components, processes, functions, attributes, and the like. , Procedures, subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, and variables. Functions provided within components and 'parts' or 'modules' may be combined into a smaller number of components and '~ parts' or '~ modules', or additional components and '~ parts' or It can be further separated into 'modules'.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit and scope of the invention as defined by the appended claims. You will understand.

1 is a block diagram of a driver's distraction / drowsiness driving detection system according to an embodiment of the present invention.

2 is a flowchart illustrating the operation of the distraction / drowsiness driving detection system using the head movement information of the driver according to the present embodiment.

3 is a flowchart illustrating an operation of detecting a distracted driving of a driver by using head movement information of a driver generated according to an embodiment of the present invention.

4 is a flowchart illustrating an operation of detecting a drowsy driving of a driver by using head movement information of a driver generated according to an embodiment of the present invention.

DESCRIPTION OF THE REFERENCE NUMERALS

10: image acquisition unit, 20: face detection unit

30: detection discrimination unit, 40: head angle and position measuring unit

50: head movement tracking unit 60: distraction / drowsiness driving determination unit

Claims (8)

A face detector extracting a face region from the input image as a face image; An eye detection determiner that determines whether eyes can be found from the face image; Considering the relative position between the parts constituting the face in the face image, depending on whether the eye is detected, if the eye is detected by extracting the head angle and position by the first measuring method, if not detected the second measuring method A head angle and position measuring unit for extracting the head angle and position; A head movement tracking unit for generating head movement information after tracking head movements using a cylinder and an ellipse model set based on the extracted head angle and position information; And Distraction / drowsiness driving determination unit for determining whether distraction / drowsiness driving using the head movement information Including, distraction / drowsiness driving monitoring system. The method according to claim 1, According to the first measurement method, the head angle and position information is obtained through an AAM (Active Appearance Model) -based face precision detection unit, and the AAM-based face precision detection unit is configured using the AAM algorithm based on a face model prepared in advance. Distracting / drowsiness driving monitoring system, characterized in that by fitting the face by transforming the face model in the face area to match the actual face, and obtaining the head angle and position information of the head through the fitted face model information. The method according to claim 1, According to the second measurement method, a left and right tilt angle measuring unit using the face image, a face normalization module for normalizing a face based on the mouth and nose of the face image, and based on the left and right tilt angle information and the face normalization information Distraction / drowsiness driving monitoring system, characterized in that to obtain the head angle and position information through the left and right angle measurement through the face pose measurement. The method of claim 3, And the left and right tilt angle measuring unit and the left and right rotation angle measuring unit obtain a left and right rotation angle of the head using a linear model based on a predetermined angle. The method according to claim 1, The head motion tracking unit sets initial values of a cylinder and an ellipse model using the head angle and position information, and provides head motion information in real time while tracking a face through a face image. Drowsiness driving surveillance system. The method according to claim 1, The distraction / drowsiness driving determination unit performs distraction / drowsiness driving determination at the same time. The method according to claim 1, The distraction driving determination system, the distraction / drowsiness driving monitoring system, characterized in that it is determined that the distraction operation when the left and right rotation angle information exceeds a preset allowable value and the time exceeds the preset allowable time. The method according to claim 1, The drowsiness operation determination, when the up-and-down rotation angle information exceeds a preset allowable value and the time exceeds the preset allowable time or the number of times exceeds the preset allowable number, distraction / Drowsiness driving surveillance system.

Images