KR20060022935A - Drowsiness detection method and apparatus based on eye image - Google Patents

Abstract

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a drowsiness detection method and apparatus thereof based on an eye image, and in particular, irradiates near-infrared light to a human eye from a near-infrared light source, and in response to the light, whether or not the image appears in the human eye and the degree and time of distortion thereof. The present invention relates to a method and apparatus for detecting drowsiness based on an eye image to detect and determine drowsiness based on information.

The present invention irradiates the human eye with pilot light as a reference light image for the determination of drowsiness from an infrared light source, obtains a face image of the person irradiated with the infrared light, and responds to the irradiated infrared light in the acquired face image. Detects the position of the pupil by searching the pilot light image (light reflected from the retina acting as a concave mirror or condensed into the pupil or an image of a light source formed on the pupil acting as a convex mirror), and the pilot light image formed on the pupil Is characterized in that it is determined whether the drowsiness and alarm based on how distorted from the original and the distortion time information.

Eye detection, image recognition, drowsiness detection, drowsiness alarm

Description

Eye image based drowsiness detection method and apparatus {DROWSINESS DETECTION METHOD AND APPARATUS BASED ON EYE IMAGE}

1 is a block diagram of an eye image based drowsiness detection apparatus of the present invention

2 is a diagram illustrating an example of an eye image according to an eye image processing process of the first exemplary embodiment of the present invention;

Figure 3 is a flowchart of the drowsiness detection method according to the first embodiment of the present invention

4 illustrates an example of an eye image according to an eye image processing process according to the second exemplary embodiment of the present invention.

5 is a flowchart of a drowsiness detection method according to a second embodiment of the present invention;

<Explanation of symbols for the main parts of the drawings>

10: eyes 20: camera

21: image processing unit 30: camera phone

40: alarm generating unit

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an eye image-based drowsiness detection method and apparatus therefor, in particular, irradiating near-infrared light to a human eye from a near-infrared light source, and in response to the light, whether or not the image appears in the human eye and its degree and time of distortion. The present invention relates to a drowsiness detection method based on an eye image and a device for detecting and determining whether or not drowsiness is based on information.

In particular, the present invention irradiates a near-infrared light source to the eyes of a vehicle driver, and includes a near-infrared camera that captures an eye image of a person including an image reflected by the irradiated light—a light collected by a pupil or a light source formed in a pupil. Eye image-based drowsiness detection method and apparatus for detecting and determining the driver's drowsiness according to the degree of eyes closed by detecting the position of the pupil from the acquired image It is about.

If the drowsiness comes while driving the vehicle is very likely to develop into a large personnel accident, prevention of drowsy driving is a very important problem in preventing vehicle safety accidents. Drowsy driving is caused by a variety of factors including the driver's personal health, condition, driving and driving environment. To prevent this, drowsy driving prevention and measures are recommended, such as exercising enough rest and light stretching at appropriate driving distances, or recommending stopping and resting when drowsiness comes while driving. For this reason, the above-mentioned rules are hard to be observed, and there are also cases of drowsiness while driving in a vehicle. Therefore, drowsiness driving is not reduced in traffic accidents.

Various technical approaches have been taken to prevent such drowsy driving. For example, it detects and determines whether the driver is asleep by using the physical characteristics of the drowsiness, especially the characteristic winding pattern of the eyelids on the face, and alerts the driver by generating an alarm sound when the driver is drowsy. The technique is known.

To this end, a face image is obtained by photographing a human face with a camera, and a method of detecting whether the user is dozing by the distance between the upper and lower eyelid boundaries by finding the position of the eye from the acquired facial images is used. .

In the prior art, the method of detecting the eye position extracts all boundary lines due to the brightness difference from the face image acquired by the camera, and calculates the geometric position of the face such as the distance between the two eyes and the distance to the nose. We extracted the eyelid boundary line through the binarization process and detected the degree of drowsiness according to the distance between the upper and lower eyelid boundary.

However, the disadvantage of this method is that because the lighting conditions of the face continuously change due to the change of light direction, backlight, etc. during driving of the vehicle, it is difficult to detect the eyes by the boundary extraction, and the brightness difference becomes small at night. The extraction of the boundary becomes more difficult.

In addition, according to the prior art, since the geometrical characteristics of both eyes, nose and nose changes according to the direction of the face or the head down, it is frequently detected that the eyebrows, nose, glasses, etc. are not eyes, but drowsy driving. Reliability of information that can be used as a basis for judgment is low, which causes a large error in detecting and determining whether or not drowsiness, and due to such problems, it is not practical.

In addition, the prior art has a disadvantage that can not exclude the hardware burden on the drowsiness detection and alarm device because the use of a large brightness near-infrared light source for lighting in the night or dark environment.

An object of the present invention is to irradiate light to the human eye from a light source and obtain an eye image including an image of light formed on the human eye in response to the light, and based on the distortion degree and distortion time information of the light in the acquired eye image The present invention provides a method for detecting drowsiness based on an eye and a device thereof to determine whether or not a drowsiness is detected.

Another object of the present invention is to irradiate light to the human eye from a near-infrared light source, and the difference in brightness between the pupil and the brightness around the pupil, which are reflected from the retina and focused on the pupil due to the concave mirror characteristics of the eye retina, are the largest. The present invention provides an eye image-based drowsiness detection method and apparatus capable of generating a drowsiness alert by detecting an eye position by using the eye position, and determining the degree of eye closure based on the detected degree of deformation of the pupil and time information.

Another object of the present invention is to detect the pupil using the fact that near-infrared light is reflected from the retina and focused on the pupil, and to determine whether it is drowsy based on the distortion degree and distortion time information of the pupil shape, such as at night or in a dark tunnel. Accurately locates the driver's eye even when the contrast conditions are bad, even in the light reflected by the lens when wearing glasses, and the brightness value of each pixel and the brightness of the surrounding pixel in the original image without the binarization process for extracting the boundary line as in the prior art By detecting the pupils by the difference from the values, the eye detection algorithm is simple and makes it possible to find the pupils at high speed, and because the near-infrared light source is used for the purpose of detecting the pupil brightness, the brightness is large for the conventional night or dark lighting. To avoid the use of near-infrared light sources It is an object of the present invention to provide a drowsiness detection method and apparatus based on an eye image to be used as a small number.

Another object of the present invention is to irradiate light to the human eye from the near-infrared light source, and to search for the eye position using the point that the difference in brightness between the image of the light source formed on the pupil and its surroundings is the maximum due to the convex mirror characteristic of the pupil. In addition, the present invention provides an eye image-based drowsiness detection method and apparatus for generating a drowsiness alarm by grasping the degree of eye closure in accordance with the degree of deformation of the shape of the retrieved light source and time information.

Another object of the present invention is to detect the shape of the reflected light by using the fact that near-infrared light is reflected in the convex pupil and to determine whether it is drowsiness based on the distortion degree and distortion time information of the shape, such as night or dark tunnels Accurately locates the driver's eye even when the contrast conditions are bad, even in the light reflected by the lens when wearing glasses, and the brightness value of each pixel and the brightness of the surrounding pixel in the original image without the binarization process for extracting the boundary line as in the prior art By detecting the pupil using the characteristic that the difference with the value is the maximum, the eye detection algorithm is simple, makes it possible to find the pupil at high speed, and the near-infrared light source is used for the purpose of forming the image on the pupil. Eliminate the use of high brightness near-infrared light sources for lighting in the dark It provides a high to a relatively small size and the brightness also a drowsiness detection of the eye based on the image to be used for the light source method and apparatus it is an object.

Eye image-based drowsiness detection method of the present invention for achieving the above object, the light irradiation step of irradiating light to a human eye from a light source, the position and shape of the pupil by acquiring the face image of the person irradiated with the light A pupil recognition step of recognizing, a drowsiness determination and alerting step of detecting and determining a drowsiness and performing a drowsiness alert based on the degree of deformation of the recognized pupil shape and its holding time information; Characterized in that comprises a.

In addition, in the eye image-based drowsiness detection method of the present invention, the pupil recognition step is to recognize the location of the pupil of the light is reflected from the retina by the near-infrared light to collect the brightness value of each pixel in the face image and its surroundings The pupil position of the eye is detected by finding the position of the pixel having the largest difference in brightness value.

In addition, in the eye image-based drowsiness detection method of the present invention, the pupil recognition step is a position where the difference between the brightness value of the pupil and the brightness value of the periphery is the maximum position of the pupil where the light reflected from the retina is collected by irradiating near-infrared light And the surrounding pixel group having a brightness value similar to the brightness of the pixel of the maximum brightness difference to recognize the shape of the pupil.

In addition, in the eye image-based drowsiness detection method of the present invention, the drowsiness determination and alerting step is to determine the degree of eye closure in accordance with the degree of deformation of the recognized pupil shape from the circle, and completely closed the eyes to recognize the recognized pupil If the disappeared time is longer than the set time is characterized in that the sleepiness alarm.

In addition, the eye image-based drowsiness detection method of the present invention for achieving the above object, the light irradiation step of irradiating light to the human eye from a light source, the light source obtained by acquiring the face image of the person irradiated with the light is formed on the pupil An optical image recognition step of recognizing an image, a drowsiness determination and alarm step of detecting and determining drowsiness and performing a drowsiness alarm based on the degree of deformation of the shape of the recognized light source and its holding time information; Characterized in that comprises a.

In addition, in the eye image-based drowsiness detection method of the present invention, in the optical image recognition step, the difference between the brightness value of each pixel in the face image and the brightness value of the surroundings is maximum in recognizing an image of a light source formed on the pupil by irradiating near-infrared light. The position of the pixel is detected by detecting the position of the pupil.

In addition, in the eye image-based drowsiness detection method of the present invention, in the optical image recognition step, the difference between the brightness value of each pixel in the face image and the brightness value of the surroundings is maximum in recognizing an image of a light source formed on the pupil by irradiating near-infrared light. The position of the pupil is sensed from the point of in and the surrounding pixel group having a brightness value similar to the brightness of the pixel of the maximum brightness difference is grouped to recognize the shape of the image of the light source.

In addition, in the eye image-based drowsiness detection method of the present invention, the drowsiness determination and alerting step is to determine the degree of eye closure according to the degree of deformation of the shape of the light source formed on the pupil from the circle, and completely closed the eyes The drowsiness alarm may be performed when the time at which the recognized light source disappears for more than a predetermined time.

In addition, the eye image-based drowsiness detection method of the present invention for achieving the above object, the step of irradiating the human eye with pilot light as a reference light image for the determination of drowsiness from the near infrared light source, the face of the person irradiated with the near infrared light Acquiring an image, retrieving a pilot light image appearing in response to the irradiated near-infrared light from the acquired face image, and detecting a pupil position; how distorted the pilot light image formed on the pupil from a circle and its distortion time Determining and alerting whether to be drowsy based on the information; Characterized in that comprises a.

In addition, in the eye image-based drowsiness detection method of the present invention, the image of the pilot light is obtained by grouping pixels having the same or similar brightness difference at the periphery around the position where the brightness difference with the periphery is maximum, and obtaining the image form of the light. Or recognizes the image of the light source and compares the degree of distortion of the pupil image or the image of the light source with a preset threshold to determine drowsiness if the distortion time longer than the threshold value lasts for more than the set time or the time the image disappears for more than the set time. It is done.

In addition, the eye image-based drowsiness detection apparatus of the present invention for achieving the above object, the illumination means for irradiating a predetermined near-infrared light to the face of a person for eye image acquisition, and the face image of the person irradiated with the near-infrared light The image acquisition means for acquiring and the position where the difference in brightness value between adjacent pixels is maximum in the face image acquired by the image acquiring means to detect the pupil, and in response to the light irradiated from the light source, And an image processing means for detecting sleepiness based on the degree of deformation and time information of the image, and an alarm means for alerting the driver to sleepiness according to the processing result of the image processing means.

An eye image based drowsiness detection method and an eye image based drowsiness detection apparatus of the present invention made as described above will be described below with reference to the accompanying drawings.

In the eye image-based drowsiness detection method of the present invention, pilot light as a reference light image for drowsiness determination is irradiated. In other words, to determine the drowsiness is to make a specific image to the pupil and detect the image and the image formed on the pupil and determine whether or not drowsiness based on the degree of deformation and the duration of the image. The pilot light uses infrared light (infrared light source) in order not to disturb the driver's vision. In particular, near-infrared light is used. When near-infrared light is irradiated toward a human face, particularly the eye, the near-infrared light appears in the pupil in approximately two reactions. One is that the light reflected from the retina is condensed into the pupil by the concave mirror effect of the retina, and the other is the reflection of the pilot light as an original image on the pupil having the convex mirror effect.

The former detects the pupil and detects drowsiness on the basis of the detected degree of deformation of the pupil and the duration of the deformation. The latter detects the image of the light source in the pupil and the detected shape of the light source. Drowsiness is detected based on the degree of deformation and the duration of deformation.

To this end, a camera having a built-in near-infrared light source (infrared LED, etc.) may be used as the lighting means, or an image acquisition device in which the near-infrared light source and the camera are separated may be provided. The camera uses not only a general CCD camera or a CMOS camera but also a camera that can be built in or external to a digital camera or a camera phone.

1 shows an example of a drowsiness detecting apparatus according to the present invention.

As an image acquisition means for acquiring a face image of a human, in particular, an eye 10 image, a camera 20 having a built-in near infrared light source or a camera phone 30 having a built-in camera separately from a near infrared light source (not shown) is provided. . The image processor 21 detects eyes from the face image acquired by the image capturing means, detects pupils, detects an image of a light source formed on the eyes, and determines whether the user is sleepy according to the distortion degree and duration. And an alarm generator 40 generating an alarm sound according to the determination result of the image processor 21 to alert the driver.

The image processor 21 and the alarm generator 40 may be built in the camera phone 30. In this case, a camera phone with drowsiness detection and alarm function will be implemented.

The image acquisition means 20 and 30 are installed at appropriate places around the driver's seat to face the driver's face.

[First Embodiment]

2 is a diagram illustrating an example of an eye image according to an eye image processing process according to a first embodiment of the present invention, and FIG. 3 is a flowchart illustrating a drowsiness detection method according to the first embodiment of the present invention.

When near-infrared light is irradiated on the driver's face from the near-infrared light source, the light is reflected in the retina of the eye through the pupil of the eye 10 and focused at the pupil area, so that the pupil part appears brightest in the facial image. As described above, after irradiating light from the near-infrared light source and acquiring a face image of a person using the image acquisition means 20 and 30, the image processing unit 21 processes the image as follows.

That is, as described above, the pupil area around the pupil is low in brightness, so that the difference in brightness between the pupil and its periphery has a maximum value in the entire face image. Therefore, when the position where the difference in brightness is maximized for the entire image of the face or all the pixels in a certain area is found, the position of the pixel becomes a point of the pupil of the pupil.

If a group of pixels having a brightness value equal to or greater than or equal to a threshold is found around the pixel at the position where the searched brightness difference is the maximum value, the area is immediately a pupil. In this way, the position of the pupil and the position (size) and shape of the pupil are found. The shape of the pupil maintains a circle at normal times, but as the eyelid is closed, the eyelid hides the passage of the light source (the pupil). The image of the pupil disappears completely from the acquired eye image when the eye starts to deform and the eye is completely closed. Therefore, when the shape of the pupil phase is deformed, that is, the pupil's distorted degree is longer than the preset threshold, or when the pupil phase completely disappears for more than the preset time, it is judged to be drowsiness. Raise the appropriate alarm.

In FIG. 2, (a) is an image of a face around the eyes, (b) is an image of the pupil shining brightly at the detected eye position, (c) is a comparison area around the pupil, and (d) is an eyelid characterized by drowsiness As this descends, the image of the pupil is deformed, (e) shows the shape of the eyes with the eyelids completely closed.

Figure 3 shows each step of the drowsiness detection method according to the first embodiment of the present invention described above. The first step S1 is a near infrared ray irradiation step. Since the amount of reflection of light through the pupil varies according to the relative position of the light source and the eye, the light source is adjusted in front of the driver by adjusting the position and direction of the light source so that the light of the light source is reflected at the center of the retina. The phone 30 and the near infrared light source are operated.

The second step S2 is a step of acquiring the driver's face image. The camera and the near-infrared light source are operated to acquire the driver's face image (see a in FIG. 2). The captured driver's face image is converted into a digital signal and processed as follows.

The third step S3 is a step of calculating the pixel brightness difference for detecting the position of the pupil. As shown in (b) of FIG. 2, the pupil appears brightest and the pupil periphery appears dark, so it is to find a place where the difference in brightness is greatest among the face images. To this end, the image processor 21 reads the brightness value of each pixel in the input face image and searches for a search (comparative) area displayed in a rectangular or circular shape spaced apart from a pixel as shown in FIG. After calculating the average of the brightness values of the pixels therein, the brightness difference between two values between adjacent pixels is calculated.

The fourth step S4 is a step of determining the pixel position having the maximum brightness difference. That is, the position of the pixel having the largest value of the brightness difference between the adjacent pixels is found.

The fifth step S5 is a step of determining whether the pupil is detected in the face image, particularly the eye image. In this step, if the brightness value of the found pixel is greater than or equal to the threshold value, the pupil is regarded as a pupil. If the brightness value is less than or equal to the threshold value, the eyelid is considered closed as shown in FIG. Of course, time information should be considered at this stage. In other words, it is determined that drowsiness is a case in which the eyelids are closed (the time when the pupils are not detected) lasts more than a predetermined time set to determine that they are drowsiness, so that the case of simply blinking the eyelids is not mistaken. It is.

The sixth step S6 is a step of detecting the shape of the pupil. In the previous step, the pupil is defined as a pupil by finding a group of pixels with the same or similar brightness within a certain distance from the pupil (pixel position of maximum brightness difference) and grouping them into a single mass. It is to look at the form. If the pupil's position and shape are correctly detected in the pupil, the pupil remains circular.

The seventh step S7 is a step of determining the degree of distortion of the pupil (deformation due to drowsiness). That is, the degree of deformation of the shape of the group of pixels found in the previous step from the round shape is calculated and compared with the set threshold.

The eighth step S8 is to perform drowsiness detection and alarm. That is, when eyes are closed by drowsiness, near-infrared rays are blocked by the eyelids, and the pupils of the eyes are deformed into a shape that is not rounded, so that it can be determined that it is a drowsy state, so that the alarm generator 40 operates appropriately. By drawing attention to the driver.

As shown in (d) of FIG. 2, when the eyes are half-closed, the image of the pupil formed in the pupil appears as a deformed shape rather than a rounded shape. As shown in (e) of FIG. Since it disappears, this phenomenon is considered along with the time information to detect and alert drowsiness.

When the drowsiness comes up, the eyelids are not completely wounded from the beginning, but it is common to repeat some of them after reopening, and eventually the eyelids are completely closed.In the case of simply blinking, the pupil disappears. It appears very short and the duration of the distortion of the pupil by the upper eyelid appears very short. To distinguish this drowsiness pattern from the normal eye blink pattern, as described above, the presence or absence of the pupil and the degree of deformation of the pupil are described. Considered together is the most desirable and the basis for accurate determination.

In the first embodiment of the present invention, since the light source is for determining the pupil position as the passage of the reflected light rather than the illumination of the face part, the luminance of the light source may not be high.

In addition, the algorithm does not require the geometric features of the face to search for the eye position, and finds the pupil by the difference between the brightness value of each pixel and the brightness of the surrounding pixels in the original image without the binarization process for extracting the boundary of the eyelid. This simple and eye detection speed is very fast. In particular, the near-infrared light source uses the location and shape of the pupil that is reflected and condensed from the retina, which acts as a concave mirror, so that it can be applied to the light reflected from the lens when wearing glasses as well as in environments with poor external contrast conditions such as at night or in dark tunnels. In spite of this, it is possible to accurately locate the eye.

Second Embodiment

4 is a diagram illustrating an example of an eye image according to an eye image processing process according to a second embodiment of the present invention, and FIG. 5 is a flowchart illustrating a drowsiness detection method according to a second embodiment of the present invention.

When near-infrared light is irradiated on the driver's face from the near-infrared light source, the light is reflected from the pupil, which serves as a convex mirror of the eye 10, so that the image of the light source forms on the pupil area, which is the brightest in the face image. In the second embodiment of the present invention, the color of the pupil appears in various ways according to the race, and since it has various colors such as brown, gray, and blue but also serves as a convex mirror, the image of the light source is concentrated on the pupil. This phenomenon is used to detect and determine drowsiness based on the image of the light source formed on the pupil as a kind of pilot light image. In the second embodiment of the present invention, after irradiating light from a near-infrared light source and acquiring an image of a human face using the image acquisition means 20 and 30, the image processing unit 21 processes it as follows.

As described above, when the near-infrared light source is irradiated to the driver's face, the pupil part of the human eye acts as a convex mirror, thereby forming a bright light source image on the pupil part. At this time, in order to make the image of the light source circular, a near-infrared LED is arranged so that the shape of the light source is circular.

When the bright light image of circular behavior is formed on the pupil area, the peripheral area is low in brightness so that the difference in brightness between the pilot light image and the peripheral part has the maximum value among the entire face images. Therefore, when the position where the difference in brightness is maximized for the entire image of the face or all the pixels in a certain area is found, the position of the pixel becomes a point of the image of the bright light source formed on the pupil.

When a set of pixels having a brightness value equal to or greater than or equal to a threshold value is found around the pixel at the position where the searched brightness difference is the maximum value, and the groups are grouped, the area becomes the shape and size of the image of the light source. In this way, the position of the pupil and the size and shape of the image of the light source are found. The shape of the image of the light source is normally maintained in a circle, but as the eyelid is closed, the eyelid covers the passage of the light source. When the eye starts to deform and the eyes are completely closed, the image of the light source disappears completely from the acquired eye image. Therefore, when the shape deformation of the light source image, that is, the distortion degree of the light source image is longer than the preset threshold, or longer than the preset time, or the time when the light source image disappears completely, the user determines that it is drowsy. The appropriate alarm is triggered.

In Fig. 4, (a) is a face image around the eyes, (b) is an image of a light source shining brightly on the pupil at the detected eye position, (c) is a comparison area around, and (d) is a characteristic of drowsiness. As the eyelids descend, the image of the light source deforms, and (e) shows the shape of the eyes with the eyelids completely closed.

Figure 5 shows each step of the drowsiness detection method according to the second embodiment of the present invention described above. The first step S11 is a near infrared ray irradiation step. Since the position of the image of the light source formed on the pupil part varies according to the relative position of the light source and the eye, the light source is installed in front of the driver by adjusting the position and direction of the light source so that the light of the light source forms the center of the pupil as much as possible. Or operate the camera phone 30 and the near infrared light source.

The second step S12 is obtaining a driver's face image. The camera and the near-infrared light source are operated to acquire a driver's face image (see a in FIG. 4). The captured driver's face image is converted into a digital signal and processed as follows.

The third step S13 is a step of calculating the pixel brightness difference for detecting the position of the image of the light source. As shown in (b) of FIG. 4, the light source image reflected from the pupil is dark and the light source image reflected from the pupil is the brightest in the face image, especially the eye image, so that the difference in brightness among the face images is greatest. The image processor 21 reads the brightness value of each pixel in the input face image, and as shown in (c) of FIG. After calculating the average of the brightness values of the pixels present, the brightness difference between adjacent pixels is calculated.

The fourth step S14 is a step of determining the pixel position having the maximum brightness difference. That is, the position of the pixel having the largest value of the brightness difference between the adjacent pixels is found.

The fifth step S15 is a step of determining whether the image of the light source is detected in the face image, particularly the eye image. In this step, if the brightness value of the found pixel is equal to or greater than the threshold value, it is regarded as an image of the light source. If the brightness value is less than or equal to the threshold value, the eyelid is considered closed as shown in FIG. Of course, time information should be taken into account at this stage. In other words, do not misjudge the case that the eyelid is closed by simply blinking the eyelid by judging that the time when the eyelid is closed (the time when the image of the light source is not detected) lasts more than a predetermined time set in order to determine the drowsiness. It is to avoid.

The sixth step S16 is a step of detecting the shape of the image of the light source. Considering the image of the light source in the previous step, it finds a group of pixels having the same or similar brightness within a certain distance from the image of the light source (pixel position of the maximum brightness difference), binds them together, and forms this area as a mass. It is defined as a phase and the shape of the image of the light source is examined. The image of the light source remains circular if the pupil has correctly detected the position and shape of the image of the light source.

A seventh step S17 is a step of determining the degree of distortion (deformation due to drowsiness) of the image of the light source. That is, the degree of deformation of the shape of the group of pixels found in the previous step from the round shape is calculated and compared with the set threshold.

The eighth step S18 is to perform drowsiness detection and alarm. That is, when the eyes are closed by drowsiness, the near-infrared rays are blocked by the eyelids, so that the image of the light source is deformed into a shape that is not round, so that it can be determined that it is a drowsy state. By drawing attention to the driver.

As shown in (d) of FIG. 4, when the eyes are half-closed, the image of the light source formed on the pupil appears in a deformed form rather than a rounded form. When the eyes of the driver are completely closed, as in FIG. Since it disappears, this phenomenon is considered along with the time information to detect and alarm drowsiness.

When the drowsiness comes up, it is common to see a pattern in which the eyelids are not completely wounded at the beginning, but after repeating some of them again, and ultimately, the eyelids are completely closed. Is very short and the duration of distortion of the light source image by the upper eyelid is very short. Therefore, to distinguish this drowsiness pattern from the normal eye blink pattern, as described above, the light source image and the degree of deformation of the light source are described. It is most desirable to consider together with the time information and is the basis for accurate determination.

In the second embodiment of the present invention, since the light source is intended to form an image on the eyes instead of the illumination of the face part, the luminance of the light source may not be high.

In addition, the algorithm does not require the geometric features of the face to search for the eye position, and finds the pupil by the difference between the brightness value of each pixel and the brightness of the surrounding pixels in the original image without the binarization process for extracting the boundary of the eyelid. This simple and eye detection speed is very fast. In particular, since the near-infrared light source uses the image formed on the pupil that acts as a convex mirror, the position of the eye can be accurately detected in the environment with poor external contrast conditions such as at night or in a dark tunnel, despite the light reflected from the lens when wearing glasses. Can be.

Eye image-based drowsiness detection method and apparatus of the present invention is a situation where the brightness of the driver's face continuously changes due to the change in the direction of the sun, the shade, etc. while driving, the environment of acquiring the eye image such as dark night conditions or tunnels is poor In this case, the pupil of the driver can be accurately detected, and the risk of malfunction due to the light reflected from the lens of the glasses can be minimized.

In addition, the eye image-based drowsiness detection method and the device of the present invention is almost no error compared to the eye and drowsiness detection method by the eyelid boundary detection method, the algorithm is simple, and has a fast image processing speed, so that a mobile phone equipped with a camera It can be easily mounted on a mobile terminal device.

Claims (12)

A light irradiation step of irradiating light to a human eye from a light source, A pupil recognition step of acquiring a face image of the person irradiated with the light and recognizing the position and shape of the pupil; A drowsiness determination and alerting step of detecting and determining a drowsiness and performing a drowsiness alert based on the degree of deformation of the recognized pupil shape and its holding time information; Eye image based drowsiness detection method comprising the. The pixel recognition method according to claim 1, wherein the pupil recognition step includes a pixel having a maximum difference between the brightness value of each pixel and the brightness value in the surrounding image in recognizing the location of the pupil where the light reflected from the retina is collected by irradiating near-infrared light. Eye image-based drowsiness detection method characterized by detecting the location of the pupil by finding the position of the eye. The method of claim 1, wherein the pupil recognition step detects a position of the pupil where the light reflected from the retina is collected by irradiating near-infrared light from a point where the difference between the brightness value of the pupil and the brightness value around the pupil is maximum, and the maximum brightness. Eye image-based drowsiness detection method characterized in that the recognition of the shape of the pupil by grouping the surrounding pixel group having a brightness value similar to the brightness of the difference pixels. The method of claim 1, wherein the sleepiness determination and alarming step determines the degree of eye closure according to the degree of deformation of the recognized pupil shape from a circular shape, and closes the eyes completely to set a time at which the recognized pupil disappears. Eye image-based drowsiness detection method characterized by performing a drowsiness alarm if continued. A light irradiation step of irradiating light to a human eye from a light source, An image recognition step of acquiring an image of the face of the person to whom the light is irradiated and recognizing an image of the light source formed on the pupil; A drowsiness determination and alerting step of detecting and determining a drowsiness and performing a drowsiness alert based on the degree of deformation of the shape on the recognized light source and the holding time information; Eye image based drowsiness detection method comprising the. The method of claim 5, wherein the optical image recognition step detects an image of a light source formed on the eye by irradiating near-infrared light, thereby finding a position of a pixel having a maximum difference between the brightness value of each pixel and the surrounding brightness value in the face image. Eye image based drowsiness detection method, characterized in that for detecting the position of the eye. The method according to claim 5, wherein the optical image recognition step detects an image of a light source formed on the pupil by irradiating near-infrared light, and determines the position of the pupil from a point where the difference between the brightness value of each pixel and the brightness value of the surroundings is maximum in the face image. And detecting the shape of the image of the light source by binding a group of surrounding pixels having a brightness value similar to the brightness of the pixel having the maximum brightness difference. The method according to claim 5, wherein the drowsiness determination and alarming step determines the degree of eye closure in accordance with the degree of deformation of the shape of the light source image formed on the pupil from a circle, and closes the eyes completely so that the recognized image of the light source disappears. The drowsiness detection method of the eye image, characterized in that to perform a drowsiness alarm if the duration is longer than the set time. Irradiating the human eye with pilot light as a reference light image for drowsiness determination from a near infrared light source, Acquiring a face image of a person to which the near infrared light is irradiated; Detecting a pupil position by searching a pilot light image appearing in response to the irradiated near-infrared light from the acquired face image; Determining and alerting to sleepiness based on how distorted the pilot light image formed in the pupil is from a circle and the distortion time information; Eye image based drowsiness detection method comprising a. 10. The method of claim 9, wherein the pilot light image is obtained by grouping pixels having the same or similar brightness difference at the peripheral part around the position where the brightness difference with the peripheral part is maximum, and obtaining the image form of the light and recognizing it as an image of a pupil or a light source. The eye image-based drowsiness is determined by comparing the degree of distortion of the image of the pupil image or the light source with a preset threshold when the distorted time longer than the threshold value continues for more than the set time or the time the image disappears continues for more than the set time. Detection method. Illumination means for irradiating a predetermined near-infrared light to the face of the person for eye image acquisition; Image acquisition means for acquiring a face image of the person to which the near-infrared light is irradiated; Detects the position where the difference in brightness value between adjacent pixels is maximum in the face image acquired by the image capturing means, detects the pupil, and detects the image or pupil of the light source shown in the pupil in response to the light emitted from the light source. Image processing means for determining sleepiness based on the time information; Eye image-based drowsiness detection device characterized in that it comprises an alarm means for performing a drowsiness alert to the driver in accordance with the processing result of the image processing means. In a cell phone with a built-in camera, Illumination means for irradiating a predetermined near-infrared light to the face of the person for eye image acquisition; A camera for acquiring a face image of a person to which the near infrared light is irradiated in association with the lighting means; Detects the position where the difference in brightness value between adjacent pixels is maximum in the face image acquired by the camera, detects the pupil, and detects the image or pupil of the light source appearing in the pupil in response to the light emitted from the light source, and the degree of deformation and time information. Image processing means for determining whether drowsiness is based on; And a built-in alarm means for alerting the driver to a drowsiness alarm according to the result of the processing of the image processing means.

Images