JPH0883344A - Picture processor and personal state judging device - Google Patents

Abstract

PURPOSE: To detect features in a short time even when disturbance light is sharply changed by measuring the illuminance of a peripheral part around a subject to be detected and extracting the features of the subject by picture processing using a rectinal reflection image when the illuminance is dark or extracting features by picture processing independent of the rectinal reflection image at the time of bright illuminance. CONSTITUTION: A bright/dark signal is inputted from an illiminance sensor, and when the signal is, for instance, less than a previously determined illuminance threshold of 500Lx the signal is judged as a dark signal and the degree of awakening is judged. When the signal is judged as a bright signal, binarizing processing is executed to the signal by the use of bright illuminance threshold. In an obtained binary image, eyebrows 2, eye areas 8, nose holes 3, and a mouth area 9 become black areas and the other part becomes a white area. Then the eye areas 8 are extracted. At this time, eyebrows 2 or the like are expressed as a peak P1 of the number of black picture element. Since a peak p1 corresponding to the eyebrows 2 and a peak P2 corresponding to the eye areas 8 are large and a peak p3 corresponding to the nose holes 3 is small with respect to peak patterns, the 2nd peak p2 can be specified as the eye areas 8.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus for extracting a facial feature of a person to be detected, and a person state determination for detecting the behavior of the person to be detected such as a dozing state or a look aside. It relates to the device.

[0002]

2. Description of the Related Art FIG. 30 shows, for example, JP-A-60-1583.
It is a block diagram of the eye position detection apparatus using the conventional driver imaging device shown in Japanese Patent Laid-Open No. 03-2003. In the figure, 1 is a driver, 10a and 10b are CCD cameras, and 13 is an infrared LE.
D and 14 are LED drive circuits that supply a current to the infrared LED 13, and 100 is a driver behavior detection circuit.

Next, the operation will be described. Infrared LED
13 illuminates the driver 1 in the driver's seat. The image of the driver 1 is input to the CCD cameras 10a and 10b installed at a position where a predetermined area including the face of the driver 1 can be photographed. The input image is input to the driver behavior detection circuit 100,
The positions of eyes and the orientation of the face have been extracted by image processing such as edge detection and pattern recognition.

On the other hand, as shown in the paper "Trial manufacture of eye-gaze detection device which allows pupil extraction processing and head movement" (Journal of the Institute of Electronics, Information and Communication Engineers D-II Vol.J76-D-II No.3) In addition, when the face is illuminated by the coaxial epi-illumination device, the retina reflection image can be significantly captured, and the eye position can be detected by very simple image processing such as binarization processing.
It is the illumination that has the same structure as the illumination direction of the illumination light and the optical axis of the camera). FIG. 31 shows a configuration of a drowsiness detection device using this device.
No. 502 specification. In the figure,
1 is a detection target person, 10 is a CCD camera installed at a position where a predetermined area including the face of the detection target person 1 can be photographed, 1
1 is a visible light cut filter for cutting visible light components, 1
2 is a half mirror installed at an angle of 45 degrees with respect to the CCD camera, 13 is an emission wavelength of 860 nm, and a directivity angle is ± 20.
Infrared LED that emits light at a time and reflects light to the half mirror 12 to illuminate the person to be detected 1, 14 is an LED drive circuit that drives the infrared LED, and 30 is an image that temporarily stores the image signal output from the CCD camera. A photographed image input memory, which is an image memory, 31 is a timing generation circuit that gives a timing signal to a CCD camera, a photographed image input memory 30, etc., 40 is a feature extraction circuit that extracts a pupil from the contents of the photographed image input memory, and 50 is feature extraction. A wakefulness determination circuit that determines the wakefulness from the output of the circuit, and 51 is an alarm output circuit that outputs a warning.

Next, the operation will be described. In FIG. 31, the half mirror 12 reflects half of the illumination light of the infrared LED 13 irradiated on the optical path l1 and irradiates the face of the detection target person 1 on the optical path l2. The image of the person to be detected 1 passes through the half mirror 12 in the optical path 13 and half of the light is CCD.
After reaching the camera 10, the CCD camera 10 detects the person 1 to be detected.
Capture the image of. At this time, the optical axes of the optical paths l2 and l3 are coaxial with each other when viewed from the detection target (coaxial epi-illumination). At this time, the captured image of the person to be detected is as shown in FIG. In FIG. 32, 2 is eyebrow, 3 is nostril, 4 is iris,
Reference numeral 5 is a sclera, 6 is a pupil, and 7 is a face surface. With the coaxial epi-illumination having the above configuration, the pupil 6 is observed as if the pupil were shining due to the light reflected by the retina,
Compared to the surface of the face and other feature points, the brightness is significantly higher. This is because the retina has the property of returning reflected light in the same direction as incident light. FIG. 33 shows the luminance distribution of the captured image. The waveform shown in the lower portion of FIG. 33 shows the luminance distribution along the line AA in the obtained facial image (the upper portion of FIG. 33), and the luminance of the pupil position is clearly different from that of the other portions. It turns out that is high. The face image captured by the CCD camera 10 is once input to the captured image memory 30. FIG. 34 shows a processing flow of drowsiness detection by this apparatus. This processing flow is the feature extraction circuit 40 in FIG.
And the awakening degree determination circuit 50. The feature extraction circuit 40 inputs the face image in ST10 and extracts only the pupil in ST11, and thus performs the binarization process with an appropriate luminance threshold th1. The obtained binarized image is as shown in FIG. In this image, only the pupil has two circles as a white area, and the other portions have a black area. Therefore, in ST12, white pixels are projected in the Y-axis direction (the projected image is shown on the right side of FIG. 35), and in ST13. By obtaining the area of the projected image, the apparent size of the pupil can be calculated. In ST14, when the area is equal to or more than a predetermined value, it is determined that the eyes are open, and when the area is less than the predetermined value, it is determined that the eyes are closed. This eye opening / closing information is input to the awakening degree determination circuit 50, and the awakening degree determination circuit 50 outputs ST20.
In step S21, it is determined whether eye closing information is continuously input for 3 seconds or more. If the eye closing state is 3 seconds or more, it is determined that the person is in a dozing state, and an alarm output command is output to the alarm generation circuit 51 in ST21. . When the eye closed state is less than 3 seconds, the process proceeds to ST10.

[0006]

The conventional image processing apparatus for extracting the feature on the face is configured as described above, and the driver photographing apparatus shown in FIG. 30 detects the position of the pupil. Then, for example, a filter operation is performed on the obtained image to detect an edge, and a Hough transform process for obtaining a circle corresponding to the pupil is performed based on the edge shape to perform pattern recognition to search for the pupil. Also, in the above edge detection, there is a problem that a circle cannot be obtained unless the edge can be detected well due to noise or the like, and image processing is complicated and takes time. The prior art drowsiness detection device (FIG. 31) using coaxial incident light operates normally when used in a condition where the ambient light is small, but does not work under conditions where the ambient light is large, such as when used in a vehicle. There is a problem in that the retina reflection image cannot be taken due to extremely strong ambient light (sunlight) which is 100 to 1000 times or more stronger than the illuminating light for the purpose. FIG. 36 shows an explanatory view for explaining the influence of the above-mentioned ambient light. The waveform shown in the lower part of FIG. 36 shows the luminance distribution along the line AA in the obtained facial image (the upper part of FIG. 36), and the curve A is a daytime image in clear weather, which is imaged by coaxial epi-illumination. The curve B is a daytime image in rainy weather, which is imaged by coaxial incident light, and the curve C is a nighttime image, which is imaged by coaxial incident light.
In the case of coaxial incident light, as the ambient light becomes larger (curve C; night → curve B; daytime / rainy weather → curve A; daytime ・
It can be seen that the brightness of the portion other than the pupil 6 becomes high and the pupil is difficult to identify. The image captured by the curve A is as shown in FIG. In this image, the sclera 5 appears slightly darker than the face surface 7, the iris 4 appears darker and the pupil 6 appears even darker than the sclera 5. As described above, the image processing method in the above-mentioned drowsiness detection device has a problem that the feature points on the face cannot be extracted under the condition that the ambient light condition changes greatly.

The present invention has been made in order to solve the above-mentioned problems, and the characteristics on the face of the person to be detected and the state of the person to be detected even under the condition that the ambient light greatly changes, such as in-vehicle conditions. It is an object of the present invention to provide an image processing device and a human state determination device that can detect the image in a relatively short time.

[0008]

An image processing apparatus according to claim 1 of the present invention is an optical input unit for inputting an image of a predetermined area including a face of a detection target person, illuminating the detection target person,
An optical axis connecting the detection target person and the light input means, and illumination means arranged so that the illumination light illumination direction is substantially coaxial at least in the vicinity of the detection target person, and detects the light and darkness around the detection target person. The feature on the face can be extracted from the photographed image of the person to be detected output from the light input / output unit by the two or more image processes according to the light and dark, and A feature extraction means is provided for selecting one image process from two or more image processes or one extraction result from two or more image processes according to the output brightness signal. .

An image processing apparatus according to a second aspect of the present invention is an optical input means for inputting an image of a predetermined area including the face of a person to be detected and outputting a light / dark signal indicating the light / dark of the input image. Illuminating means for illuminating a detection target person, and an optical axis connecting the detection target person and the light input means, and an illumination light illumination direction are arranged so as to be substantially coaxial at least in the vicinity of the detection target person, and Features on the face can be extracted from the photographed image of the person to be detected output from the light input means by two or more image processes according to the light and dark, and the light and dark signals output by the light input means can be used to obtain 2 Select one image processing from two or more image processing,
Alternatively, a feature extraction means for selecting one extraction result from the extraction results obtained by two or more image processes is provided.

An image processing apparatus according to a third aspect of the present invention is a light input means for inputting an image of a predetermined area including a face of a detection target person, illuminating the detection target person, and the detection target person and the light. Illuminating means arranged so that the optical axis connecting the input means and the illumination light illumination direction are substantially coaxial at least in the vicinity of the detection target person, and the detection target person or another person can be switched depending on the brightness. With the switch that outputs a signal corresponding to light and dark, and the feature on the face can be extracted by two or more image processes according to light and dark from the photographed image of the detection target person output from the light input unit. , One of the two or more image processes is selected according to the brightness signal output from the switch, or 1 is selected from the extraction results of the two or more image processes.
A feature extraction means for selecting one extraction result is provided.

According to a fourth aspect of the present invention, there is provided an image processing apparatus, wherein light input means for inputting an image of a predetermined area including a face of a detection target person, illuminating the detection target person, and the detection target person and the light The optical axis connecting the input means and the illumination light illumination direction are arranged so as to be substantially coaxial at least in the vicinity of the detection target person, and the image signal of the detection target person output from the light input means It is possible to extract the feature on the face from the photographed image of the person to be detected, which is output from the light input unit, by the brightness determination unit that determines the lightness or darkness of the photographed image by two or more image processes according to the lightness and darkness. In addition, one image processing is selected from the two or more image processings by the brightness signal output from the brightness determining means, or one extraction result is selected from the extraction results by the two or more image processings. To Those having feature extraction means for-option.

In the image processing apparatus according to claim 5 of the present invention, each of the feature extracting means extracts a pupil by a retina reflection image when a dark signal is input and extracts a nostril when a bright signal is input. It was designed to do.

According to a sixth aspect of the present invention, there is provided an image processing apparatus, wherein light input means for inputting an image of a predetermined area including a face of a detection target person, illuminating the detection target person, and the detection target person and the light Illumination means arranged so that the optical axis connecting the input means and the illumination light illumination direction are substantially coaxial at least in the vicinity of the detection target person, and a photographed image of the detection target person output from the light input means. The facial feature can be extracted by two or more image processes depending on the lightness and darkness, and the feature extraction means for selecting the final extraction result according to the extraction result by the image process is provided. .

In the image processing apparatus according to claim 7 of the present invention, the feature extracting means selects the extraction result by the image processing for light when the pupil cannot be detected by the image processing for dark. Is.

According to an eighth aspect of the present invention, in the feature extracting means of the sixth aspect, when the size of the pupil extracted by the image processing depending on whether it is light or dark is small, a bright image is obtained. The extraction result of the process is selected.

An image processing apparatus according to a ninth aspect of the present invention is the image processing apparatus described above, further comprising an illumination / photographing control means for controlling at least one of a photographing condition of the light inputting means and an illuminating condition of the illuminating means. The illumination / photographing control means extracts the light / darkness information from any of the brightness / darkness detecting means, the light inputting means, the switch, the brightness determining means, or the feature extracting means, and changes the above-mentioned photographing condition or the above lighting condition. .

In the image processing apparatus according to the tenth aspect of the present invention, the illumination / photographing control means turns off the illumination light when the light / dark information is bright, and the feature extracting means performs the image processing for light from the photographed image to the face. The above feature is extracted, and when the light and dark information is dark, the illumination light is turned on, and the feature extracting means extracts the feature on the face from the photographed image by the image processing for darkness.

According to an eleventh aspect of the present invention, there is provided a person state judging device comprising any one of the above image processing apparatuses and a person state judging means for judging a person state from an output of the image processing apparatus.

[0019]

According to the image processing apparatus of claim 1 of the present invention, the illuminance around the detection target person is measured by the light / dark detection means, and when dark, the image processing utilizing the retina reflection image is used to characterize the detection target person. When it is bright, it operates so as to extract the features of the detection target person using image processing that does not focus on the retinal reflection image when it is bright. You can catch it.

Further, the image processing apparatus according to claim 2 obtains light and dark information around the person to be detected by the light input means having a function of outputting a light and dark signal indicating the light and dark of the input image, and the light and dark information is obtained in the same manner as described above. Since the operation is performed so as to select the appropriate image processing, the feature on the face can be captured even if the brightness around the detection target person changes.

Further, the image processing apparatus according to the third aspect obtains the light and dark information around the person to be detected by the switch which outputs a signal corresponding to the light and dark when the person to be detected or another person switches in accordance with the light and dark. Similarly to the above, the operation is performed so as to select the image processing according to the lightness and darkness, so that the feature on the face can be captured even if the brightness around the detection target person changes.

Further, in the image processing apparatus according to the fourth aspect, the lightness / darkness information around the person to be detected is determined by the lightness / darkness judging means for judging the lightness / darkness of the photographed image using the image signal of the person to be detected outputted from the light inputting means. In the same manner as described above, the image processing is performed so as to select the image processing according to the brightness, so that the feature on the face can be captured even if the brightness around the detection target person changes.

According to the image processing apparatus of the fifth aspect, when the dark signal is input, the pupil is extracted by the retinal reflection image, and when the bright signal is input, the nostril is extracted. Feature extraction can be performed.

Further, since the image processing apparatus according to the sixth aspect operates so as to test the extraction result once and then select the image processing or the extraction result in accordance with the extraction result of the image processing in the feature extracting means, the detection accuracy is improved. Goes up.

Further, when the pupil cannot be detected by the image processing for darkness, the image processing apparatus according to claim 7 operates so as to select the extraction result by the image processing for light, and the result by the image processing for darkness. The detection accuracy is good because of priority.

According to the image processing apparatus of the eighth aspect, the size of the pupil is obtained by either image processing of light and dark, and the light and dark information is obtained from the size of the pupil.

According to the image processing apparatus of claim 9, the light / dark information is taken out from any one of the light / dark detecting means, the light inputting means, the switch, the lightness / darkness determining means, or the feature extracting means, and the light inputting means is photographed. Change the condition or the lighting condition of the lighting means.

The image processing apparatus according to the tenth aspect turns off the illumination light when the light / dark information is bright, and turns on the illumination light when the light / dark information is dark.

The person state determining apparatus according to claim 11 uses the extraction result output from the image processing apparatus having each of the above configurations to detect the open / closed state of the eyelids, the direction of the face, etc. Determine various person states.

[0030]

【Example】

Example 1. A doze detection device according to an embodiment of the present invention will be described below with reference to the drawings. In FIG. 1, 20 is an illuminance sensor that is installed around the detection target person 1 and detects illuminance, and 41 is a feature extraction circuit 41 that can perform two image processes of a dark feature extraction process and a bright feature extraction process. Is.
Reference numerals 10 to 14, 30, 31, 50 and 51 are constituent elements that operate in the same manner as the conventional example.

Next, the operation will be described. In FIG. 1, the half mirror 12 reflects half of the illumination light of the infrared LED 13 irradiated on the optical path l1 and irradiates the face of the detection target person 1 on the optical path l2. The image of the person to be detected 1 passes through the half mirror 12 in the optical path 13 and half of the light is CCD.
Upon reaching the camera 10, the CCD camera captures the image of the person to be detected 1. At this time, the optical axes of the optical paths l2 and l3 are almost coaxial or within a predetermined angle (2
Degree) (coaxial epi-illumination). In this embodiment, it is set to 0 °, and the angle formed by the half mirror 12 and the optical axis 13 of the CCD camera 10 is 45 °. CCD
The image signal output from the camera 10 is once input to the captured image memory 30.

FIG. 2 is a processing flow showing the operations of the feature extraction circuit 41 and the awakening degree determination circuit 50. The operation until the face image is input in ST10 is the same as that of the conventional example shown in FIG. In ST50, a brightness signal is input from the illuminance sensor 20, and in ST51, for example, a predetermined illuminance threshold value, for example, when the illuminance is 500 Lx or less, it is determined to be “dark”, the process proceeds to ST11, and the same as the conventional example. The awakening degree is determined by the action. When it is determined to be “bright” in ST51, the binarization process is performed with the brightness threshold value th2 for light (<threshold value for darkness th1) in ST41. The image in the bright case is as shown in Fig. 37.
As shown in FIG. 3, the obtained binarized image includes eyebrows 2, eye regions 8,
An image in which the nostrils 3 and the mouth area 9 are black areas and the other portions are white areas. Next, in ST42 and ST43, the eye region is extracted. In the extraction, first in ST42, luminance projection of black pixels in the Y-axis direction is performed as shown in the right part of FIG. For example, when the obtained binarized image is represented by G2 (x, y) (x = 0 to 255, y = 0 to 255), the projection on the Y axis is

[0033]

[Equation 1]

It is shown by. At this time, the eyebrows 2, the eye area 8,
Nostrils 3, mouth areas 9 and the like have peaks P1, P2, P with black pixels
3, P4. In this mountain pattern, the mountain P1 corresponding to the eyebrows and the mountain P2 corresponding to the eye area are large, and the mountain P3 corresponding to the next nostril is small, so that the second mountain P2 is specified as the eye area in ST43. To do. Then ST4
The eye opening / closing judgment is made at 4. FIG. 4 is a binarized image when the eyes are closed. In this way, the area of the mountain P2 in the eye region is small when the eyelid is closed and becomes large when the eye is opened. Therefore, when the area of the mountain P2 is larger than the predetermined value, it is determined that the eye is open, and when the area is small, the eye is closed. The operation after ST20 is the same as that of the conventional example.

In the above embodiment, when it is bright, the black pixels in the binarized image are projected in the Y-axis direction to extract the eye region, and the eye open / closed is determined based on the area of the mountain in the eye region. It may be the width of the eye area mountain. Further, even when it is dark, the white pixels in the binarized image are projected in the Y-axis direction to extract the pupil, and the eye open / closed is determined based on the area of the pupil, but the width of the mountain indicating the pupil may be used. Further, the image processing method for bright images is not limited to the above method, and eye open / closed determination may be performed by edge processing of a binarized image.

As described above, in the present embodiment, the illuminance sensor measures the illuminance around the person to be detected, and when dark, the characteristics of the person to be detected are extracted using image processing using the retina reflection image, When it is bright, it operates so as to extract the features of the detection target person using image processing that does not focus on the retina reflection image, so even if the surrounding brightness of the detection target person changes, the features on the face are always caught and doze Will be able to monitor.

Example 2. FIG. 5 shows a person state determining apparatus according to another embodiment of the present invention. When the face is directed to the left or right, the apparent pupil distance or nostril distance becomes narrower according to the face angle. This is an embodiment of the inattentive detection device that detects inattentiveness and gives an alarm. In FIG. 5, reference numeral 41 denotes a feature extraction circuit, which operates to extract features for inattentive detection. Reference numeral 52 is an inattentive judgment circuit.

Next, the operation will be described. FIG. 6 is a flowchart showing the operation of the feature extraction circuit 41 and the inattentiveness determination circuit 52. A face image is input in ST10, and ST50 is entered.
At, the brightness signal is input from the illuminance sensor 20. ST
In 51, as in the first embodiment, for example, when the illuminance threshold value is equal to or less than a predetermined illuminance threshold value, it is determined to be “dark” and the process proceeds to ST11.
After shifting to the binarization processing, in ST120, white pixels are projected in the X-axis direction as shown in FIG. In ST130, two pupil positions xq1 and xq2 are extracted. That is, the position of the pupil can be extracted by finding the positions of the peaks of the two peaks appearing in the projected image in the X-axis direction corresponding to the two pupils. In ST140, the apparent distance dpp between the pupils is obtained from the positions of the two pupils obtained in ST130, and the face orientation is estimated from the obtained apparent distance. The face angle θf is obtained from cos (θf) = dpp / Dpp. However, Dpp is the apparent pupillary distance when the face is directed to the front, and dpp is the current apparent pupillary distance.

On the other hand, in ST51, when it is equal to or more than the predetermined illuminance threshold value, it is judged as "bright", and in this case, the face angle is estimated from the interval of the nostrils. That is, ST
41, as in the first embodiment, the brightness threshold value th for brightness th
2 (<threshold value for darkness th1), the binarization process is performed, and S
At T420, luminance projection of the black pixel in the Y-axis direction is performed. Figure 3
When the luminance projection is performed on the Y axis as shown in FIG. 3, the nostril 3 appears as a small black mountain P3 below the large mountains P1 and P2. In ST420, the center yp3 of the mountain P3 is further obtained, and a window for nostril extraction is set. The window has a vertical pixel width corresponding to, for example, 2 cm above and below yp3, and has a horizontal width of 0 to 255. This window looks like the top of Figure 8. After that, luminance projection is performed in the X-axis direction. The luminance projection of the black pixel in the X-axis direction is

[0040]

[Equation 2]

It is shown by. Here, y0 is the number of pixels corresponding to 1 cm above and below. When luminance projection is performed in the X-axis direction, FIG.
Mountains Q1 and Q2 corresponding to the two nostrils appear as below, so at ST45, the nostril centers xq1 of Q1 and Q2,
By finding xq2, the position of the left and right nostrils nR (xq
1, yp3), nl (xq2, yp3). In ST46, the apparent distance between the nostrils is obtained from the positions of the two nostrils obtained in ST45, and the orientation of the face is estimated from the obtained apparent distances. Face angle θf is cos (θf) = dnn / D
nn is required. However, Dnn is the apparent nostril spacing when the face is directed to the front, and dnn is the current apparent nostril spacing.

The information about the face orientation is input to the inattentiveness determination circuit 52, and the inattentiveness determination circuit 52 determines in ST22 that the obtained orientation of the face is 20 degrees or more, and this state is 5
If it continues for 5 seconds or more, if it continues for 5 seconds or more,
It is determined that the user is looking aside, and the alarm generation circuit 51 is determined in ST23.
The alarm output command is output to. If it is less than 5 seconds, the process proceeds to ST10.

As described above, in the present embodiment, the illuminance sensor measures the illuminance in the vicinity of the person to be detected, the pupil is extracted by utilizing the retinal reflection image when it is dark, and the image is not focused on the retinal reflection image when it is bright. By extracting the nostrils using processing, features with good detection accuracy are extracted according to light and dark,
The detection accuracy of the inattentive judgment is improved.

Example 3. FIG. 9 is a block diagram showing a doze detection device according to a third embodiment of the present invention. In FIG.
15 is a CC with AGC (auto gain control)
The D camera has a function of outputting an AGC signal, which is a light / dark signal indicating the light / dark of an input image, to the outside. That is, when the illuminance is high, the CCD camera has a function of decreasing the gain of electric signal amplification after photoelectric conversion, and when the illuminance is high, a CCD camera having a function of increasing the gain of electric signal amplification after optical conversion. , This gain information is output to the outside. The operation is similar to that of the first embodiment, and instead of the brightness signal from the illuminance sensor 20, the CCD camera 15 outputs a brightness signal corresponding to the gain information to the feature extraction circuit 41. Therefore, in addition to the effect of the first embodiment, the illuminance sensor can be omitted, so that an inexpensive device can be provided.

Example 4. 10 is a block diagram showing a doze detecting device according to a fourth embodiment of the present invention. In FIG. 10, reference numeral 21 is a light / dark switch that a person switches according to the light / dark state. That is, the person to be detected or another person switches the switch 21 according to the light and dark around the person to be detected, and outputs a signal corresponding to the light and dark. The operation is similar to that of the first embodiment, and instead of the light / dark signal from the illuminance sensor 20, the light / dark signal is output from the switch 21 to the feature extraction circuit 41. Therefore, similarly to the third embodiment, the detection accuracy is improved and the illuminance sensor can be omitted, so that an inexpensive device can be provided.

In the fourth embodiment, the light / dark switch 21 that is switched according to the light / dark state is used. However, the above-mentioned device is applied to an automobile, and the light on / off switch is also used as the light / dark switch 21 to change the light / dark state. You may make it take out the corresponding signal. FIG. 11 is a partial configuration diagram showing an automobile equipped with the drowsiness detection device according to the present embodiment. In FIG. 11, 22 is a headlight switch,
A dark signal is output to the feature extraction circuit 41 when it is on, and a bright signal is output when it is off. With such a configuration, the light / dark signal is interlocked with the on / off switch of the light of the automobile, and the light / dark signal is generated by the operation naturally performed by the driver, so that the operation is not troublesome.

Example 5. 12 is a block diagram showing a doze detection device according to a fifth embodiment of the present invention. In FIG. 12, reference numeral 42 is a brightness / darkness determination circuit which inputs the data of the captured image memory 30 and determines the brightness. That is, G1 (x,
If y) is the content of the image memory, the brightness / darkness determination circuit 42
Is the value G1total for determining the brightness of the captured image

[0048]

[Equation 3]

Further, it is determined that "G1total" is "bright" when it is a predetermined value or more, and "dark" when it is less than G1total.
To judge. The feature extraction circuit 41 inputs the light / dark information of the light / dark determination circuit 42 instead of the light / dark signal from the illuminance sensor 20, and operates similarly to the first embodiment.

By detecting light and dark using the image memory in this way, a sensor for detecting light and dark can be omitted, and in addition to the effects of the first embodiment, an inexpensive device can be provided.

In the above embodiment, G1total uses the entire image data, but a part of the image may be used.

In the above embodiment, the brightness determination circuit 42 inputs the data of the photographed image memory 30 to determine the brightness. However, as shown in FIG. 13, the image signal from the CCD camera 10 is input, for example, 1/30 of the input image signal
Brightness and darkness are determined according to the integration amount for a second, and the feature extraction circuit 41
Alternatively, a light / dark signal may be output.

Example 6. Feature extraction circuit 4 of each of the above embodiments
In 1, the image processing for light and the image processing for dark were switched by inputting a signal from the outside, but one extraction is performed using the extraction result of each image processing without depending on the signal from the outside. The result may be selected. FIG. 14 is a block diagram showing such a drowsiness detection device. In FIG. 14, 43 is a dark feature extraction circuit, 44 is a bright feature extraction circuit, and 45 is a selection circuit. The darkness feature extraction circuit 43
The data of the photographed image memory 30 is input, and the eyelid open / close degree and the Y coordinate of the pupil are extracted according to the flow of FIG. That is, in a dark environment to some extent, only the pupil can be extracted as in the first embodiment, but in a bright environment, the brightness of the tip of the nose and the brightness of the pupil are close to each other, and when binarization is performed with the threshold value th1, As shown in 16, when the pupil and the tip of the nose are in a white area and luminance projection is performed on the Y axis, a mountain R1 due to the pupil and a mountain R2 due to the tip of the nose appear, and it is impossible to determine which is the pupil. In a brighter environment, many white areas appear, and it is impossible to judge whether the eyelids are open or closed. As a feature extraction result, the dark feature extraction circuit 43 outputs a value (Y coordinate yr1, open / closed degree dr1) corresponding to R1 when the pupil can be extracted, and undetectable information when the pupil cannot be specified. In FIG. 15, ST
Input a face image at 10 and 2 at threshold th1 at ST11.
Perform value conversion processing. In ST12, white pixels are projected in the Y-axis direction, and in ST60, the number of peaks in the Y-axis projection image showing the white region is detected.
Go to step 5, and determine that the pupil cannot be detected. Since the pupil can be extracted when there is only one mountain, the apparent size of the pupil can be calculated by obtaining the area of the projected image in ST13. In ST61, when the area is equal to or more than a predetermined value, the process proceeds to ST63, it is determined that the eye is open, and the Y coordinate yr1 of the pupil and the width dr1 of the mountain corresponding to the degree of opening and closing of the eyelid are obtained from the projected image. When the above area is less than the predetermined value, ST6
It is judged that the eyes are closed in 2. In ST64, ST62, ST6
The results of 3 and ST65 are output to the selection circuit.

On the other hand, the light feature extraction circuit 44 inputs the data in the photographed image memory 30, and extracts the eyelid open / closed degree and the Y coordinate of the pupil according to the flow of FIG. That is, in a bright environment, the mountains P1, P2, P3, and P4 corresponding to the eyebrows, the eye region, the nostrils, and the mouth can be extracted separately as in the first embodiment, but they are slightly dark and the contrast ratio on the face becomes small. Then, if binarization is performed with the threshold value th2, a black region appears other than the eyebrows, the eye region, the nostrils, and the mouth as shown in FIG. 18, and the mountain P2 of the eye region can be specified even if Y-axis luminance projection is performed. Disappear.
As the feature extraction result, the bright feature extraction circuit 44 outputs the eyelid open / close degree corresponding to P2 when it can be detected, and outputs undetectable information when it cannot be detected. In FIG. 17, ST1
A face image is input at 0, and binarization processing is performed at threshold th2 at ST41. In ST42, the black pixel is projected in the Y-axis direction, and in ST70, the peak of the Y-axis projected image showing the black region is detected,
When the mountain is unclear, that is, when the number of black pixels in the projected image is 0 or less than 4, for example, it is determined in ST75 that the eye region cannot be detected. When the mountain is clear, that is, when there are four areas, the second mountain P2 is specified as the eye area in ST43. In ST71, it is determined whether or not the area of the mountain P2 in the eye region is larger than a predetermined value,
If it is larger, it is determined in ST73 that the eye is open, and the Y coordinate of the eye region and the width of the mountain corresponding to the degree of eyelid opening / closing are obtained from the projected image. If it is smaller than the upper limit, it is determined that the eyes are closed in ST72.
In ST74, the results of ST72, ST73, and ST75 are output to the selection circuit.

The selection circuit 45 selects one result from the output results of the dark feature extraction circuit 43 and the bright feature extraction circuit 44 according to the flow of FIG. In this embodiment, bright is selected unless the bright feature extraction result is undetectable, and if the bright feature extraction result is undetectable, the dark detection result is selected. That is, in ST80, the extraction result of the dark feature extraction circuit 43 is input, and in ST81, the extraction result of the bright feature extraction circuit 44 is input. If the bright feature extraction result can be detected in ST82, the bright feature extraction result is selected in ST84,
If the bright feature extraction result cannot be detected, the dark feature extraction result is selected in ST83. In ST85, the selected extraction result is output to the awakening degree determination circuit 50, and the awakening degree determination circuit 5
0, and the alarm generation circuit 51 determines the dozing state according to the flow shown in FIG.
Generate an alarm. In FIG. 20, in ST100, the position of the pupil (or eye region) is input from the feature extraction circuit 41,
At ST101, this position is stored. The standard pupil (eye region) height is calculated by taking data for 5 minutes at the above-mentioned position in ST102 and obtaining the average value of the data for 5 minutes in ST103. In ST104, the position of the pupil (or eye area) is continuously input from the feature extraction circuit 41,
In ST105, the average pupil (eye region) height obtained from the data for 2 minutes is calculated. In ST106, the average pupil (eye region) height is compared with the standard pupil (eye region) height, and when the average pupil (eye region) height is lower than the standard pupil (eye region) height by, for example, 5 cm or more, it is determined to be a dozing state. A judgment is made and a buzzer (alarm) is sounded in ST107.

As described above, in the present embodiment, the feature extraction is performed by the light and dark image processing regardless of the light / dark state, and the successful feature extraction result is used. Is effective.

In the above embodiment, the Y-coordinate of the pupil or the eye region and the eyelid open / closed degree are also obtained in the opened state, and the awakening degree determination circuit determines the doze according to the position of the pupil or the eye region. Similar to 5 to 5, only the open / closed state may be output as the extraction result, and the dozing may be determined based on this.

In the sixth embodiment, the bright one is preferentially selected, but the dark one may be preferentially selected.

In the sixth embodiment, the dark feature extraction circuit 4
3. The bright feature extraction circuit 44 outputs three types of results, i.e., open eye, closed eye, and undetectable depending on whether it is detectable or undetectable, but the dark feature extraction circuit (1) determines whether the first candidate pupil is open or closed. Result and position (2) In addition to (1), the pupil second candidate pupil (y in FIG.
Open / closed judgment result (at the position of r2) and position (3) Multiple candidates are also detected corresponding to the projected image such as undetectable,
The open / close determination may be executed for each of (1) and (2). Therefore, there are five types of detection results, and the selection circuit 45
Enter multiple candidates.

The operation of the selection circuit 45 is shown in FIG. Figure 2
In ST1, the extraction result of the dark feature extraction circuit 43 is input in ST80, and the extraction result of the bright feature extraction circuit 44 is input in ST81. If the dark feature extraction result cannot be detected in ST86, the bright feature extraction result is selected in ST84. On the other hand, if the feature extraction result for darkness can be detected in ST86, it is determined in ST110 whether only the first candidate is detected. If only the first candidate is detected, ST11
At 1, the feature extraction result for darkness, that is, the first candidate is selected. If a candidate other than the first candidate is also detected, in ST82, it is determined whether or not the bright feature extraction result can be detected. If not, in ST112, the first candidate and the second dark feature extraction result are detected. Among the candidates, the candidate whose pupil position is closer to the previous extraction result is selected. If it is determined in ST82 that detection is possible, in ST113, the candidate having the closer pupil position to the bright feature extraction result is selected from the first candidate and the second candidate of the dark feature extraction result. In ST85, the selected extraction result is output to the awakening degree determination circuit 50.

As described above, by extracting the second candidate and selecting a more reliable extraction result, the detection accuracy is improved, and the feature can be well captured even in the middle region of light and dark, and the undetectable region is reduced. There is.

Example 7. In the sixth embodiment, the image processing for darkness and the image processing for light are performed, and one extraction result is selected using the extraction results in each image processing. However, this selection is performed in the same manner as in the first embodiment. Alternatively, the selection may be made according to the output of the illuminance sensor 20. 22 is a block diagram showing a doze detection device according to a seventh embodiment of the present invention. In FIG. 22, the operations of the dark feature extraction circuit 43 and the bright feature extraction circuit 44 are the same as in the sixth embodiment. The flow of the selection circuit 45 is shown in FIG. In FIG. 23, ST8
When 0, the extraction result of the darkness feature extraction circuit 43 is input, and ST8
At 1, the extraction result of the bright feature extraction circuit 44 is input. ST
If the bright feature extraction result can be detected in 82, ST86
Check whether the feature extraction result for darkness can be detected.
If it can be detected, the brightness signal is input from the illuminance sensor 20 in ST87, and it is determined in ST88 whether the brightness signal from the illuminance sensor 20 is a bright signal. If it is a bright signal, the extraction result of the bright feature extraction circuit 44 is selected in ST84. If it is a dark signal, the extraction result of the dark feature extraction circuit 43 is selected in ST83. On the other hand, if the bright feature extraction result cannot be detected in ST82, the dark feature extraction circuit 4 is operated in ST83.
Select the extraction result of 3. In ST85, the extraction result selected in ST83 or ST84 is output to the awakening degree determination circuit 50.

By doing so, in addition to the effect of the first embodiment, when features can be extracted by both bright and dark image processing, a more accurate one is selected depending on the illuminance, so that bright and dark can be selected. Detection accuracy increases in intermediate environments.

Example 8. In the sixth embodiment, the feature extraction circuit 41 performs dark image processing and bright image processing, and selects one extraction result by using the extraction results of each image processing. It is also possible to select either the light or dark image processing according to the extraction result by (1) to obtain the final extraction result. FIG. 24 is a configuration diagram showing a doze detection device according to an eighth embodiment of the present invention, and FIG. 25 is a flow of the feature extraction circuit 41 and the awakening degree determination circuit 50. In FIG. 25, first, feature extraction for dark is performed in ST90 to ST93. In ST94, it is determined whether or not the pupil has been detected by the feature extraction for darkness.
In ST14, feature extraction for darkness is performed, and the result of this extraction is used as a final extraction result, and in ST20, doze detection is performed as in each of the above-described embodiments. That is, it is determined whether or not eye closing information is continuously input for 3 seconds or more, and if the eye closing state is 3 seconds or more, it is determined that the person is in a dozing state, and an alarm output command is output to the alarm generating circuit 51 in ST21. To do. Eye closed state is 3
If it is less than a second, the process moves to ST10. On the other hand, if the pupil cannot be extracted in ST94, bright feature extraction is performed in ST40 to ST44, and the dozing detection is performed in ST47 with this extraction result as the final extraction result. That is, it is judged whether or not the eye closing information is continuously input for 3 seconds or more, and 3
If the eyes are closed for more than a second, it is determined that the person is in the dozing state, and an alarm output command is output to the alarm generation circuit 51 in ST48. When the eye closed state is less than 3 seconds, the process proceeds to ST40.

As described above, in the present embodiment, it is determined whether or not the feature extraction is possible by the dark image processing, and the subsequent image processing method is selected. The same effect as the first embodiment is obtained. Further, since the dark image processing is prioritized, the detection accuracy is good.

In the eighth embodiment, whether or not the pupil can be detected by the feature extraction for darkness is determined by only one feature extraction process. However, it may happen that the eye is closed only once. Therefore, for example, the feature extraction processing for darkness may be performed for about 2 seconds to determine whether or not darkness can be used.

Further, in the eighth embodiment, after determining whether or not the pupil can be detected by the feature extraction for darkness, the image processing is fixed to one of bright and dark, but even after fixing to light. , For example, perform feature extraction processing for darkness once every 10 minutes,
You may re-determine whether dark use can be used.

In the eighth embodiment, it is determined whether the feature extraction can be performed by the dark image processing. However, the subsequent image processing method is selected depending on whether the feature extraction can be performed by the bright image processing. The bright image processing may be prioritized.

Example 9. FIG. 26 is a flow chart of the feature extraction circuit 41 and the awakening degree determination circuit 50 related to the doze detection device according to the ninth embodiment of the present invention. Note that the configuration of the doze detection device is the same as that of FIG. Since the pupil is small when it is bright and becomes large when it is dark, in the present embodiment, when the size of the pupil extracted by the image processing according to either light or dark is small, the image processing for light and the image processing for light are performed. The extraction result by the image processing is selected, and when the size of the extracted pupil is large, the image processing for darkness and the extraction result by the image processing for darkness are selected. In FIG. 26, first, in ST90 to ST93, the pupil is extracted by image processing for darkness, and in ST95, the width of the projected image in the Y-axis direction is detected to obtain the diameter of the pupil.
If the pupil diameter is, for example, 4 mm or more, ST10 to ST
The image processing for dark is executed at 14, and if less, the image processing for light is executed at ST40 to ST44. The following operation is the same as that of the eighth embodiment.

As described above, when the bright or dark image processing is selected according to the size of the pupil, the accuracy of feature extraction is improved, and an inexpensive device without using an illuminance sensor or the like is used. Then, the same effect as the first embodiment is obtained.

Although the image processing method is selected according to the size of the pupil in the above embodiment, the same configuration as that shown in FIG.
The same effect can be obtained by selecting the extraction results of the bright feature extraction circuit and the dark feature extraction circuit according to the size of the pupil obtained by either circuit.

Example 10. FIG. 27 shows the first embodiment of the present invention.
It is a block diagram which shows the drowsiness detection apparatus by 0. In FIG. 27, reference numeral 16 denotes a shutter speed setting circuit, which inputs light / dark information from the illuminance sensor 20 and slows the shutter speed of the CCD camera 10 when the illuminance is below a certain level. If the illuminance is above a certain level, the shutter speed of the CCD camera 10 is increased. Other operations are the same as those in the first embodiment.

As described above, in this embodiment, C
Since the shutter speed of the CD camera is changed, the quality of the input image is improved and the feature extraction accuracy is improved in addition to the effect of the first embodiment.

Example 11. 28 is a first embodiment of the present invention.
It is a block diagram which shows the drowsiness detection apparatus by 1. In FIG. 28, reference numeral 17 is a filter exchange circuit for switching between the visible light cut filter 11 and the infrared cut filter 18 as the optical filter mounted on the CCD camera 10. The filter exchange circuit 17 inputs the light / dark information from the illuminance sensor 20, selects the visible light cut filter 11 when the illuminance is below a certain level, and selects the infrared cut filter 18 when the illuminance is above a certain level. Other operations are the same as those in the first embodiment.

As described above, in this embodiment, when the image is bright, the image is taken with visible light, so that an image with good contrast can be obtained.
Increased detection accuracy for light.

In the eleventh embodiment, the optical filters are exchanged according to the light and darkness. However, the coaxial incident-light photographing system having a visible light cut filter and the non-coaxial photographing system having an infrared cut filter without a half mirror are used. It is also possible to switch between shooting with a coaxial shooting system when dark and shooting with a non-coaxial shooting system when bright. In this way, the circuit for filter replacement can be omitted, and the image quality is improved because the image is taken without passing through the half mirror in the bright state.

In the tenth and eleventh embodiments, the first embodiment is used.
Although the photographing conditions of the light input means are controlled based on the brightness information of the illuminance sensor 20 shown in FIG. 5, the light input means 15, the switch 21, the brightness determination means 42, or the feature extraction means 41 described in the second to ninth embodiments. Brightness information may be extracted from any of the above to change the above-mentioned shooting conditions.

Example 12. FIG. 29 is a first embodiment of the present invention.
It is a block diagram which shows the drowsiness detection apparatus by 2. In FIG. 29, the LED drive circuit 14 inputs the brightness information from the illuminance sensor 20, and if the illuminance is below a certain level, the infrared LED 13
Is turned on, and when the illuminance is above a certain level, the infrared LED 13 is turned off. Other operations are the same as those in the first embodiment.

As described above, in this embodiment, since the infrared light is illuminated only in the dark, the face image can be surely obtained without bothering the detection target person. Further, when it is bright, no lighting is performed, so that power consumption can be saved.

In the twelfth embodiment, the ON / OFF of the optical LED is controlled according to the light and dark. However, a mechanism for moving the half mirror is provided so that the coaxial epi-illumination is performed in the dark and the half mirror is not passed in the light. The subject may be photographed directly. In this way, in the bright state, the image quality is improved because the image is taken without passing through the half mirror.

In the twelfth embodiment, the illumination condition of the illumination means is controlled based on the brightness information of the illuminance sensor 20 shown in the first embodiment, but the light input means 1 shown in the second to ninth embodiments.
The illumination condition may be changed by extracting the brightness information from any one of the switch 5, the brightness 21, the brightness determination unit 42, or the feature extraction unit 41.

In each of the above embodiments, the camera is CC
Don't worry about D. For example, a CID camera may be used.

Further, although the image pickup system is described as a single camera, a set of a plurality of coaxial incident illuminations and cameras may be used.

The coaxial epi-illumination is shown only in the case of the half mirror, but the coaxial epi-illumination is not limited to such a structure. For example, a light emitting element may be provided at the center of the lens of the camera, and the driver may be directly illuminated.

[0085]

As described above, according to the first aspect of the present invention, the light input means for inputting the image of the predetermined area including the face of the detection target person, the illumination of the detection target person, and the detection target And an optical axis connecting the person and the light input means, and an illumination means arranged so that the illumination light illumination direction is substantially coaxial at least in the vicinity of the detection target person, and a light-dark detection for detecting light and dark around the detection target person. Means and a feature on the face from the photographed image of the person to be detected output from the optical input means,
The image can be extracted by two or more image processes depending on the light and dark, and one image process is selected from the two or more image processes by the light and dark signal output from the light and dark detecting means, or two or more image processes. Since the feature extraction means for selecting one extraction result from among the extraction results according to 1. is provided, the features on the face of the detection target person can be detected in a relatively short time under both bright and dark conditions.

According to a second aspect of the present invention, the light input means for inputting an image of a predetermined region including the face of the person to be detected and outputting a light / dark signal indicating the light / dark of the inputted image, From the light inputting means, which is illuminated, the optical axis connecting the detection target person and the light input means, and the illumination means arranged so that the illumination light illumination direction is substantially coaxial at least in the vicinity of the detection target person. From the captured image of the person to be detected that is output, the facial features can be determined according to the lightness and darkness.
The image can be extracted by one or more image processes, and one image process is selected from two or more image processes by the light / dark signal output by the light input unit, or an extraction result of two or more image processes is displayed. Since the feature extraction means for selecting one of the extraction results is provided, the features on the face of the detection target person can be detected in a relatively short time under both bright and dark conditions, as described above.

According to a third aspect of the present invention, light input means for inputting an image of a predetermined area including the face of the detection target person, the detection target person is illuminated, and the detection target person and the light input means are connected to each other. Illuminating means arranged so that the optical axis to be connected and the illumination light illumination direction are substantially coaxial at least in the vicinity of the detection target person, and the detection target person or other person responds to light and dark by switching in correspondence with light and dark. The switch for outputting the signal, and the feature on the face from the photographed image of the person to be detected output from the light input means, according to the brightness.
The image can be extracted by one or more image processes, and one image process is selected from the two or more image processes by the brightness signal output from the switch, or the extraction result by the two or more image processes is selected. Since the feature extraction means for selecting one of the extraction results is provided, similar to the above, the feature on the face of the detection target person can be detected in a relatively short time under both bright and dark conditions.

According to a fourth aspect of the present invention, light input means for inputting an image of a predetermined area including the face of the detection target person, the detection target person is illuminated, and the detection target person and the light input means are connected to each other. Illumination means arranged so that the optical axis to be connected and the illumination light illumination direction are substantially coaxial at least in the vicinity of the detection target person, and a captured image using the image signal of the detection target person output from the light input means Intensity determination means for determining the lightness and darkness, and facial features from the photographed image of the detection target person output from the light input means can be extracted by two or more image processes according to the lightness and darkness, A feature that one image processing is selected from the two or more image processings or one extraction result is selected from the extraction results by the two or more image processings by the brightness signal output from the brightness determining means. Extraction Because with a stage, as described above, even in bright conditions, can be detected in a relatively short time features on the face of the detection subject in dark conditions.

Further, according to the fifth aspect, in each feature extraction means, when the dark signal is input, the pupil is extracted by the retina reflection image, and when the bright signal is input, the nostril is extracted. Therefore, various feature extractions can be performed.

According to a sixth aspect of the present invention, light input means for inputting an image of a predetermined region including the face of the detection target person, the detection target person is illuminated, and the detection target person and the light input means are connected to each other. The illuminating means arranged so that the optical axis to be connected and the illumination light illuminating direction are substantially coaxial at least in the vicinity of the detection target person, and on the face from the photographed image of the detection target person output from the light input means. The feature can be extracted by two or more image processes depending on the brightness and the feature extraction means for selecting the final extraction result according to the extraction result by the image process is provided. The feature on the face of the detection target person can be detected in a relatively short time under both the dark condition and the condition. Furthermore, the image extraction is performed once to derive the extraction result, and the final extraction result is output. detection Degree is increased.

According to the seventh aspect, when the pupil cannot be detected by the dark image processing, the operation is performed so as to select the extraction result by the bright image processing, so that the detection accuracy is further improved.

According to the eighth aspect, since the size of the pupil is obtained by image processing of either light or dark, and the light and dark information is obtained from the size of the pupil, the bright condition and the dark condition are the same as above. However, the feature on the face of the person to be detected can be detected accurately in a relatively short time.

According to a ninth aspect, the light / dark detecting means,
Since the light / dark information is taken out from any one of the light input means, the switch, the brightness determination means, or the feature extraction means to change the photographing condition of the light input means or the illumination condition of the illumination means, the detection accuracy is further improved. Has the effect of increasing.

According to the tenth aspect, since the illumination light is turned off when the light and dark information is bright, and the illumination light is turned on when the light and dark information is dark, unnecessary illumination power is used when it is bright. There is no.

Further, according to the eleventh aspect, since the person state is determined by using the extraction result output from the image processing apparatus having each of the above-described configurations, the apparatus for detecting the person state regardless of light and dark. There is an effect that can be obtained.

[Brief description of drawings]

FIG. 1 is a configuration diagram showing a doze detection device according to a first embodiment of the present invention.

FIG. 2 is a flowchart showing the operations of the feature extraction circuit and the awakening degree determination circuit according to the first embodiment of the present invention.

FIG. 3 is an explanatory diagram illustrating an operation of the doze detection device according to the first embodiment of the present invention.

FIG. 4 is an explanatory diagram illustrating an operation of the doze detection device according to the first embodiment of the present invention.

FIG. 5 is a configuration diagram showing an inattentiveness detection device according to Embodiment 2 of the present invention.

FIG. 6 is a flowchart showing the operations of the feature extraction circuit and the inattentiveness judgment circuit according to the second embodiment of the present invention.

FIG. 7 is an explanatory diagram illustrating an operation of the inattentiveness detection device according to the second embodiment of the present invention.

FIG. 8 is an explanatory diagram illustrating an operation of the inattentiveness detection device according to the second embodiment of the present invention.

FIG. 9 is a configuration diagram showing a doze detection device according to a third embodiment of the present invention.

FIG. 10 is a configuration diagram showing a doze detection device according to a fourth embodiment of the present invention.

FIG. 11 is a partial configuration diagram showing an automobile equipped with a drowsiness detection device according to a fourth embodiment of the present invention.

FIG. 12 is a configuration diagram showing a doze detection device according to a fifth embodiment of the present invention.

FIG. 13 is a configuration diagram showing another doze detection device according to the fifth embodiment of the present invention.

FIG. 14 is a configuration diagram showing a doze detection device according to a sixth embodiment of the present invention.

FIG. 15 is a flowchart showing the operation of the darkness feature extraction circuit according to the sixth embodiment of the present invention.

FIG. 16 is an explanatory diagram illustrating an operation of the doze detection device according to the sixth embodiment of the present invention.

FIG. 17 is a flowchart showing the operation of the bright feature extraction circuit according to the sixth embodiment of the present invention.

FIG. 18 is an explanatory diagram illustrating an operation of the doze detection device according to the sixth embodiment of the present invention.

FIG. 19 is a flowchart showing the operation of the selection circuit according to the sixth embodiment of the present invention.

FIG. 20 is a flow chart showing the operation of the awakening degree judgment circuit according to the sixth embodiment of the present invention.

FIG. 21 is a flowchart showing the operation of another selection circuit according to the sixth embodiment of the present invention.

FIG. 22 is a configuration diagram showing a doze detection device according to a seventh embodiment of the present invention.

FIG. 23 is a flowchart showing the operation of the selection circuit according to the seventh embodiment of the present invention.

FIG. 24 is a configuration diagram showing a doze detection device according to an eighth embodiment of the present invention.

FIG. 25 is a flowchart showing the operations of the feature extraction circuit and the awakening degree determination circuit according to the eighth embodiment of the present invention.

FIG. 26 is a flowchart showing the operations of the feature extraction circuit and the awakening degree determination circuit according to the ninth embodiment of the present invention.

FIG. 27 is a configuration diagram showing a doze detection device according to a tenth embodiment of the present invention.

FIG. 28 is a configuration diagram showing a doze detection device according to an eleventh embodiment of the present invention.

FIG. 29 is a configuration diagram showing a doze detection device according to a twelfth embodiment of the present invention.

FIG. 30 is a configuration diagram showing a conventional eye position detecting device.

FIG. 31 is a configuration diagram showing a conventional drowsiness detection device.

FIG. 32 is an explanatory diagram illustrating an operation of the conventional doze detection device.

FIG. 33 is an explanatory diagram illustrating an operation of the conventional drowsiness detection device.

FIG. 34 is a flowchart showing an operation of the conventional doze detection device.

FIG. 35 is an explanatory diagram illustrating an operation of the conventional doze detection device.

FIG. 36 is an explanatory diagram illustrating an operation of the conventional drowsiness detection device.

FIG. 37 is an explanatory diagram illustrating an operation of the conventional drowsiness detection device.

[Explanation of symbols]

1 detection target person, 3 nostrils, 6 pupils, 8 eye regions, 1
0 CCD camera, 12 half mirror, 13 infrared L
ED, 15 CCD camera with AGC, 16 shutter speed setting circuit, 17 filter exchange circuit, 18 infrared cut filter, 20 illuminance sensor, 21 light / dark switch, 22 headlight switch, 30 photographed image input memory, 31 timing generation circuit, 41 Features Extraction circuit, 42 brightness determination circuit, 43 darkness feature extraction circuit,
44 bright feature extraction circuit, 45 selection circuit, 50 awakening degree determination circuit, 51 alarm generation circuit, 52 inattentive determination circuit.

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Toshihide Satake 8-1-1 Tsukaguchihonmachi, Amagasaki City Mitsubishi Electric Corporation Industrial Systems Research Institute (72) Kenji Ogawa 840 Chiyoda-cho, Himeji City Mitsubishi Electric Corporation Himeji Works (72) Inventor Hiroyoshi Suzuki 840 Chiyoda-cho, Himeji City Mitsubishi Electric Corporation Himeji Works

Claims (11)

[Claims] 1. An optical input unit for inputting an image of a predetermined area including a face of a detection target person, an optical axis for illuminating the detection target person, and connecting the detection target person and the light input unit, and illumination light. Illuminating means arranged so that the illumination direction is substantially coaxial at least in the vicinity of the person to be detected, brightness detection means for detecting light and dark around the person to be detected, and the detection object output from the light input means. It is possible to extract facial features from a photographed image of a person by two or more image processes depending on the light and dark, and one of the two or more image processes is performed by the light and dark signal output from the light and dark detecting means. An image processing apparatus provided with a feature extraction means for selecting image processing or selecting one extraction result from the extraction results of the two or more image processing. 2. An optical input means for inputting an image of a predetermined area including a face of a detection target person and outputting a light / dark signal indicating the brightness of the input image, illuminating the detection target person, and the detection target person. And an optical axis connecting the optical input means and the illumination light illumination direction are arranged so as to be substantially coaxial at least in the vicinity of the detection target person, and the detection target person output from the light input means. The facial features can be extracted from the captured image by two or more image processes depending on the brightness and one of the two or more image processes by the light / dark signal output by the light input unit. An image processing apparatus provided with a feature extraction means for selecting image processing or selecting one extraction result from the extraction results of the two or more image processing. 3. An optical input means for inputting an image of a predetermined area including a face of a detection target person, an optical axis for illuminating the detection target person and connecting the detection target person and the light input means, and illumination light. Illumination means and illumination means arranged so as to be substantially coaxial at least in the vicinity of the detection target person, a switch for outputting a signal corresponding to the light and dark by switching the detection target person or other person corresponding to the light and dark, And the feature on the face can be extracted from the photographed image of the person to be detected output from the light input unit by two or more image processes depending on the brightness, and the brightness / darkness signal output from the switch can be used to An image processing apparatus comprising a feature extraction means for selecting one image processing from one or more image processings or selecting one extraction result from extraction results by the two or more image processings. 4. An optical input means for inputting an image of a predetermined area including a face of a detection target person, an optical axis for illuminating the detection target person, and connecting the detection target person and the light input means, and illumination light. Illumination means arranged so that the illumination direction is substantially coaxial at least in the vicinity of the detection target person, and the brightness for determining the brightness of the captured image using the image signal of the detection target person output from the light input means. The feature on the face can be extracted from the photographed image of the person to be detected outputted from the judging means and the light inputting means by two or more image processes according to the lightness and the lightness judging means outputs. Image processing including feature extraction means for selecting one image processing from the two or more image processings or selecting one extraction result from the two or more image processings according to the brightness signal. apparatus . 5. The feature extracting means according to any one of claims 1 to 4, when a dark signal is input, a pupil is extracted by a retina reflection image, and when a bright signal is input, a nostril is extracted. Image processing device. 6. An optical input unit for inputting an image of a predetermined area including a face of a detection target person, an optical axis for illuminating the detection target person, and connecting the detection target person and the light input unit, and illumination light. Illuminating means is arranged so that the illuminating direction is substantially coaxial at least in the vicinity of the detection target person, and the feature on the face from the photographed image of the detection target person output from the light input means is adjusted according to the lightness and darkness. An image processing apparatus that is capable of being extracted by two or more image processes and that includes a feature extraction unit that selects a final extraction result according to the extraction result of the image process. 7. The image processing apparatus according to claim 6, wherein when the pupil cannot be detected by the image processing for dark, the extraction result by the image processing for light is selected. 8. The feature extracting means according to claim 6, when the size of the pupil extracted by the image processing depending on whether light or dark is small, selects the extraction result by the image processing for light. Image processing device. 9. The image processing apparatus according to any one of claims 1 to 8, further comprising illumination photography control means for controlling at least one of a photography condition of the light input means and an illumination condition of the illumination means. An image processing apparatus, wherein the control means extracts light / dark information from any of the light / dark detecting means, the light inputting means, the switch, the lightness / darkness determining means, or the feature extracting means to change the photographing condition or the illumination condition. 10. The illumination photographing control means according to claim 9,
When the light and dark information is bright, the illumination light is turned off, the feature extraction means extracts the feature on the face from the photographed image by the image processing for light, and when the light and dark information is dark, the illumination light is turned on and the feature extraction is performed. The means is an image processing apparatus adapted to extract facial features from a photographed image by dark image processing. 11. A person state judging device comprising: the image processing apparatus according to claim 1; and a person state judging means for judging a person state from an output of the image processing apparatus.