JP3364816B2 - Image processing device - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus for extracting features on a person's face, and in particular, illuminating a person to be detected with a reflected image from an eyeball as a feature point on the face, The present invention relates to an image processing device for detecting the state of.

[0002]

2. Description of the Related Art FIG. 56 shows, for example, JP-A-60-1583.
It is a block diagram of the eye position detection apparatus using the conventional driver imaging device shown in Japanese Patent Laid-Open No. 03-2003. In the figure, 1 is a driver, 10a and 10b are CCD cameras, and 13 is an infrared LE.
D and 14 are LED drive circuits that supply a current to the infrared LED 13, and 100 is a driver behavior detection circuit.

Next, the operation will be described. Infrared LED
13 illuminates the driver 1 in the driver's seat. The image of the driver 1 is input to the CCD cameras 10a and 10b installed at a position where a predetermined area including the face of the driver 1 can be photographed. The input image is input to the driver behavior detection circuit 100,
The positions of eyes and the orientation of the face have been extracted by image processing such as edge detection and pattern recognition.

On the other hand, as shown in the paper "Trial manufacture of eye-gaze detection device which allows pupil extraction processing and head movement" (Journal of the Institute of Electronics, Information and Communication Engineers D-II Vol.J76-D-II No.3) In addition, when the face is illuminated by the coaxial epi-illumination device, the retina reflection image can be significantly captured, and the eye position can be detected by very simple image processing such as binarization processing.
It is the illumination that has the same structure as the illumination direction of the illumination light and the optical axis of the camera). FIG. 57 shows a configuration of a drowsiness detecting device using this device.
No. 2502 specification. In the figure, 1 is a detection target person, 10 is a CCD camera installed at a position where a predetermined area including the face of the detection target person 1 can be photographed, 11 is a visible light cut filter that cuts visible light components, and 12 is a CCD camera. A half mirror installed at an angle of 45 degrees, 13 has an emission wavelength of 860 nm, and emits light with a directivity angle of ± 20 degrees, and an infrared LED that reflects light to the half mirror 12 and illuminates the detection target person 1, and 14 is an infrared ray LE
An LED drive circuit for driving D, 30 is an image memory for temporarily storing the image signal output from the CCD camera, 31 is a timing generation circuit for giving a timing signal to the CCD camera 10, the image memory 30, etc., and 40 is an image memory 30.
Binarization processing circuit A for performing binarization processing of the contents of the above in accordance with a predetermined threshold value th1, 41 is a binarization memory for storing the contents of the binarization processing, 50 is the contents of the binarization memory 41 A feature calculation circuit for extracting the above features, 70 is a drowsiness detection circuit for determining the awakening level from the output of the feature calculation circuit 50, and 71 is an alarm circuit for outputting an alarm.

Next, the operation will be described. In FIG. 57, the half mirror 12 reflects half of the illumination light of the infrared LED 13 irradiated on the optical path l1 and irradiates the face of the detection target person 1 on the optical path l2. The image of the person to be detected 1 passes through the half mirror 12 in the optical path 13 and half of the light is CCD.
After reaching the camera 10, the CCD camera 10 detects the person 1 to be detected.
Capture the image of. At this time, the optical axes of the optical paths l2 and l3 are coaxial with each other when viewed from the detection target (coaxial epi-illumination). At this time, the captured image of the person to be detected is as shown in FIG. In FIG. 58, 3 is a nostril, 4 is an iris, 5 is a sclera,
6 is a pupil and 7 is a face surface. With the coaxial epi-illumination having the above-mentioned configuration, the pupil 6 is caused by the light reflected by the retina,
The pupils are observed as if they are shining, and the brightness is significantly higher than that on the surface of the face or other feature points. This is because the retina has the property of returning reflected light in the same direction as incident light. FIG. 58 also shows the luminance distribution on the line AA of the captured image. In the luminance distribution shown in the lower part of FIG. 58, it is clear that the position of the pupil has a high luminance unlike the other portions. The face image captured by the CCD camera 10 is once input to the image memory 30.

FIG. 59 shows a processing flow of the doze detection by this apparatus. Since the binarization processing circuit A40 inputs the face image in ST10 and extracts only the pupil in ST11,
Binarization processing is performed with an appropriate brightness threshold th1. The obtained binarized image is as shown in FIG. In this binarized image, only the pupil has two circles as a white region, and the other portions have a black region. Therefore, the apparent size of the pupil can be calculated only by counting the number of pixels in the white region in ST12.
In ST13, when the number of pixels in the white area is equal to or more than a predetermined number, it is determined that the eyes are open, and when the number of pixels is less than the predetermined number, the eyes are closed. This eye-opening / eye-closing information is input to the drowsiness detection circuit 70, and the drowsiness detection circuit 70 determines in ST14 whether or not the eye-closing information is continuously input for 3 seconds or more, and if the eye-closing state is 3 seconds or more, it falls asleep. When it is determined that the state is the state, an alarm output command is output to the alarm circuit 71 in ST15. When the eye closed state is less than 3 seconds, the process proceeds to ST10.

[0007]

Since the conventional driver photographing apparatus is constructed as described above, when the person to be detected is the naked eye and the metallic accessory is not worn, the retina reflection image is taken. Although it can be detected, there is a problem that when a reflection image is generated by the illumination light, for example, when glasses or earrings are attached, it is impossible to distinguish the reflection image from the retina reflection image.

FIG. 61 shows an explanatory view for explaining the influence of the reflected image from the glasses. FIG. 61 is a diagram in which silver-rimmed eyeglasses are mounted, but the lens reflection image 8 in which the illumination light is reflected by the eyeglass lens,
An eyeglass frame reflection image 9 in which the illumination light is reflected by the eyeglass frame is observed. When this is binarized by the binarization processing circuit A of the conventional apparatus, five white pixel areas are formed as shown in FIG. 62, and the lens reflection image 8 and the spectacle frame reflection image 9 are white even when the eyes are closed. It remains as a pixel, and closed eyes cannot be detected. As described above, the above-described conventional device has a problem that it is not possible to extract feature points on the face when wearing glasses or wearing a metallic accessory.

The present invention has been made in order to solve the above problems, and provides an apparatus capable of extracting characteristic points on the face even when wearing glasses or wearing a metallic accessory. The purpose is to get.

[0010]

An image processing apparatus according to a first aspect of the present invention is an illumination light control means for controlling the presence / absence of illumination light, a bright photographed image output means for outputting an image of a person to be detected with the illumination light, The dark point image output means for outputting the image of the detection target person without illumination light, and the difference image output means for outputting the difference image between the bright and dark image, the feature point extraction means, It has a first threshold value for extracting a reflection image from an eyeball and a second threshold value for extracting a reflection image from a wearing object worn by a detection target person, and the difference image is a first threshold value. The first binarized image processed by
A reflected image from an eyeball is extracted as a feature point on the face by using a clear imaged image and a second binarized image obtained by processing with a second threshold value.

An image processing apparatus according to a second aspect of the present invention is a second binarization processing image in which a difference image is processed by a first threshold value and a bright image is processed by a second threshold value.
By using the binarized image, the reflection image from the eyeball is extracted as a feature point on the face.

An image processing apparatus according to a third aspect of the present invention provides an illumination light control means for controlling the intensity of illumination light or an input gain of an optical input means, an image of a person to be detected having a high illumination light or a large input gain. A bright-photographed image output means for outputting, a dark-photographed image output means for outputting an image of a person to be detected with weak illumination light or a small input gain, and a differential image output means for outputting a differential image between the bright-photographed image and the dark-photographed image. The feature point extracting means includes a first threshold value for extracting a reflection image from the eyeball of the detection target person and a second threshold value for extracting a reflection image from the wearing object worn by the detection target person. And a first binarized image obtained by processing the difference image with a first threshold , and
A reflected image from an eyeball is extracted as a feature point on the face by using a second binarized image obtained by processing a darkness imaged image with a second threshold value.

Further, in the image processing apparatus according to the fourth aspect, the feature point extracting means is the dark photographed image photographed i-th, the bright photographed image photographed i−1 th, and the image photographed i−2 th. And a first difference image obtained by performing a difference process on the i-th imaged bright-photographed image from the i-th imaged dark-photographed image, and the i-1st image. The second difference image obtained by performing the difference processing on the i−2nd dark photograph image obtained from the bright photograph image photographed in the above is obtained, and the first difference image and the second difference image are used to detect from the eyeball. The reflected image of is extracted as a feature point on the face.

Further, in the image processing apparatus according to the fifth aspect, the feature point extracting means is the dark photographed image photographed i-th, the bright photographed image photographed i−1 th, and the image photographed i−2 th. And the dark-shot image obtained by performing the difference processing between the i-th shot bright-shot image and the i-th shot dark-shot image described above. The obtained difference image is obtained, and the reflected image from the eyeball is extracted as a feature point on the face by the difference image.

According to a sixth aspect of the present invention, in the image processing apparatus, a coaxial illumination means for illuminating a predetermined area including the face of the person to be detected coaxially with the light input means, and a predetermined area including the face of the person to be detected are optically input. Non-coaxial lighting means for illuminating the means non-coaxially,
Illumination light control means for controlling the presence or absence of light emission of the coaxial illumination means and the non-coaxial illumination means, a coaxial image output means for outputting an image of the detection target person by the coaxial illumination light, and an image of the detection target person by the non-coaxial illumination light. Non-coaxial image output means, differential image output means for outputting a differential image between the coaxial image and the non-coaxial image, the feature point extraction means, and a first threshold value for extracting a reflection image from the eyeball of the detection target person. , A second threshold value for extracting a reflection image from the wearing object worn by the detection target person, and a first second value obtained by processing the difference image with the first threshold value.
By using a binarized image and a second binarized image obtained by processing the non-coaxial image with a second threshold value, a reflection image from an eyeball is extracted as a feature point on the face.

Further, in the image processing device according to the seventh aspect, the feature point extracting means takes the i-th captured non-coaxial image, the i-1th captured coaxial image, and the i-2nd captured image. Using the non-coaxial image, a first difference image obtained by performing a difference process on the i-th imaged non-coaxial image from the i-1th imaged coaxial image, and the i-1 th imaged image. A second difference image obtained by performing a difference process on the i-2nd non-coaxial image captured from the coaxial image
With the first difference image and the second difference image, the reflection image from the eyeball is extracted as a feature point on the face.

Further, in the image processing apparatus according to the eighth aspect, the feature point extraction means takes the i-th captured non-coaxial image, the i-1th captured coaxial image, and the i-2nd captured image. The non-coaxial image was used to obtain the i-th imaged coaxial image from the i-th imaged coaxial image by subtracting the i-th imaged non-coaxial image from the i-th imaged coaxial image. The difference image is obtained, and the reflection image from the eyeball is extracted as a feature point on the face by the difference image.

Further, in the image processing apparatus according to the ninth aspect, the feature point extraction means is a first binarized image obtained by processing the difference image with a first threshold value, and a dark image or a non-coaxial image. Using a second binarized image obtained by processing with the second threshold value, the reflection image from the eyeball is extracted as a feature point on the face.

Further, in the image processing device according to the tenth aspect, the feature point extracting means processes the first difference image obtained by processing the first difference image with the first threshold value and the second difference image into the second difference image. The reflected image from the eyeball is extracted as a feature point on the face by using the second difference processed image processed by the threshold value of 1.

The image processing apparatus according to claim 11 is
Illumination light control means for controlling the intensity of the illumination light or the input gain of the light input means, bright photographic image output means for outputting the image of the person to be detected with strong illumination light or large input gain, weak illumination light or input gain Is provided with dark photography image output means for outputting the image of the detection target person, and the feature point extraction means is a first threshold value for extracting a reflection image from the eyeball of the detection target person, and the detection target person wears it. And a second threshold value for extracting a reflection image from the mounted object, the first binarized image obtained by processing the bright photographed image by the first threshold value, and the dark photographed image as the second threshold value. By using the second binarized image processed by the above method, the reflection image from the eyeball is extracted as a feature point on the face.

Further, in the image processing apparatus according to the twelfth aspect, the feature point extracting means is a binarization-processed image of the i-th captured dark-photographed image and an i-1-th photographed bright-photographed image. By using the binarized image and the binarized image of the i-2nd dark shot image, the i-1-th binarized image of the bright shot image is shot at the i-th shot. The first mask image obtained by performing the masking process on the binarized image of the dark photographed image and the binarized image of the bright photographed image which is photographed at the i-1th are photographed at the i-2nd. The second mask image obtained by performing the masking process on the binarized image of the dark photographed image obtained is obtained, and the reflection image from the eyeball is characterized by the facial features by the first mask image and the second mask image. It is extracted as a point.

Further, in the image processing apparatus according to the thirteenth aspect, the feature point extraction means is a binarized image of the i-th captured dark-photographed image and an i-1-th photographed bright-photographed image. Using the binarized image and the binarized image of the i-2nd dark shot image, the binarized image of the i-1th bright shot image is set to the i-th bin A mask image obtained by performing a mask process is obtained from the binarized image of the photographed dark photographed image and the binarized image of the i-2nd photographed dark photographed image, and by the mask image, The reflected image from the eyeball is extracted as a feature point on the face.

An image processing apparatus according to claim 14 is
Coaxial illumination means for illuminating a predetermined area including the face of the detection target person coaxially with the light input means, non-coaxial illumination means for illuminating a predetermined area including the face of the detection target person non-coaxially with the light input means, coaxial illumination means Illumination light control means for controlling the presence or absence of light emission of the non-coaxial illumination means, coaxial image output means for outputting the image of the detection target person by the coaxial illumination light, and non-coaxial image output for outputting the image of the detection target person by the non-coaxial illumination light. The feature point extraction means includes a first threshold value for extracting a reflection image from the eyeball of the detection target person, and a second threshold value for extracting a reflection image from the wearing object worn by the detection target person. A first binarized image having a threshold value and a coaxial image processed with a first threshold value, and a second binarized image having a non-coaxial image processed with a second threshold value, Extract the reflection image from the eyeball as a feature point on the face Than it is.

Further, in the image processing apparatus according to the fifteenth aspect, the feature point extracting means is a binarized image of the i-th captured non-coaxial image and a binary image of the (i-1) -th captured coaxial image. Using the binarized image and the binarized image of the i−2nd captured non-coaxial image, the binarized image of the i−1th captured coaxial image is recorded in the i-th non-coaxial image. The first mask image obtained by masking with the binarized image of the coaxial image, and the i-2nd non-coaxial image of the binarized image of the coaxial image captured above A second mask image obtained by performing mask processing on the image binarized image is obtained, and the reflection image from the eyeball is extracted as a feature point on the face by the first mask image and the second mask image. To do.

Further, in the image processing apparatus according to the sixteenth aspect, the feature point extraction means is a binarized image of the i-th captured non-coaxial image and a binary image of the (i-1) -th captured coaxial image. Using the binarized image and the binarized image of the non-coaxial image captured i-2, the binarized image of the coaxial image captured i-1 is captured i above. A mask image obtained by performing mask processing on the binarized image of the non-coaxial image and the binarized image of the i-2nd non-coaxial image captured is obtained, and a reflection image from the eyeball is obtained by the mask image. Is extracted as a feature point on the face.

The image processing apparatus according to claim 17 is
The image is subjected to degeneracy processing for the image subjected to the difference processing or the image subjected to the mask processing.

The image processing apparatus according to claim 18 is
This is for enlarging the image to be subjected to difference processing or the image to be masked.

The image processing apparatus according to claim 19 is
The enlargement or contraction process is given directionality in accordance with the positional relationship between the coaxial illumination and the non-coaxial illumination.

An image processing apparatus according to claim 20 is
Has a mounting object determining means determines the presence or absence of attachment of the detection subject, the feature point extracting means, depending on the determination result of the mounting object determining means, first binarized processed image and the second binary Conversion
Image processing for mounting using processed images and first binarization processing
The image processing is switched between non-wearing image processing using images , or the processing result of each image processing is selected.

The image processing apparatus according to claim 21 is
The mounted object determining means detects the presence or absence of the mounted object based on the number of reflection images or the maximum brightness of the illumination light.

Further, in the image processing apparatus according to claim 22 ,
In addition, the feature point extraction means can illuminate when there is an attached object.
It strengthens the light .

In the image processing apparatus according to claim 23,
In addition, the feature point extraction means can
The input gain of the force means is increased.

Further, in the image processing apparatus according to the twenty-fourth aspect, the feature point extraction means includes the first binarized image and the image.
And image processing for mounting using the second binarized image and
Image processing for non-wearing using the binarized image No. 1 and selecting either one of the image processing results depending on the image processing result.

[0034]

An image processing apparatus according to claim 1 controls the presence or absence of illumination light, and a bright photographed image of a person to be detected with illumination light,
A first binarization-processed image obtained by processing the difference image with the dark-captured image of the detection target person without illumination light by the first threshold ;
The captured image is processed with the second threshold value, and the reflected image from the eyeball is extracted using the binarized image.

An image processing apparatus according to a second aspect of the present invention comprises a first binarized image obtained by processing a difference image with a first threshold value,
A reflection image from the eyeball is extracted by using the second binarized image obtained by processing the bright photographed image with the second threshold value.

An image processing apparatus according to a third aspect of the present invention controls the intensity of illumination light or the input gain of a light input means, and a bright photographed image of a person to be detected having a high illumination light or a large input gain.
The first 2 in which the difference image with the dark image of the detection target person whose illumination light is weak or whose input gain is small is processed by the first threshold value.
The binarized image and the dark image are processed by the second threshold.
The reflected image from the eyeball is extracted by using the second binarized image.

According to a fourth aspect of the present invention, there is provided an image processing apparatus, wherein a first differential image obtained by performing a differential process on a bright photographed image photographed at the (i-1) th to a dark photographed image photographed at the ith, and an (i-1) th image. The second difference image obtained by performing the difference processing on the i-2nd dark photograph image taken from the second bright image taken in the above is obtained, and the first difference image and the second difference image Extract the reflection image.

An image processing apparatus according to a fifth aspect of the present invention performs a difference process between the i-1th bright photographed image and the i-th dark photographed image and the i-2nd dark photographed image. The obtained difference image is obtained, and the reflection image from the eyeball is extracted from the difference image.

According to a sixth aspect of the image processing apparatus, the difference image between the coaxial image and the non-coaxial image is processed by the first threshold value.
The binarized image of No. 1 and the non-coaxial image as the second threshold
A reflection image from the eyeball is extracted using the second binarized image that has been further processed.

According to a seventh aspect of the present invention, there is provided an image processing apparatus, wherein a first differential image obtained by performing a differential process on a i-th captured coaxial image to an i-th captured non-coaxial image, and an i-1th captured image. A second difference image obtained by performing difference processing on the i-2nd non-coaxial image taken from the taken coaxial image is obtained, and the reflection image from the eyeball is obtained by the first difference image and the second difference image. To extract.

[0041] The image processing apparatus according to claim 8, coaxial images taken in (i-1) -th, obtained by differential processing and non-coaxial image taken in a non-coaxial image and i-2-th shot in i-th The difference image is obtained, and the reflection image from the eyeball is extracted from the difference image.

An image processing apparatus according to a ninth aspect of the present invention includes a first binarized image obtained by processing the difference image with a first threshold value,
The reflection image from the eyeball is extracted using the dark imaged image or the second binarized image obtained by processing the non-coaxial image with the second threshold value.

An image processing apparatus according to a tenth aspect of the present invention is a first difference processed image obtained by processing a first difference image with a first threshold value and a second difference processed image obtained by processing a second difference image with a first threshold value.
The reflected image from the eyeball is extracted by using the difference processed image of.

An image processing apparatus according to the eleventh aspect controls the intensity of illumination light or the input gain of the light input means to process a bright imaged image by a first threshold value, a first binarized image, and a dark imaged image. A reflection image from the eyeball is extracted using the second binarized image obtained by processing the image with the second threshold value.

The image processing apparatus according to claim 12 is i-1.
The first mask image obtained by masking the binarized image of the brightly photographed image taken at the second time with the binarized image of the dark photographed image taken at the i-th image, and the i-1 th imaged The second obtained by masking the binarized image of the bright photographed image with the binarized image of the dark photographed image i-2
, And the reflection image from the eyeball is extracted from the first mask image and the second mask image.

The image processing apparatus according to the thirteenth aspect is i-1.
The binarized image of the bright photographed image captured next is the binarized image of the i-th captured dark captured image and the binarized image of the i-2nd captured dark captured image. The mask image obtained by the mask processing is obtained, and the reflection image from the eyeball is extracted from the mask image.

According to a fourteenth aspect of the present invention, in an image processing apparatus, a first binarized image obtained by processing a coaxial image with a first threshold value and a second binarized image obtained by processing a non-coaxial image with a second threshold value. The reflected image from the eyeball is extracted using the image.

The image processing apparatus according to claim 15 is i-1.
The first mask image obtained by masking the binarized image of the coaxial image captured second with the binarized image of the non-coaxial image captured ith, and the coaxial image captured i-1 The second mask image obtained by masking the binarized image of the image with the binarized image of the non-coaxial image captured i-2nd is obtained, and the first mask image and the second mask image are obtained. The reflected image from the eyeball is extracted from the mask image.

The image processing apparatus according to the sixteenth aspect is i-1.
The binarized image of the coaxial image captured second is masked with the binarized image of the non-coaxial image captured i-th and the binarized image of the non-coaxial image captured i-2. The obtained mask image is obtained, and the reflection image from the eyeball is extracted from the mask image.

An image processing apparatus according to a seventeenth aspect performs a degeneracy process on an image to be subjected to difference processing or an image to be masked.

An image processing apparatus according to the eighteenth aspect enlarges an image to be subjected to difference processing or an image to be masked.

In the image processing apparatus according to the nineteenth aspect , the enlargement or contraction processing is given a directionality in accordance with the positional relationship between the coaxial illumination and the non-coaxial illumination.

[0053] The image processing apparatus according to claim 20, to determine the presence or absence of attachment of the detection subject, the determination result Therefore, first
Switching between the image processing for wearing using the binarized image and the second binary image, and the image processing for non-wearing using the first binary image , or The processing result of each image processing is selected.

An image processing apparatus according to a twenty-first aspect detects the presence / absence of a wearing object based on the number of reflection images by illumination light or the maximum brightness.

The image processing apparatus according to the twenty-second aspect is characterized by
The extraction means intensifies the illumination light when there is a wearing object.

The image processing apparatus according to claim 23 is characterized by
The extraction means is an input device for the optical input means when there is a wearing object.
Raise the inn.

An image processing apparatus according to a twenty-fourth aspect of the present invention selects either one of the image processing results depending on the image processing result for wearing and the image processing result for non-wearing.

[0058]

EXAMPLES Example 1. An embodiment of the present invention will be described below with reference to the drawings. In FIG. 1, reference numeral 60 denotes a binarization processing circuit B having a first threshold value th1 for extracting a retina reflection image and a second threshold value th2 for extracting a reflection image from eyeglasses. Reference numeral 61 is a binarization memory that stores the contents of the binarization process. Figure 2
FIG. 2A is a photographed image when eyeglasses are worn, and FIG. 2B is a diagram showing the luminance distribution of the photographed image taken along line BB. As shown in FIG. 2B, the lens reflection image 8 and the spectacle frame reflection image 9 have higher luminance than the retina reflection image 6.

Next, the operation will be described. In FIG. 1, the operation until the face image is input to the image memory 30 is the same as that of the conventional example. FIG. 3 shows a processing flow of the doze detection by the doze detection device of the present embodiment. In FIG. 3, in ST20, the binarization processing circuit B60 determines that pixels having a density between the first threshold th1 and the second threshold th2 are white pixels (value 1), and pixels having other densities are black pixels ( The value is converted to 0) and output to the binarization memory 41. At this time, the contents of the binarization memory 41 are shown in FIG. As shown in FIG. 4, the retina reflection image 6 is a circular white region, and the lens reflection image 8 and the spectacle frame reflection image 9 are donut-shaped white regions with holes. The feature calculation circuit 50 reads into the binarization memory 61 and discriminates the retina reflection image from the shape of the second binarized image. That is, in ST21, labeling is performed in order from the upper left white area, and in ST22, the donut-shaped white area is excluded. The donut-shaped white area is determined by, for example, Euler number calculation which is well known in image processing. S
A diagram excluding the donut-shaped white region at T22 is shown in FIG. FIG. 5 is an image in which only the retina reflection image remains as a white region. Hereinafter, after step ST12, the process becomes the same as the conventional example, and the retinal reflection image 6 can be extracted and the dozing can be detected by the same operation as the conventional example.

Example 2. In the first embodiment described above, the one in which the pupil is extracted by using the retina reflection image to detect the drowsiness is shown. In the present embodiment, a method of detecting the line-of-sight direction and looking aside using a corneal reflection image is shown. FIG. 6 is a configuration diagram showing the inattentiveness detection device of the present embodiment, in which 80 is an edge extraction circuit, 81 is an image memory, and 7
2 is an inattentive judgment circuit. In the present embodiment, non-coaxial illumination is used, and the inattentive and line-of-sight directions are obtained from the positional relationship between the iris and the Purkinje image, which is the reflected image of the cornea due to the illumination light. FIG. 7 is a diagram when the face is directed to the front and the eyeball is directed to the left. When the infrared LED 13 is installed slightly above the camera, the Purkinje image 2 is observed slightly above the center of the iris 4 when the eyeball is facing the camera, and the Purkinje image 2 is iris when the eyeball is facing right. 4 to the left of the iris 4, the Purkinje image 2 is observed to the right of the iris 4. The amount of movement of the Purkinje image is about 1 mm per 10 degrees of rotation of the eyeball, and the angle of the eyeball can be estimated from the positional relationship between the Purkinje image 2 and the center of the iris 4.

Next, the operation will be described. By the same operation as in the first embodiment, the face image of the detection target person 1 is displayed in the image memory 3
Input to 0. FIG. 7 is an input image of the person to be detected 1 captured by this embodiment. The Purkinje image 2 is extracted by the feature calculation circuit 50 from the binarized image converted by the binarization processing circuit B60 by the same operation as in the first embodiment. The feature calculation circuit 50 further calculates the position of the Purkinje image 2 by calculating the center of gravity of the white area. next,
A method of deriving the center position of the iris 4 will be described. The edge extraction circuit 80 performs edge extraction processing from the image memory 30 and stores the edge extraction result in the image memory 81. Figure 8
Shows the edge extraction result. The feature calculation circuit 50 uses the well-known Hough transform process for obtaining a circle on the image obtained by the edge extraction, sets the size of the iris near 2 cm, performs the Hough transform process, and obtains the position of the iris. At this time, in order to reduce the amount of calculation, the Hough transform area is limited to, for example, 2 cm square of the Purkinje image 2. Feature calculation circuit 5
0 further obtains the direction of the eyeball, that is, the line-of-sight direction from the positional relationship between the Purkinje image 2 and the iris 4, and the inattentive detection circuit 7
2 outputs an alarm command to the alarm circuit 71, for example, when the direction of the eyeball is at a position of 20 degrees or more from the front for 5 seconds or more, it is determined to be looking aside.

As the facial feature points extracted in this way,
When a reflection image from an eyeball is used, the feature point on the face can be distinguished from the wearing object such as glasses by using the configuration of the present invention, and the feature point can be reliably extracted.

Example 3. FIG. 9 is a configuration diagram showing a doze detection device according to a third embodiment of the present invention. A white area degeneracy processing circuit 62 and a binarization memory 63 for storing a processing result by the processing circuit 62 are added to the apparatus of the first embodiment. Although the retinal reflection image 6 is extracted by excluding the donut-shaped white area in the first embodiment, the retinal reflection image 6 is extracted by degenerating the white area of the binarization memory 61 in the present embodiment.

In the first embodiment, the case where the binarization processing is optimally performed has been shown. However, depending on the direction and material of the eyeglasses, a donut-shaped reflection image as shown in FIG. 4 may not be obtained. As in 10, donuts are chipped (9a) or small white areas (9b) appear. The white area degeneracy processing circuit 61 is composed of, for example, a 5 × 5 minimum filter,
A white area degeneration process is performed. The result is shown in FIG. By doing so, the area of the retina reflection image 6 becomes smaller, but the white areas of the lens reflection image 8 and the spectacle frame reflection image 9 can be excluded.

In the third embodiment, the white area is degenerated, but the opening processing may be performed after the degeneration processing, for example, by using a 5 × 5 max filter to perform the enlargement processing. At this time, only the retina reflection image 6 is restored to the original size shown in FIG.

Further, in the third embodiment, the degeneration processing circuit and the enlargement processing circuit are realized by a 5 × 5 minimum filter and a 5 × 5 enlargement processing circuit, but 1 × 5, 5 × 1 may be used instead of 5 × 5. It may be realized by a one-line minimum filter or a magnifying filter. If one line is used, the effect of removing the reflection image of the glasses is reduced, but the circuit becomes simple. Furthermore, the degeneracy process is not limited to the minimum filter, and may be a filter that takes the second value from the minimum value.

Example 4. FIG. 12 is a configuration diagram showing a doze detection device according to a fourth embodiment of the present invention. In FIG. 12, reference numeral 32 is a degeneration processing circuit that performs degeneration processing of a grayscale image (256-tone image) that is a captured image, 33 is a magnification processing circuit that magnifies the grayscale image, and 40a is a threshold value th1 for the retina reflection image (first Threshold of 1)
, A second binarization processing circuit for performing binarization processing with 40b, a second binarization processing circuit for performing binarization processing with a threshold value th2 (second threshold value) for eyeglass reflection, and 41a for the first binarization processing circuit. First binarization memory for storing the first binarization image processed by the binarization processing circuit 40a, 41b is the second binarization processing circuit 41.
the second binary storing the second binarized image processed in b.
It is a digitized memory. 65 designates the first binarized image as the second
41C is a masking circuit for masking with the binarized image, and 41C is a third binarizing memory for storing the binarized image (third binarized image) obtained by the masking process. The mask circuit 65 operates so that the pixels corresponding to the white area of the second binarized image in the white area of the first binarized image are made black.

Next, the operation will be described. In FIG. 12, a face image is input to the image memory 30, the grayscale image is subjected to degeneracy processing by the degeneracy processing circuit 32, and binarized by the first threshold value th1 in the first binarization processing circuit 40a. The binarized memory 41a stores the binarized image. On the other hand, the gray-scale image is enlarged by the enlargement processing circuit 33, and then the second binarization processing circuit 40b performs the enlargement processing by the second threshold value th2.
The binarization processing is performed, and the binarized image is stored in the second binarization memory 41b. FIG. 13 shows the first binarized image, and FIG.
Is a second binarized image. In FIG. 13, the retina reflection image 6, the lens reflection image 8, and the spectacle frame reflection image 9 are smaller than those in FIG. In FIG. 14, it can be seen that the lens reflection image 8 and the spectacle frame reflection image 9 are larger than those in FIG. The mask circuit 65 operates so that the pixels corresponding to the white areas of the second binarized image in the white areas of the first binarized image are made black. That is, the first binarized image is P1 (X, Y), and the second binarized image is P2.
(X, Y), third obtained by masking
If the binarized image of is Q (X, Y), P1 (X, Y) = 1, AND P2 (X, Y) = 0, Q (x, y) = 1 Others, Q (X, Y) = 0. This image is similar to FIG. The feature calculation circuit 50 calculates the apparent size of the pupil from the binarized image in the same manner as in the conventional case, and detects drowsiness.

In the fourth embodiment, the mask processing is realized by the method using the above-mentioned mathematical expression, but the mask processing is not limited to this method. For example, the second binarized image is inverted and then inverted. It may be realized by taking an AND image of the second binarized image and the first binarized image.

Although the degeneration processing circuit 32 and the enlargement processing circuit 33 are used in this embodiment, either one or both may be omitted. If both are omitted, the third binarized image obtained by the masking process becomes an image including a donut-shaped white area similar to that in FIG. In the same manner as 2
Donut-shaped white areas are excluded from the shape of the binarized image. Alternatively, in the same manner as in the third embodiment, the binarized image is degenerated to exclude the donut-shaped white area.

Example 5. FIG. 15 is a block diagram showing an image processing apparatus according to the fifth embodiment of the present invention. Although the mask circuit 65 is used in the fourth embodiment, the feature calculation circuit 50 does not include the mask circuit 65 and the third binarization memory 41c. N1, N for the binarized image
2, N3, for the second binarized image, M1, M2,
Label with M3, find its position, and
The same image processing can be performed by removing the white area near the white area of the second binarized image from the white area of the binarized image. That is, except for M1 closest to N1, M2 closest to N2, and M4 closest to N3, it is determined that the remaining M3 and M5 are pupils.

[0072]

[Table 1]

In the fifth embodiment, the degeneration processing circuit 32
Although the enlargement processing circuit 33 is used, either one or both may be omitted.

Example 6. 16 is a block diagram showing a doze detection device according to a sixth embodiment of the present invention. In the fifth embodiment, the contents of the image memory 30 are expanded or shrunk, and then binarized by two thresholds. However, the binarization is performed first, and then the white region is expanded or shrunk. , Even if you put a mask, the same effect. By doing so, the hardware also requires a smaller scale for the enlargement / reduction process of the binarized image than for the enlargement / reduction process of the grayscale image. In FIG. 16, reference numeral 66 denotes a binarization processing circuit C having a first threshold value th1 for extracting a retina reflection image and a second threshold value th2 for extracting a reflection image from eyeglasses. The result of the binarization processing by the first threshold value th1 for extraction is stored in the binarization memory 41a as the second threshold value t for extracting the eyeglass reflection image.
The result of the binarization processing by h2 is output to the binarization memory 41b. The image of the second binarization memory 41b (the second binary
The binarized image is output to the binarization memory 68 after being enlarged by the white area enlargement processing circuit 67. The mask circuit 65 converts the white area of the first binarized image stored in the first binarized memory 41a into the enlarged second binarized image stored in the binarized memory 68. Mask processing is performed in the white area. As a result, the white area of only the retina reflection image 6 remains in the third binarized image stored in the third binarization memory 41c.

In the sixth embodiment, if the white area degeneracy processing is performed on the third binarized image, the effect of removing the eyeglass reflection image is further obtained.

Example 7. 17 is a configuration diagram showing a doze detection device according to a seventh embodiment of the present invention. In the sixth embodiment, the white area enlargement processing circuit 6
7 is executed for the image of the second binarization memory 41b, the white area degeneracy circuit 62 is used for the first binarization memory 41a.
It is even more effective when executed on the image.

In the seventh embodiment, the configuration in which the white area enlargement processing circuit 67 and the binarization memory 68 are omitted is also the sixth embodiment.
Has the same effect as.

Example 8. In the above embodiment, the feature point on the face is extracted by distinguishing it from the wearing object while the illumination light (LED) remains in the ON state, but in the present embodiment, the illumination light is switched on / off to extract the feature point. The one that is accurately extracted by distinguishing from the attached object is shown. In a bright environment such as a vehicle interior, as shown in FIG. 18A, the entire face is photographed brightly, and the retinal reflection image does not appear prominently only with the LED-on image due to the influence of ambient light. On the other hand, as shown in FIG. 18B, the retina reflection image does not appear in the LED-off image, and the pupil is photographed darkly. Therefore, the image obtained by subtracting the LED off bright image from the LED on dark image is LE as shown in FIG.
The retinal reflection image appears more prominently than in the D-on bright image. This embodiment is an embodiment of an image processing apparatus that performs difference processing to distinguish between a spectacle reflection image and a retinal reflection image for use in a bright environment.

FIG. 19 is a block diagram showing a doze detecting device according to the eighth embodiment of the present invention. In the figure, 30a is an image memory, 30b is an image memory that stores the contents of the previous frame of the image memory 30a, 34 is a difference circuit, 30c is an image memory, and the LED drive circuit 14 is an infrared LED 13 that is based on a signal from the timing generation circuit 31. The light is switched on and off for each frame cycle. FIG. 20 shows the LED light emission timing chart and the image memories 30a, 30b, and 3.
0c, and the image memory 30a sequentially stores the OFF image bi, the ON image ai + 1, the OFF image bi + 2, the ON image ai + 3, ... Corresponding to ON / OFF of the LED. The image memory 30b stores the off image bi, the on image ai + 1, the off image bi + 2, the on image ai + 3, ... With a delay of one frame from the image memory 30a. Image memory 30c
Is one frame behind the image memory 30b, and the difference image a
i + 1-bi, ai + 3-bi + 2, ... Are stored.

Next, the operation will be described. The LED drive circuit 14 turns on / off the infrared LED 13 every frame cycle.
The light is switched off to emit light, and the bright and dark images are alternately stored in the image memory 30a and the image memory 30b according to the presence or absence of illumination light. The difference circuit 34 obtains the difference image between the bright and dark photographed images, and the image memory 30c
To store. The image stored in the image memory 30c is an image in which a retina reflection image is prominently displayed. The first binarization processing circuit 40a inputs this difference image from the image memory 30c and performs binarization processing with the first threshold th1. Obtained 2
The binarized image includes a retinal reflection image and a spectacle reflection image as shown in FIG. On the other hand, the second binarization processing circuit 40b
Inputs a bright photographed image from the image memory 30a and performs binarization processing with the second threshold value th2. This binarized image includes a spectacle reflection image as shown in FIG. The binarized image stored in the binarized memories 41a and 41b is masked by the masking circuit and becomes an image including a donut-shaped white area shown in FIG. The subsequent operation is similar to that of the first embodiment.

By turning on and off the illumination light in this way and taking the difference between the obtained bright and dark photographed images, the ambient light component can be removed and the retina reflection image is emphasized.
When the binarization processing is performed with, the retinal reflection image can be reliably extracted.

Example 9. 22 is a configuration diagram showing a doze detection device according to a ninth embodiment of the present invention. In this embodiment, the white area degeneration circuit 6 is added to the eighth embodiment.
2 and the image memory 63 are added. In the present embodiment, the image including the donut-shaped white area stored in the binarization memory 41c is subjected to the degeneration process in the same manner as in the third embodiment to exclude the donut-shaped white area.

Example 10. 23 is a configuration diagram showing a doze detection device according to a tenth embodiment of the present invention. In the present embodiment, a white area enlargement processing circuit 67 and a binarization memory 68 are added after the second binarization processing circuit 40b of the eighth embodiment, and since the eyeglass reflection image is enlarged, the mask circuit In the binarized image subjected to the mask process in 65, the donut-shaped white area as seen in Example 8 does not appear, and only the retina reflection image is obtained. Therefore, the snooze can be detected in the same manner as in the fourth embodiment.

In the above embodiment, the white area expansion processing circuit is inserted after the second binarization processing circuit 40b, but the white area degeneration processing circuit is provided after the first binarization processing circuit 40a. May be included. Alternatively, both the white area expansion processing circuit and the white area degeneration processing circuit may be included.

The enlargement processing may be performed before the binarization processing is performed by the second binarization processing circuit 40b, or the difference image may be displayed before the binarization processing is performed by the first binarization processing circuit 40a. Degeneration processing may be performed.

Example 11. 24 is a configuration diagram showing a doze detection device according to an eleventh embodiment of the present invention. In the present embodiment, the binarization processing circuit B60 binarizes the difference image in the eighth embodiment shown in FIG. 21A by the binarization processing circuit B60 as in the first embodiment. The binarization processing circuit B60 directly obtains a binarization processing image similar to that in FIG. 21C without using the circuit 65.

Example 12 25 is a configuration diagram showing a doze detection device according to a twelfth embodiment of the present invention. In this embodiment, the illumination light is switched between strong and weak to extract feature points with high accuracy. In FIG. 25, 30a is an image memory and 30b is an image memory 30a.
The image memory for storing the contents of the preceding frame, 34 is the difference circuit, 30c is the image memory, and the LED drive circuit 14 causes the infrared LED 13 to emit light by switching the strength / weakness for each frame cycle by the signal of the timing generation circuit 31. L
ED emission timing chart and image memories 30a, 30
The content of b is the same as in Example 8, and stores a strong light emission image (hereinafter, referred to as a bright image) and a weak light emission image (hereinafter, referred to as a dark image) in accordance with the intensity of the LED. The image memory 30c stores the difference image.

Next, the difference image in this embodiment will be described. FIG. 26 shows the input / output characteristics of the CCD camera 10. The retinal reflection image input to the camera 10 and the brightness of the reflected light from the wearing object such as spectacles are in the order shown on the horizontal axis of FIG. 26 depending on the intensity of the illuminating light. Is quite bright. Therefore, the output (density value) from the camera becomes a value close to overflow. That is, the density value of the reflected image of the eyeglasses does not change much regardless of whether the light emission is weak or strong. Therefore, when the dark photographed image is subtracted from the bright photographed image, the density difference of the reflection image of the eyeglasses is small, and an image in which the reflection image of the eyeglasses is rather dark and only the retina reflection image is bright appears in the image memory 30c. The image in the image memory 30c is shown in FIG. In this image, the centers of the lens reflection image 8 and the spectacle frame reflection image 9 are dark, but a bright donut area appears around them.

In FIG. 25, the binarization processing circuit A40
Then, the difference image stored in the image memory 30c is binarized by the threshold value th1 as shown in FIG. The subsequent operation is similar to that of the first embodiment.

In the twelfth embodiment, a white area degeneracy processing circuit may be provided after the binarizing memory 41 or the binarizing memory 41c as in the third embodiment.

Further, the binarization processing circuit A40 may be the binarization processing circuit B60, and a similar binarization processing image is obtained.

As described above, the intensity of the illumination light is emitted to obtain the difference between the obtained bright and dark photographed images, so that the operation of eliminating the influence of the reflection image of the mounted object is eliminated, so that the feature point detection accuracy is further improved.

Example 13 28 is a configuration diagram showing a doze detection device according to a thirteenth embodiment of the present invention. In the present embodiment, an enlargement processing circuit 33 for enlarging a dark shot image and an image memory 30d are inserted after the image memory 30b in the apparatus of the twelfth embodiment. As a result, the reflected image from the eyeglasses at the time of weak light emission is magnified.
In c, an image in which the spectacle reflection is dark is obtained as shown in FIG. If this image is binarized by the binarization processing circuit A40, only the retina reflection image can be extracted.

Example 14. 30 is a configuration diagram showing a doze detection device according to a fourteenth embodiment of the present invention. In the present embodiment, a degeneration processing circuit 33 for degrading a bright photographed image and an image memory 30e are inserted after the image memory 30a in the apparatus of the thirteenth embodiment. This makes it possible to remove the spectacle reflection even if the face moves a little.

Example 15. In the twelfth, thirteenth, and fourteenth embodiments, the feature points of the eyeball are extracted using only the difference image. However, like the eighth, ninth, and tenth embodiments, two binarized images are obtained and the masking process is performed. You may That is, as shown in FIG.
A binarization processing circuit 40b is provided, a weak light emission image is input from the image memory 30a, binarization processing is performed using the second threshold value th2, and a second reflection image, which is a reflection image of glasses, is obtained. The binarization processing circuit 40a converts the difference image into the first threshold th.
Even if the image similar to that in FIG. 4 is obtained by performing the binarization processing at 1 to obtain the retinal reflection image which is the first binarization processing image and masking both the binarization processing images by the mask circuit 65. Good.

Further, with respect to the present embodiment, either one or both of the first binarization-processed image degeneracy process and the second binarization-processed image enlargement process may be performed.

The enlargement processing may be performed before the binarization processing is performed by the second binarization processing circuit 40b, or the difference image may be displayed before the binarization processing is performed by the first binarization processing circuit 40a. Degeneration processing may be performed.

Example 16. In the above-mentioned Examples 12 to 15, the retinal reflection image was obtained by using the difference image with respect to the intensity of the illumination light. However, the gain of the camera was made variable, and instead of the LED strong light emission, the camera gain was large, and instead of the LED weak light emission. Even if the camera gain is small, the same effect can be obtained. 31 is a configuration diagram showing a doze detection device according to a sixteenth embodiment of the present invention. In the figure, 10
Is a CCD camera having a function of changing the size of the output image signal in accordance with the input gain signal, and 16 is a gain signal generation circuit for generating a gain signal.

The gain signal generation circuit 16 outputs a signal with a large gain and a signal with a small gain to the CCD camera 10 according to the timing signal according to the frame cycle of the timing generation circuit 31. An image having a different camera gain is input to the image memory 30a at each frame cycle, and the image in the image memory 30a is transferred to the image memory 30b at each image input timing. As a result, the image in the previous frame is stored in the image memory 30b. Is stored. When a bright image having a large gain is input to the image memory 30a and a dark image having a small gain is input to the image memory 30b, the difference circuit 34 performs a calculation for subtracting the dark image having a small gain from the bright image having a large gain. . The image in the image memory 30c is the same as that in FIG. The subsequent processing is the same as in the twelfth embodiment.

In this embodiment, the object to be detected is photographed by switching the magnitude of the gain for each frame period, but any object that changes the gray level of the photographed image may be used, and the aperture of the camera may be changed. That is, instead of the gain magnitude, the aperture opening degree and the aperture opening degree may be switched for each frame cycle. Further, the shutter speed of the camera may be variable, and shooting with a slow shutter speed and shooting with a fast shutter speed may be switched for each frame cycle.

Thus, instead of the intensity of the illumination light, C
If the camera gain of the CD camera and the input gain of the camera (optical input means) such as the aperture and the shutter speed are changed and the same operation is performed, SN may decrease, but L
There are advantages such as extending the life of the ED.

Example 17 32 is a block diagram showing a doze detection device according to a seventeenth embodiment of the present invention. In this embodiment, the detection target person 1 is a CCD
An LED 13a that illuminates the camera 10 coaxially, and an LED 13 that illuminates the detection target person 1 non-coaxially with the CCD camera 10.
b is emitted alternately, and the coaxial illumination and the non-coaxial illumination are switched to accurately extract the characteristic points. That is, in the case of coaxial illumination, a reflection image of a spectacle frame or a reflection image and a retina reflection image from a wearing object such as eyeglasses are captured (coaxial image), and in the case of non-coaxial illumination, only a reflection image from the wearing object is imaged. Is done (non-coaxial image). Therefore, the spectacle reflection can be removed by taking the difference image between the coaxial image and the non-coaxial image.

Next, the operation will be described. In FIG. 32, the LED drive circuit 14 alternately switches the infrared LED 13a and the infrared LED 13b by the signal of the timing generation circuit 31 for each frame period to emit light. LED
The light emission timing chart and the contents of the image memories 30a and 30b are the same as those in the eighth and twelfth embodiments, and the coaxial image and the non-coaxial image are sequentially stored corresponding to the LEDs that emit light. The enlargement processing circuit 33 and the image memory 30d perform enlargement processing on the non-coaxial image, and the difference circuit 34 subtracts the non-coaxial enlargement processed image from the coaxial image in the image memory 30a to generate a difference image. Hereinafter, as in the thirteenth embodiment, the binarization processing circuit A40
Then, the difference image stored in the image memory 30c is binarized with the threshold value th1 to obtain the retina reflection image of FIG.

In this embodiment, the enlargement processing circuit 33 is provided to enlarge the non-coaxial image and generate a difference image between the non-coaxial image and the coaxial image. This is to ensure the removal of the spectacle reflection image. That is, since the positions of the LEDs 13a and 13b are different,
As a result of the position of the reflected image being slightly different between the coaxial image and the non-coaxial image, if the difference is taken as it is, a part of the reflected image of the glasses remains,
By enlarging the non-coaxial image, enlarging the spectacle reflection image, and taking the difference, all the spectacle reflection images can be removed.

As in the fourteenth embodiment, if the degeneracy processing circuit 32 is inserted after the image memory 30a and the coaxial image is degeneracy processed and the difference from the non-coaxial enlargement processed image is taken, the face moves slightly. Will also be able to remove eyeglass reflections.

If the LEDs are arranged in the vertical direction, the removal effect can be improved by replacing the 5 × 5 max filter with a 7 × 5 max filter and strengthening the enlargement direction in the vertical direction.

Example 18. In the seventeenth embodiment, the feature points of the eyeball are extracted using only the difference image, but as with the eighth and fifteenth embodiments, two binarized images may be obtained and the masking process may be performed.
That is, as shown in FIG. 19, the second binarization processing circuit 40
b, the non-coaxial image is input from the image memory 30a,
The binarization processing is performed with the second threshold value th2 to obtain the eyeglass reflection image which is the second binarization processed image, while the binarization processing circuit 4
0a, the difference image is binarized with the first threshold value th1, a retinal reflection image that is a first binarized image is obtained, and both binarized images are masked by the mask circuit 65. An image similar to that of 4 may be obtained.

Further, with respect to the present embodiment, either one or both of the first binarization-processed image degeneracy process and the second binarization-processed image enlargement process may be performed.

Further, the enlargement processing may be performed before the binarization processing is performed by the second binarization processing circuit 40b, or the difference image may be displayed before the binarization processing is performed by the first binarization processing circuit 40a. Degeneration processing may be performed.

Example 19 33 is a block diagram showing a doze detection device according to a nineteenth embodiment of the present invention. The present embodiment does not obtain the retinal reflection image by using the difference image for the intensity of the illumination light, the magnitude of the input gain, or the coaxial or non-coaxial illumination as in the above-described embodiment, but the intensity of the illumination light, or the input Two images for high and low gain, coaxial and non-coaxial illumination,
Binarization processing is performed using the threshold value of 1 and the second threshold value, and the reflection image from the eyeball is extracted as a feature point on the face using these two binarized images. In FIG. 33,
The LED driving circuit 14 causes the infrared LED 13 to emit light by switching between strong / weak for each frame cycle according to a signal from the timing generation circuit 31. The first binarization processing circuit 40a inputs the bright captured image from the image memory 30a, and outputs the first threshold t
Binarization processing is performed at h1 to obtain an image including the retina reflection image and the eyeglass reflection image, which is the first binarized image. on the other hand,
In the second binarization processing circuit 40b, the dark image is input from the image memory 30b, binarization processing is performed with the second threshold value th21, and the eyeglass reflection image which is the second binarization processing image is included. Get the image that is displayed. The masking circuit 65 masks the first binarized image with the second binarized image to obtain an image similar to that shown in FIG. The following operation is the same as that of each of the above-mentioned embodiments.

In the present embodiment, the binarization memory 4
After 1c, as in the third embodiment, a white area degeneration processing circuit may be added.

Further, either one of the degeneracy process of the first binarized image and the enlarging process of the second binarized image,
Alternatively, the mask processing may be performed after performing both.

Example 20. 34 is a configuration diagram showing a doze detection device according to a twentieth embodiment of the present invention. In the present embodiment, the apparatus shown in FIG. 33 is enlarged by the enlargement processing circuit 33 before being binarized by the second binarization processing circuit 40b and binarized by the first binarization processing circuit 40a. The degeneracy processing circuit 32 performs a degeneracy process before the digitization process.

Although the nineteenth and twentieth embodiments have been described as switching the intensity of the illumination light, the same configuration can be applied to switching the input gain or switching between coaxial and non-coaxial illumination. Thus, Example 1
In Nos. 9 and 20, the number of image memories is reduced and the binary memory is increased instead. However, since the binary memory has smaller hardware, the memory capacity becomes smaller and the apparatus becomes smaller. Further, mask processing is performed instead of the difference processing of the grayscale image, but similarly, the processing circuit becomes simple and the apparatus becomes compact.

Example 21. FIG. 35 is a configuration diagram showing a doze detection device according to a twenty-first embodiment of the present invention, and FIG. 36 is an explanatory diagram illustrating an operation of the doze detection device according to the twenty-first embodiment. In the nineteenth and twentieth embodiments, the bright and dark photographed images stored in the image memories 30a and 30b are binarized by the binarization processing circuits 40a and 40b, respectively. However, in this embodiment, the image memory and the binarization processing circuit are binarized. In the feature calculation circuit 50, and the binarization processing circuit 40
On the other hand, as shown in FIG. 36, the binarization threshold value is switched every frame period. By doing so, the binarized image stored in the binarized memories 41a, 41b, 41c becomes the same as that of the nineteenth embodiment. In this embodiment, the number of binarization processing circuits is one, and the image memory is also reduced, so that the circuit becomes smaller.

Example 22. 37 is a block diagram showing a doze detection device according to a twenty-second embodiment of the present invention. In the present embodiment, the retina reflection image can be accurately extracted even when the face moves, and the illumination light is made to emit strong and weak light to obtain the difference between the sequentially obtained dark photographed image, bright photographed image, and dark photographed image. As a result, the reflected image from the eyeball is extracted as a feature point on the face. In FIG. 37, 30d is an image memory for storing an image obtained by enlarging the i-th captured dark captured image, 30b is an image memory for storing the i-1st captured bright captured image, and 30f is an i.
-An image memory for storing an image obtained by enlarging the dark photographed image taken second, 35 is a first difference circuit for performing a difference process between the i-th photographed bright photographed image and the i-th photographed dark photographed image , 36 is a second difference circuit that performs a difference process on the i-1st bright photographed image to the i-2nd dark photographed image, and 30g is the first difference circuit from the first difference circuit 35.
Image memory for storing the differential image of the second differential image, and 30h for storing the second differential image from the second differential circuit 36.
Reference numeral 40 denotes a binarization processing circuit A that processes the first difference image or the second difference image according to the first threshold value, and 37 denotes the first difference processing image and the second difference processing image respectively processed by the binarization processing circuit A. AND circuit for taking an AND image with the difference processed image of 40d
Is a binarization memory for storing an AND image.

Next, the operation will be described. The LED drive circuit 14 switches the intensity of the infrared LED 13 for each frame period to emit light, and the image memory 30a and the image memory 30b alternately store a dark image and a bright image corresponding to the intensity of the illumination light. . In FIG. 38, the image memories 30d, 30b, 30 when the detection target person moves from left to right
The images stored in f, 30g, and 30h are schematically shown focusing on the eyeglass reflection image and the retinal reflection image. Images 30F and 3F obtained by enlarging the i-2nd and i-th dark images.
In 0D, the spectacle reflection image is magnified and captured, and the retina reflection image is not captured. Further, since the face is moving, the spectacles reflection image moves on the captured image, the spectacles reflection image in the image 30F appears to the left, and the spectacles reflection image in the image 30D appears to the right. In the i-1th bright image 30B, a spectacle reflection image and a retinal reflection image are captured, and the spectacle reflection image appears in the center. The difference processing is performed by the first difference circuit 35,
The image 30G stored in the memory 30g and the image 30H subjected to the difference processing by the second difference circuit 36 and stored in the memory 30h are binarized by the binarization processing circuit A40 by the first threshold, and the AND operation is performed. When AND is taken in the circuit 37,
The AND image shown in the image 40D is obtained. The AND image 40D is an image in which most of the reflection image of the glasses is removed and the reflection image of the retina can be extracted.

Thus, for example, when taking a difference image in the thirteenth embodiment, the eyeglass reflection image may not be sufficiently removed when the face moves, but when processing is performed using two difference images as in the present embodiment. , The retinal reflection image can be accurately extracted. The feature calculation circuit 50 extracts only the retina reflection image from the shape and the like, and detects the dozing in the same manner as in the above embodiments.

Example 23. 39 is a configuration diagram showing a doze detection device according to a twenty-third embodiment of the present invention. This embodiment is different from the one of the twenty-second embodiment in that the bright photographed image 30B stored in the image memory 30b is used.
Is subjected to the degeneracy processing by the degeneracy processing circuit 32 to reduce the spectacle reflection image on the side to be subtracted. As a result, in the first difference image 30G and the second difference image 30H, the remaining portion of the spectacle reflection image becomes small, and as a result, the spectacle reflection image hardly remains in the AND image 40D.

In this embodiment, both the degeneration processing circuit 32 and the enlargement processing circuit 33 are shown, but the enlargement processing circuit 33 may be omitted, or both may be omitted.

In the twenty-second and twenty-third embodiments, the intensity of the illumination light is switched and the image processing is performed by taking two types of differences, but the input gain is switched as in the sixteenth embodiment.
Alternatively, the configuration may be such that the coaxial illumination and the non-coaxial illumination are switched as in the seventeenth embodiment and two types of differences are obtained as in the twenty-second and twenty-third embodiments.

Example 24. 40 is a block diagram showing a doze detection device according to a twenty-fourth embodiment of the present invention. In the present embodiment, similar to Embodiments 22 and 23, the retinal reflection image can be accurately extracted even when the face moves, and the two types of differences are different. In FIG. 40, reference numeral 38 is a difference circuit for performing a difference process on the difference image 30G obtained by subtracting the i-1st photographed bright photographed image from the i-th photographed dark photographed image, and further the i-2nd photographed dark photographed image. And 30j is an image memory for storing this difference image.

Next, the operation will be described. The LED drive circuit 14 switches the intensity of the infrared LED 13 for each frame period to emit light, and the image memory 30a and the image memory 30b alternately store a dark image and a bright image corresponding to the intensity of the illumination light. . In FIG. 41, the image memories 30d, 30b, 30 when the detection target person moves from left to right
The images stored in f, 30g, and 30j are schematically shown by focusing on the eyeglass reflection image and the retinal reflection image. Images 30F and 3F obtained by enlarging the i-2nd and i-th dark images.
The bright photographed image 30B photographed at 0D and i−1th and the image 30G that has been subjected to the difference processing by the first difference circuit 35 and stored in the memory 30g are the same as those in the above-described twenty-second embodiment. The difference circuit 38 subtracts the i-2nd dark image captured from 30G. When the difference image 30J is binarized by the binarization processing circuit A40 with the first threshold value, the AND image 40D obtained in the twenty-second embodiment is obtained.
An image similar to that obtained is obtained, and most of the reflection image of the eyeglasses is almost removed, and the retina reflection image can be extracted.

Example 25. 42 is a configuration diagram showing a doze detection device according to a twenty-fifth embodiment of the present invention. Similar to the twenty-third embodiment, this embodiment is the same as the twenty-fourth embodiment, except that the bright captured image 30B stored in the image memory 30b is subjected to the degeneracy processing by the degeneracy processing circuit 32 and is reduced to the side to be subtracted. The eyeglass reflection image becomes smaller. As a result, as in the twenty-third embodiment, almost no spectacle reflection image remains.

In this embodiment, both the degeneration processing circuit 32 and the enlargement processing circuit 33 are shown. However, the enlargement processing circuit 33 may be omitted, or both may not be provided.

In the twenty-second and twenty-third embodiments, the intensity of the illumination light is switched and the difference is doubled from the bright photographed image to perform image processing. However, similar to the sixteenth embodiment, the coaxial illumination and the non-coaxial illumination are used. And the images of Examples 22 and 2 from the coaxial image.
As in the case of 3, the configuration may take a double difference.

Example 26. 43 is a configuration diagram showing a doze detection device according to a twenty-sixth embodiment of the present invention. Depending on the shape of the spectacles, the reflected image on the spectacle lens will move just by slightly shifting the position of the illumination, and the reflected image on the spectacle lens will remain even if the difference between coaxial and non-coaxial illumination is taken. Sometimes it cannot be extracted. The present embodiment is an apparatus capable of accurately extracting a retina reflection image even in such a case, and performs non-coaxial illumination with two optical axes that are symmetrical with respect to the optical axis in coaxial illumination. In FIG. 43, the person to be detected is illuminated from the left and right by the LEDs 13b and 13c with respect to the optical axis of the coaxial illumination, and non-coaxial illumination is performed.

Next, the operation will be described. The LED drive circuit 14 switches the infrared LEDs 13a, 13b, 13c to non-coaxial left, coaxial, non-coaxial right, coaxial, non-coaxial left, coaxial, non-coaxial right ... A coaxial image and a non-coaxial image corresponding to coaxial and non-coaxial are alternately stored in the. When the non-coaxial left image is input to the image memory 30a, it is enlarged by the enlargement processing circuit 33 and stored in the image memory 30d. Next, when the coaxial image is input to the image memory 30a, it is stored in the image memory 30b. At this time, the non-coaxial left enlarged image in the image memory 30d is transferred to the image memory 30f. This process is repeated every time an image is input. For example, when the i-th non-coaxial right image is stored in the image memory 30d, the image memory 30b stores the i-1th coaxial image, and the image memory 30f stores the i-2th non-coaxial left image. . The images stored in the image memories 30d, 30b, 30f, 30g, 30h are the same as those in FIG. That is, the image 30F is i-
It is an enlarged image of the second non-coaxial left non-coaxial image captured, and the spectacle reflection image appears to the left. Image 30D is i
It is a magnified image of the non-coaxial image on the right of the non-coaxial right taken the second time, and the spectacle reflection image appears to the right. 30B is a coaxial image of the coaxial illumination taken at the (i-1) th position, and the reflection image of the glasses appears in the center. By taking two types of differences with respect to these images, as in the twenty-second embodiment, most of the spectacle reflection image is almost removed, and the retina reflection image can be extracted accurately.

Note that the configuration of symmetric non-coaxial illumination with respect to coaxial illumination can be applied to Examples 23 to 25. Further, as the non-coaxial illumination that is symmetrical with respect to the coaxial illumination, the vertical direction or the oblique direction may be used.

Example 27. FIG. 44 is a configuration diagram showing a doze detection device according to a twenty-seventh embodiment of the present invention. In this embodiment, the two types of difference images shown in the above-mentioned Embodiments 22 to 23 are used, and similarly to Embodiments 19 to 20, the binarization processing is performed first, and this binarized image is processed. By using it, two types of mask images are obtained instead of two types of difference images, and feature extraction is performed. Figure 44
In, the LED emits light by sequentially switching between weak, strong, weak, and strong. The i-2 dark captured image is binarized by the binarization processing circuit 40b and stored in the binarization memory 41b. The (i-1) th bright photographed image is binarized by the binarization processing circuit 40a and stored in the binarization memory 41a. At this time, the i-2th dark image in the binarization memory 41b is transferred to the binarization memory 41e. Next, the i-th captured dark image is binarized by the binarization processing circuit 40b and stored in the binarization memory 41b. Then i
The second dark shot image is stored in the binarization memory 41b, the i-1th bright shot image is stored in the binarization memory 41a, and the i-2th weak light emission image is stored in the binarization memory 41b. Become. Next, the mask circuit 65a performs a mask process on the i-1th bright photographed image of the binarization memory 41a with the i-th dark photographed image of the binarization memory 41b, and outputs the result to the binarization memory 41f. Output. On the other hand, the mask circuit 65b masks the i-1st bright photographed image of the binarization memory 41a with the i-2nd dark photographed image of the binarization memory 41e, and stores the result in the binarization memory 41g. Output. Then A
The ND circuit 37 ANDs the contents of the binarization memory 41f and the contents of the binarization memory 41g, and binarizes the memory 41h.
Output the result to.

For the twenty-fourth to twenty-sixth embodiments, the binarization process and the masking process may be performed first as in the present embodiment.

With such a configuration, the number of binarization memories is increased as compared with the 22nd to 26th embodiments, but the image memory is small and the difference circuit can be omitted. Therefore, a compact and low-priced device can be obtained. There is.

Example 28. 45 is a block diagram showing a doze detection device according to a twenty-eighth embodiment of the present invention. The present embodiment is different from the structure of the third embodiment in that a spectacle wearing determination means 69 for determining the presence or absence of spectacles is provided. The spectacle wearing determination means 69 performs a labeling process on the white areas in the binarized memory 61 and counts the number of white areas. Since there are two reflected images when the glasses are not worn and three or more when the glasses are worn, the presence or absence of the glasses is determined based on the number of white areas. The feature calculation circuit 50 switches between the feature extraction logic for wearing spectacles and the logic for not wearing spectacles according to the determination result of the spectacle wearing determination means 69. For example, in FIG.
Since there is one, it is determined that the user wears glasses, and the processing for extracting the retina reflection image 6 from the contents of the binarization memory 63 is performed as in the third embodiment. When it is determined that the glasses are not worn, the binarization memory 6
Processing for extracting the retina reflection image 6 from the contents of 1 is performed.

Although the information of the retina reflection image 6 is reduced by the degeneration process, in the case of the present embodiment, the accuracy of pupil extraction when the eyeglasses are worn is improved.

In the above embodiment, the spectacle wearing determination means 6
9 counts the number of white areas in the binarization memory 61,
Although it was determined that the spectacles were worn, the spectacles reflected image had significantly higher brightness than the retina reflected image, and therefore the spectacles wearing determination means 69.
May take in a signal from the image memory 30, look at the maximum brightness, and if it is equal to or higher than a predetermined value, it may be determined to be wearing glasses.

Further, in each of the above-mentioned embodiments, the case of wearing the spectacles has been described, but the same effect can be obtained even with a wearing article such as earrings.

FIG. 46 is a block diagram showing another doze detecting device according to the twenty-eighth embodiment of the present invention. In the embodiment shown in FIG. 45, it is determined by the image processing whether or not the glasses are worn.
5, the feature calculation circuit 50 detects the signal of whether the glasses are worn or not.
You may enter in. The operation of the feature calculation circuit 50 switches between the feature extraction logic for wearing spectacles and the logic for not wearing spectacles in the same manner as in the above embodiment depending on the presence or absence of spectacles. That is, when the spectacles are worn, the process of extracting the retinal reflection image 6 from the contents of the binarized memory 63 is performed, and when the spectacles are not worn, the process of extracting the retinal reflection images 6 from the contents of the binarized memory 61 is performed.

FIG. 47 is a block diagram showing still another doze detecting device according to the twenty-eighth embodiment of the present invention. In FIG. 47, the feature calculation circuit 50 not only switches between the feature extraction logic for wearing spectacles and the logic for not wearing spectacles, but further changes the shooting condition depending on the presence or absence of spectacles. That is, since the retina reflection image 6 becomes dark when the spectacles are worn by the feature calculation circuit 50, when the spectacles are worn, the radiant power of the infrared LED 13 is increased as compared with the naked eye.
The drive circuit 14 is controlled. Other operations are the same as those of the embodiment shown in FIG. By doing so, since the illumination is performed with an appropriate amount of light, it is possible to correctly extract the feature points even when wearing glasses or with the naked eye.

With respect to the apparatus shown in FIG. 45, the photographing method may be switched depending on whether or not the glasses are worn. Also, the radiant power of the LED was changed by judging whether or not the glasses were worn, but when the glasses were worn, the gain of the camera was increased.
The same effect can be obtained by lowering the gain of the camera with the naked eye.

Example 29. In the twenty-eighth embodiment, the feature calculation circuit 50
Depending on the presence or absence of eyeglasses, the image processing for wearing eyeglasses and the one for switching image processing for not wearing eyeglasses are shown, but in advance, image processing for wearing eyeglasses and retina reflection image by image processing for not wearing eyeglasses are performed. The image processing result may be selected and the image processing result may be selected depending on the presence or absence of glasses.

[0141] Alternatively, feature calculation circuit 50, and an image processing of the image processing and non-ophthalmic mounting for wearing glasses, but it may also select an image processing result of either the image processing result. For example, in the case of FIG. 47, the number of white areas in the binarization memory 61 and the number of white areas in the binarization memory 63 are compared, and the number of white areas in the binarization memory 61 and the binarization memory 6 are compared.
If the number of white areas of 3 is the same, it is determined that no glasses are attached and 2
The feature extraction result by the binarization memory 61 is selected, and if the numbers are different, the feature extraction result by the binarization memory 63 is selected for wearing glasses. Further, since the area of the white region of the binarization memory 61 and the area of the white region of the binarization memory 63 are stored for a certain period of time, and the area of the white region becomes smaller due to blinking,
The feature extraction result having a larger area reduction ratio may be selected.

Further, in the above-described embodiment, the image processing or the image processing result is selected according to the presence / absence of wearing of the glasses in contrast to the configuration of the third embodiment, but it is applied to other embodiments. May be.

Example 30. Furthermore, in each of the above-described embodiments, in the binarization processing circuit, the first threshold value th1 and the second threshold value th2 use predetermined predetermined values, but the captured image, the illuminance in the vicinity of the detection target person,
Alternatively, if the first threshold value and the second threshold value are determined based on the binarized image, the retina reflection image and the reflection image from the mounted object can be more accurately discriminated. In this embodiment, the thresholds th1 and th2 are set according to the input image. FIG. 48 is a block diagram showing an example of such a drowsiness detection device. In the figure, reference numeral 64 denotes a histogram calculation circuit, which inputs an image from the image memory 30, and based on the maximum density, average density and variance of the input image, thresholds th1 and th2 are set as follows.
To set. th1 = average density + 3 × dispersion (1) th2 = maximum density− (maximum density−th1) / 2 (2) Binarization processing using the threshold values th1 and th2 thus obtained By doing so, the retina reflection image can be accurately extracted.

In the above embodiment, the maximum density is input from the histogram calculation circuit 64 in the feature extraction circuit 50, the LED light amount is changed according to the maximum density of the input image, and the lens reflection image 8 and the spectacle frame reflection image are changed. If 9 is set not to overflow, a difference in density between the retina reflection image 6 and the lens reflection image 8 is likely to occur, and the threshold values th1 and th2
Is easy to determine. FIG. 49 shows such an embodiment, and the feature extraction circuit 50 issues a command to the LED drive circuit 14 to reduce the light amount when the maximum density is close to 255.

Example 31. Although the threshold value is determined by using the average density of the entire input image in the above-mentioned Example 30, FIG. 50 shows the density distribution of the entire image including retinal reflection at night, and the number of pixels having the density value a is n.
(A). Here, a takes a value from 0 to 255. A is the minimum density, B is the rising density of the density distribution, and C is the maximum density (density max). Figure 50
In, the face area and the background are dark and therefore have a darker density than point B. On the other hand, the retina reflection image and the reflection image of the eyeglasses are remarkably bright and have a small number of pixels, so that they are within the distribution of points B to C. Therefore, in the threshold value determination method based on the average density of the entire image of the thirty-third embodiment, the threshold value is determined from the density distribution of the face area or the background with poor S / N, and it is found that the optimum value cannot always be obtained. . In this embodiment, the rising density B of the density distribution is found, and the binarization threshold value is determined by paying attention to the density distribution of the pixels having the density B or higher, and a more optimal threshold value is obtained.

The method of obtaining the density B is calculated as follows, for example. First, to find the approximate position of point B,
The density value is divided into, for example, 16 sections, and the number of pixels in section i is N
(I) is calculated by the following equation.

[0147]

[Equation 1]

Next, the difference M (i) between the adjacent partitions is calculated as follows: M (i) = N (i) -N (i-1) where i = 1 to 15 The difference M (i) is a constant value. Sections having the above-described number Mth of pixels Mth such that sections M (i) ≧ Mth, j1, j2 ,. ． ． , J are extracted, and the section J having the maximum density value is selected. Point B is a density value having the smallest number of pixels in this section J, that is, MIN {n (16J), n (16J + 1) ,. ． ． Let n (16J + 15)} be the density value.

For the point B thus obtained, the threshold value is selected as follows: th1 = density B + α1 th2 = density max−α2 where α1 and α2 are offset values, for example α1 =
10, and α2 = 20.

Alternatively, th1 = concentration B + concentration average value above concentration B × β1 + α1 th2 = concentration max−concentration average value above concentration B × β2-α2 (where α1, α2, β1 and β2 are For example, α1 = 5, α2 = 10, β1 = β2 = 0.2) Regarding the method of selecting the threshold value, the latter threshold value requires more calculation time, but the accuracy is higher in the sense that the feature points can be extracted reliably.

Further, FIGS. 51 (a) and 51 (b) respectively show the density distribution of the bright photographed image and the density distribution of the dark photographed image at night. As shown in FIG. 51, the density of the spectacle reflection image is located higher than the density of the retinal reflection image, and in the case of bright photographing (for example, strong light emission), the retinal reflection image needs to be extracted. And th1 = density B + α1, and in the case of dark photography (for example, weak light emission), it is necessary to extract only the eyeglass reflection image, so th2 = density max−density average value of density B or more × β2-α2 Good.

Example 32. Further, as a method of setting the threshold values th1 and th2, it is possible to detect the illuminance in the vicinity of the person to be detected by the illuminance sensor and set the threshold values th1 and th2 based on the illuminance. For example, a threshold th1 is a value that increases as the illuminance increases, and a map in which th2 is constant is used, and the thresholds th1 and th2 are determined based on the detected illuminance. Alternatively, considering that the density of the retinal reflection image with respect to the average density of the face is smaller when it is bright than when it is dark, the above equation (1) is set as th1 = average density + x (s) × dispersion, Using a map in which x (s) is set to a value that decreases as the illuminance increases, the threshold value th is determined based on the maximum density, average density, and variance of the input image and the detected illuminance.
1. Determine th2.

Example 33. The brightness and size of the retina reflection image differ depending on the person to be detected. Further, the spectacles also differ in the brightness and size of the spectacles reflected image. In this embodiment, an optimum threshold value is determined based on a plurality of binarized images processed with different threshold values, and image processing by the optimum threshold value or an image processing result is selected. 52 is a block diagram showing an image processing apparatus according to a 33rd embodiment of the present invention. In FIG. 52, a binarization processing circuit C
66 is four threshold values th1 for the predetermined density
It has 1, th12, th13 and th2. th2
Is a threshold for detecting a reflection image of eyeglasses similar to each of the above-mentioned embodiments, and th11, th12, and th13 are thresholds for detecting a reflection image of retina, and th11>th12> th13.
41b is a second binarization memory for storing the result of binarization processing with the threshold value th2, 411a is a first binarization memory for storing the result of binarization processing with the threshold value th11, and 412a is 2 bits for the threshold value th12. The first 2 that stores the result of the binarization process
The binarization memory 413a is a first binarization memory for storing the result of binarization processing with the threshold value th13, 39 is a signal switcher, and the binarization memories 411a, 412a are designated by the command of the feature calculation circuit 50. One output of 413a is connected to the mask circuit 65.

Next, the operation will be described. Image memory 3
When an image is input to 0, the feature calculation circuit 50 sends a command to the signal switch 39 to connect the binarization memory 411a and the mask circuit 65. The binarized image of the binarized memory 411a is masked by the masked circuit 65 by the binarized image of the binarized memory 41b, and the white area degeneracy circuit 62 is displayed.
After that, it is stored in the binarization memory 63. The feature calculation circuit 50 checks the area of the white area and the number of white areas of this image in the binarization memory 63. Next, the feature calculation circuit 50 connects the binarization memory 412a and the mask circuit 65 and performs the same processing. Finally, the feature calculation circuit 50 connects the binarization memory 413a and the mask circuit 65 and performs the same processing. Next, the feature calculation circuit 50 uses the binarization memory 411.
a, 412a, 413a, respectively,
Area information S1, S2, S3 and number information N1, N2, N3
Is used to select the result that produces the optimum result, and thereafter, the binarization memory and the mask circuit 65 corresponding to the threshold value are selected.
Connect. The selection criteria for selecting the optimum value using the area and number of white regions are shown below. Quantity information N1, N2, N
When all 3 are 3 or more, the one having the smallest number is selected. If the numbers are the same, the result with the smallest area is the best. If there are N1, N2, and N3 values of 2, the number N = 2 and the result having the largest area is optimized. When N1, N2 and N3 are all less than 2, the retina reflection image cannot be detected and the optimum threshold value selection process is performed again.

FIG. 53 shows an example of selecting the optimum threshold value when the glasses A are worn. N1 = 0, N2 = 2, N3 = 5, S
Since 1 = 0, S2 = 20, and S3 = 40, the result by the binarization memory 412a with N = 2 is selected as the optimum.

FIG. 54 shows an example of selection of the optimum threshold value when the glasses B are worn. N1 = 0, N2 = 2, N3 = 2, S
1 = 0, S2 = 20, S3 = 30, N2 = N3 =
Since it is 2, the sizes of S2 and S3 are compared. Since S3 is larger, the result of the binarization memory 413a is selected as the optimum one.

The reason for selecting the number of white areas is that the closer the number is to two, the more likely it is that the reflection image of the glasses is removed and only the retina reflection image is extracted. Also,
The reason for selection by the area when N = 2 is that the larger the area is, the higher the ratio of extracting the retina reflection image is. The reason for selecting by area when N = 3 or more is that the larger the area is, the higher the proportion of the spectacle reflection image is mixed.

Although the three first threshold values for extracting the retina reflection image are provided in the thirty-third embodiment, the same processing may be performed by providing a plurality of second threshold values for extracting the eyeglass reflection image. .

Further, although the white area is reduced in the 33rd embodiment, the optimum threshold value may be determined based on the shape of the white area without performing the reduction processing. For example, it may be determined by the number or area of white regions excluding donut-shaped white regions.

Example 34. In Example 33, the retinal reflection image was extracted using a plurality of binarized images with a plurality of threshold values, but the threshold value may be selected based on the processing result of changing the threshold value. FIG. 55
FIG. 38 is a configuration diagram showing an image processing device according to a thirty-fourth embodiment of the present invention. In FIG. 55, the feature calculation circuit 50 sequentially sets the threshold values th11, th12, and th13 for extracting the retina reflection image in the binarization processing circuit C66, and the binarization memory 63.
The optimum threshold value is set based on the image. In this embodiment, the threshold value th2 for detecting the reflected image of glasses is constant.

For example, when the glasses A are worn, th11
And th2, the result is N as in the 33rd embodiment.
1 = 0 and S1 = 0. The feature calculation circuit 50 sets th12 and th2 in the binarization processing circuit C66, and the result N2 =
2, S2 = 20 is obtained. Next, the feature calculation circuit 50 sets th13 and th2 in the binarization processing circuit C66, and the result N3
= 5 and S3 = 40 are obtained. The feature calculation circuit 50 is the third embodiment.
The optimum threshold value th12 is selected according to the same selection criteria as in 3. In the case of the eyeglasses B, similarly, the optimum threshold value th13 is selected. N1 = 0, N2 = 2, N3 = 5, S1 = 0, S2
= 20 and S3 = 40, the result by the binarization memory 412a with N = 2 is selected as the optimum.

In the thirty-fourth embodiment, the first threshold value does not have to be three, and the second threshold value for eyeglass reflection image extraction may be changed.

Further, for each of the above-described first to twenty-ninth embodiments, as in the thirty-third embodiments, the image processing may be performed with a plurality of threshold values to set the optimum threshold value.

[0164]

【The invention's effect】

In claim 1 of the present invention, illumination light control means for controlling the presence / absence of illumination light, bright photographic image output means for outputting an image of a person to be detected with illumination light, and detection object without illumination light. And a difference image output unit for outputting a difference image between the bright and dark images, and the feature point extracting unit outputs the difference image as the first difference image.
First binarized image processed with the threshold of
Since the reflection image from the eyeball is extracted as a feature point on the face by using the binarized image obtained by processing the image with the second threshold value, the reflection image of the eyeball and the reflection image of the wearer of the detection target person.
Can be reliably identified, and feature points of the detection target can be accurately extracted.
It is possible to provide an image processing device that can perform various operations.

Further, in claim 2 , a first binarized image obtained by processing the difference image with the first threshold value and a second binarized image obtained by processing the bright photographed image with the second threshold value. Since the reflected image from the eyeball is extracted as a feature point on the face by using and, the ambient light changes in addition to the effect of claim 1.
The feature point extraction accuracy for the case is improved.

Further, in claim 3 , the illumination light control means for controlling the intensity of the illumination light or the input gain of the light input means, and the bright photographing for outputting the image of the person to be detected whose illumination light is strong or whose input gain is large. Image output means, dark-photographed image output means for outputting an image of a detection target person whose illumination light is weak or input gain is small, and differential image output means for outputting a differential image between a bright-photographed image and a dark-photographed image. The extraction means is a first binarized image obtained by processing the difference image with a first threshold value.
And the second binarized image obtained by processing the dark image with the second threshold value, the reflection image from the eyeball is extracted as a feature point on the face. In addition to the effect of claim 1, the feature point detection accuracy is further improved.

[0168] In the fourth aspect, the feature point extracting means, and the dark photographed images taken in i-th, and the bright photographic images taken in (i-1) -th, and the dark photographed images taken in i-2 th Using the above, the first difference image obtained by performing a difference process on the i-th imaged bright-photographed image from the i-th imaged dark-photographed image and the i-1-th imaged bright-photographed image A second difference image obtained by performing a difference process on the i-2nd dark image taken from the image is obtained, and the reflection image from the eyeball is obtained by the first difference image and the second difference image. Since it is extracted as the above feature point, in addition to the effect of claim 3 , even when the face of the detection target person moves, the feature point can be accurately extracted, and the extraction accuracy does not decrease.

[0169] In the fifth aspect, the feature point extracting means, and the dark photographed images taken in i-th, and the bright photographic images taken in (i-1) -th, and the dark photographed images taken in i-2 th A difference image obtained by performing a difference process between the i-1th bright-photographed image and the i-th dark-photographed image and the i-2nd dark-photographed image using Is obtained and the reflection image from the eyeball is extracted as a feature point on the face from the difference image, and therefore, the same effect as in claim 4 is obtained.

Further, in claim 6 , a coaxial illumination means for illuminating a predetermined area including the face of the detection target person coaxially with the light input means, and a predetermined area including the face of the detection target person non-coaxial with the light input means. Non-coaxial illumination means for illuminating with, coaxial illumination means for controlling the presence or absence of light emission of the coaxial illumination means and non-coaxial illumination means, coaxial image output means for outputting the image of the person to be detected by the coaxial illumination light, non-coaxial illumination light The non-coaxial image output unit that outputs the image of the detection target person and the difference image output unit that outputs the difference image between the coaxial image and the non-coaxial image are provided, and the feature point extraction unit processes the difference image with the first threshold value. First
Since the reflection image from the eyeball is extracted as the feature point on the face using the binarized image of No. 1 and the binarized image obtained by processing the non-coaxial image with the second threshold value, the effect of claim 1 In addition, the feature point extraction accuracy for the case where the ambient light changes is improved.

[0171] Furthermore, in claim 7, the feature point extraction unit, a non-coaxial images taken in i-th, coaxial image captured in (i-1) -th, and a non-coaxial pictures recorded i-2 th Using the first differential image obtained by subjecting the i-1th captured coaxial image to the i-th captured non-coaxial image, and the i-1st captured coaxial image to the above The second difference image obtained by performing the difference processing on the second captured non-coaxial image is obtained, and the reflection image from the eyeball is determined by the first difference image and the second difference image on the face. Since the points are extracted, in addition to the effect of claim 6 , even when the face of the detection target person moves, the feature points can be accurately extracted, and the extraction accuracy does not decrease.

[0172] Further, in claim 8, the feature point extraction unit, a non-coaxial images taken in i-th, coaxial image captured in (i-1) -th, and a non-coaxial pictures recorded i-2 th By using the i-1th captured coaxial image, a difference image obtained by performing a difference process between the i-th captured non-coaxial image and the i-2nd captured non-coaxial image is obtained. Since the reflection image from the eyeball is extracted as the feature point on the face from the difference image, the same effect as in claim 7 is obtained.

In the ninth aspect, the feature point extracting means may include a first binarized image obtained by processing the difference image with a first threshold value and a dark image or a non-coaxial image as a second image.
The reflection image from the eyeball is extracted as a feature point on the face by using the second binarized image processed by the threshold value of 1. Therefore, in addition to the effect of claim 3 or claim 6 , Since the influence of the image can be reduced, the feature point extraction accuracy is improved.

In the tenth aspect, the feature point extracting means processes the first difference image obtained by processing the first difference image with the first threshold value and the second difference image according to the first threshold value. using the a second differential processing images, since to extract the reflected image from the eye as feature points on the face, according to claim 4
Alternatively, in addition to the effect of claim 7 , there is an effect that feature points can be extracted by a simple image calculation.

In the eleventh aspect , illumination light control means for controlling the intensity of illumination light or the input gain of the light input means, and bright photographing for outputting an image of a person to be detected having a high illumination light or a large input gain. An image output unit, a dark-photographed image output unit that outputs an image of a detection target person whose illumination light is weak or whose input gain is small, and the feature point extraction unit is a first feature obtained by processing a bright-photographed image with a first threshold value. and binarization image, by using the second binarization image where the dark captured image treated by the second threshold value, since to extract the reflected image from the eye as feature points on the face, according to claim 3 In addition to the above effect, since the difference processing of the grayscale image is eliminated, the memory capacity for the image is reduced, the processing circuit is simplified, and the apparatus is small and inexpensive.

In the twelfth aspect of the present invention, the feature point extracting means has a binarized image of the i-th photographed dark photographed image and a binarized image of the i-1-th photographed bright photographed image. , The i-th second dark-photographed image is binarized, and the i-th first bright-photographed image
Binarization of the first mask image obtained by masking the binarized image of the i-th dark photographed image and the i-1 th bright photographed image The second mask image obtained by masking the processed image with the binarized image of the dark-photographed image i-2 above is obtained, and using the first mask image and the second mask image,
Since the reflection image from the eyeball is extracted as the feature point on the face, in addition to the effect of claim 4 , there is an effect that the memory capacity for the image becomes small, the processing circuit becomes simple, and the apparatus becomes small and inexpensive. .

Further, in the thirteenth aspect, the feature point extracting means includes a binarized image of the i-th photographed dark photographed image and a binarized image of the i-1-th photographed bright photographed image. , The i-th second dark-photographed image is binarized, and the i-th first bright-photographed image
The binarized image of the i-th captured image is 2
A mask image obtained by performing a masking process is obtained from the binarized image of the i-2nd dark-photographed image and the mask image is used to obtain a reflection image from the eyeball on the face. Since it is extracted as the above feature point, it has the same effect as that of claim 12 .

In the fourteenth aspect of the present invention, the coaxial illumination means illuminates a predetermined area including the face of the detection target person coaxially with the light input means, and the predetermined area including the face of the detection target person is non-coaxial with the light input means. Non-coaxial illumination means for illuminating, illumination light control means for controlling presence / absence of light emission of the coaxial illumination means and non-coaxial illumination means, coaxial image output means for outputting an image of a person to be detected by coaxial illumination light, detection by non-coaxial illumination light The non-coaxial image output means for outputting the image of the subject is provided, and the feature point extraction means uses the first binarized image obtained by processing the coaxial image with the first threshold value and the non-coaxial image with the second threshold value. Since the reflected image from the eyeball is extracted as a feature point on the face using the processed second binarized image, in addition to the effect of claim 6, as a result of eliminating the difference processing of the grayscale image, The memory capacity of Device processing circuit becomes simple an effect of less expensive small.

In the fifteenth aspect, the feature point extracting means includes a binarized image of the i-th captured non-coaxial image and a binarized image of the i-1th captured coaxial image. The i-th second non-coaxial image binarized image is used to convert the i-1 th coaxial image binarized image into the i-th non-coaxial image binarized image The first mask image obtained by masking with the digitized image and the binarized image of the i-1th captured coaxial image are binarized of the i-2nd non-coaxial image. Since the second mask image obtained by performing the mask processing on the processed image is obtained, and the reflection image from the eyeball is extracted as the feature point on the face by the first mask image and the second mask image,
In addition to the effect of the seventh aspect , there is an effect that the memory capacity for an image becomes small, the processing circuit becomes simple, and the apparatus becomes small and inexpensive.

In the sixteenth aspect, the feature point extracting means includes a binarized image of the i-th captured non-coaxial image and a binarized image of the i-1th captured coaxial image. The i-th second non-coaxial image binarized image is used to convert the i-th first coaxial image binarized image into the i-th non-coaxial image binarized image. 2 of the binarized image and the above i-2 non-coaxial image
Since the mask image obtained by performing the masking process with the binarized image is obtained and the reflected image from the eyeball is extracted as the feature point on the face by the mask image, the same effect as in claim 15 is obtained.

Further, in the seventeenth aspect , since the image subjected to the difference processing or the image subjected to the mask processing is degenerated, the influence of the reflected image of the mounted object can be reduced and the feature point extraction accuracy is improved.

In the eighteenth aspect , since the image to be subjected to the difference processing or the image to be masked is enlarged,
The influence of the reflected image of the mounted object can be reduced, and the feature point extraction accuracy is improved.

In the nineteenth aspect of the invention, since the enlargement or degeneracy process is given a direction corresponding to the positional relationship between the coaxial illumination and the non-coaxial illumination, the influence of the reflected image of the mounted object can be further reduced, and the characteristic point Extraction accuracy increases.

[0184] Also, in claim 20 has a mounting object determining means determines the presence or absence of attachment of the detection subject, the feature point extracting means, depending on the determination result of the mounting object determining means
The first binarized image and the second binarized image
Image processing for mounting using and the first binarized image
Since the image processing for non-wearing is switched or the processing result by each image processing is selected, the optimum image processing can be selected depending on whether or not the glasses are worn, and the accuracy of feature point extraction is improved.

Further, in the twenty-first aspect , since the mounted object determining means detects the presence or absence of the mounted object based on the number of reflection images by the illumination light or the maximum brightness, it is possible to accurately determine the presence or absence of wearing glasses, and the feature point extraction accuracy. Goes up.

In the twenty-second aspect , the attached article is
In this case, since the illumination light is strengthened , the feature points can be correctly extracted even when wearing glasses or the naked eye.

In the twenty-third aspect, the attached article is
If you want to increase the input gain of the optical input means,
Item 22 has the same effect.

Further, in the twenty-fourth aspect, the feature point extracting means is configured so that the first binarization image and the second binarization process are performed.
Image processing for mounting using images and first binarized image
Image processing for non-wearing using, and one of the image processing results is selected according to the image processing result, so that the optimum feature extraction result can be selected from the two processing results of the presence or absence of the wearing object. The accuracy of point extraction is improved.

[Brief description of drawings]

FIG. 1 is a configuration diagram showing a doze detection device according to a first embodiment of the present invention.

FIG. 2 is an explanatory diagram illustrating an operation of the doze detection device according to the first embodiment of the present invention.

FIG. 3 is a flowchart showing an operation of the doze detection device according to the first embodiment of the present invention.

FIG. 4 is an explanatory diagram illustrating an operation of the doze detection device according to the first embodiment of the present invention.

FIG. 5 is an explanatory diagram illustrating an operation of the doze detection device according to the first embodiment of the present invention.

FIG. 6 is a configuration diagram showing an inattentiveness detection device according to a second embodiment of the present invention.

FIG. 7 is an explanatory diagram illustrating an operation of the inattentiveness detection device according to the second embodiment of the present invention.

FIG. 8 is an explanatory diagram illustrating an operation of the inattentiveness detection device according to the second embodiment of the present invention.

FIG. 9 is a configuration diagram showing a doze detection device according to a third embodiment of the present invention.

FIG. 10 is an explanatory diagram illustrating an operation of the doze detection device according to the third embodiment of the present invention.

FIG. 11 is an explanatory diagram illustrating an operation of the doze detection device according to the third embodiment of the present invention.

FIG. 12 is a configuration diagram showing a doze detection device according to a fourth embodiment of the present invention.

FIG. 13 is an explanatory diagram illustrating an operation of the doze detection device according to the fourth embodiment of the present invention.

FIG. 14 is an explanatory diagram illustrating an operation of the doze detection device according to the fourth embodiment of the present invention.

FIG. 15 is a configuration diagram showing an image processing apparatus according to a fifth embodiment of the present invention.

FIG. 16 is a configuration diagram showing a doze detection device according to a sixth embodiment of the present invention.

FIG. 17 is a configuration diagram showing a doze detection device according to a seventh embodiment of the present invention.

FIG. 18 is an explanatory diagram illustrating an operation of the doze detection device according to the eighth embodiment of the present invention.

FIG. 19 is a configuration diagram showing a doze detection device according to an eighth embodiment of the present invention.

FIG. 20 is an explanatory diagram illustrating an operation of the doze detection device according to the eighth embodiment of the present invention.

FIG. 21 is an explanatory diagram illustrating an operation of the doze detection device according to the eighth embodiment of the present invention.

FIG. 22 is a configuration diagram showing a doze detection device according to a ninth embodiment of the present invention.

FIG. 23 is a configuration diagram showing a doze detection device according to a tenth embodiment of the present invention.

FIG. 24 is a configuration diagram showing a doze detection device according to an eleventh embodiment of the present invention.

FIG. 25 is a configuration diagram showing a doze detection device according to a twelfth embodiment of the present invention.

FIG. 26 is a characteristic diagram showing input / output characteristics of a CCD camera according to Embodiment 12 of the present invention.

FIG. 27 is an explanatory diagram illustrating an operation of the doze detection device according to the twelfth embodiment of the present invention.

FIG. 28 is a configuration diagram showing a doze detection device according to a thirteenth embodiment of the present invention.

FIG. 29 is an explanatory diagram illustrating an operation of the doze detection device according to the thirteenth embodiment of the present invention.

FIG. 30 is a configuration diagram showing a doze detection device according to a fourteenth embodiment of the present invention.

FIG. 31 is a configuration diagram showing a doze detection device according to a sixteenth embodiment of the present invention.

FIG. 32 is a configuration diagram showing a doze detection device according to a seventeenth embodiment of the present invention.

FIG. 33 is a configuration diagram showing a doze detection device according to a nineteenth embodiment of the present invention.

FIG. 34 is a configuration diagram showing a doze detection device according to a twentieth embodiment of the present invention.

FIG. 35 is a configuration diagram showing a doze detection device according to a twenty-first embodiment of the present invention.

FIG. 36 is an explanatory diagram illustrating an operation of the doze detection device according to the twenty-first embodiment of the present invention.

FIG. 37 is a configuration diagram showing a doze detection device according to a twenty-second embodiment of the present invention.

FIG. 38 is an explanatory diagram illustrating an operation of the doze detection device according to the twenty-second embodiment of the present invention.

FIG. 39 is a configuration diagram showing a doze detection device according to a twenty-third embodiment of the present invention.

FIG. 40 is a configuration diagram showing a doze detection device according to a twenty-fourth embodiment of the present invention.

FIG. 41 is an explanatory diagram illustrating an operation of the doze detection device according to the twenty-fourth embodiment of the present invention.

FIG. 42 is a configuration diagram showing a doze detection device according to a twenty-fifth embodiment of the present invention.

FIG. 43 is a configuration diagram showing a doze detection device according to a twenty-sixth embodiment of the present invention.

FIG. 44 is a configuration diagram showing a doze detection device according to a twenty-seventh embodiment of the present invention.

FIG. 45 is a configuration diagram showing a doze detection device according to a twenty-eighth embodiment of the present invention.

FIG. 46 is a configuration diagram showing another doze detection device according to the twenty-eighth embodiment of the present invention.

[Fig. 47] Fig. 47 is a configuration diagram showing still another doze detection device according to the twenty-eighth embodiment of the present invention.

FIG. 48 is a configuration diagram showing a doze detection device according to a thirtieth embodiment of the present invention.

FIG. 49 is a configuration diagram showing another doze detection device according to the thirtieth embodiment of the present invention.

FIG. 50 is an explanatory diagram illustrating an operation of the doze detection device according to the thirty-first embodiment of the present invention.

FIG. 51 is an explanatory diagram illustrating an operation of another doze detection device according to the thirty-first embodiment of the present invention.

FIG. 52 is a configuration diagram showing an image processing device according to a working example 33 of the invention.

FIG. 53 is an explanatory diagram illustrating an operation of the image processing apparatus according to the 33rd embodiment of the present invention.

FIG. 54 is an explanatory diagram illustrating an operation of the image processing apparatus according to the thirty-third embodiment of the present invention.

FIG. 55 is a configuration diagram showing an image processing device according to a thirty-fourth embodiment of the present invention.

FIG. 56 is a configuration diagram showing a conventional eye position detecting device.

FIG. 57 is a configuration diagram showing a conventional drowsiness detection device.

FIG. 58 is an explanatory diagram illustrating an operation of the conventional drowsiness detection device.

FIG. 59 is a flowchart showing the operation of the conventional doze detection device.

FIG. 60 is an explanatory diagram illustrating an operation of the conventional drowsiness detection device.

FIG. 61 is an explanatory diagram illustrating an influence of a reflected image of glasses.

[Fig. 62] Fig. 62 is an explanatory diagram for explaining the influence of a reflected image of glasses.

[Explanation of symbols]

1 detection target, 6 retinal reflection image, 8 lens reflection image,
9 eyeglass frame reflection image, 10 CCD camera, 12 half mirror, 13, 13a, 13b, 13c infrared LE
D, 15 switch, 30, 30a, 30b, 30c,
30d, 30e, 30f, 30g, 30h, 30I, 3
0J image memory, 31 timing generation circuit, 32
Degeneration processing circuit, 33 enlargement processing circuit, 34 difference circuit,
35 first difference circuit, 36 second difference circuit, 37
AND circuit, 38 difference circuit, 39 signal switcher, 4
0a first binary processing circuit, 40b second binary processing circuit, 41a first binary memory, 41b second binary memory, 41c third binary memory, 41d,
41e, 41f, 41g, 41h binary memory, 50
Feature calculation circuit, 60 binarization processing circuit B, 61 binarization memory, 62 white area degeneracy processing circuit, 63 binarization memory, 64 histogram calculation circuit, 65 mask circuit,
66 binarization processing circuit C, 67 white area expansion processing circuit,
68 binarization memory, 69 glasses wearing determination means, 70 dozing detection circuit, 71 alarm circuit, 72 inattentive detection circuit,
80 edge extraction circuit, 81 image memory.

Front page continued (72) Inventor Toshihide Satake 8-1-1 Tsukaguchi Honcho, Amagasaki City Mitsubishi Electric Corporation Industrial Systems Research Laboratories (72) Inventor Tai Okawa 8-1-1 Tsukaguchi Honmachi, Amagasaki Mitsubishi Electric Production Technology Center Co., Ltd. (56) Reference JP-A-2-138673 (JP, A) JP-A-5-143714 (JP, A) JP-A-6-274269 (JP, A) JP-A-62-138987 (JP, A) JP 7-134800 (JP, A) JP 7-61256 (JP, A) (58) Fields investigated (Int.Cl. ⁷ , DB name) G06T 1/00 340 B60K 28 / 06 G08B 21/06

Claims (24)

(57) [Claims] 1. A light input unit for inputting an image of a predetermined region including a face of a detection target person, an illumination unit for illuminating a predetermined region including the face of the detection target person, and the output from the light input unit. What is provided with feature point extraction means for extracting feature points on a face from a photographed image of a person to be detected, illumination light control means for controlling the presence or absence of illumination light, and a light for outputting an image of the person to be detected with illumination light. The feature point extracting means includes a captured image output means, a dark captured image output means for outputting an image of a detection target person without illumination light, and a differential image output means for outputting a differential image between a bright captured image and a dark captured image. Has a first threshold value for extracting a reflection image from the eyeball of the detection target person, and a second threshold value for extracting a reflection image from a wearing object worn by the detection target person, the difference image processing by the first threshold value The first processed binary image and
An image processing apparatus, characterized in that a reflected image from an eyeball is extracted as a feature point on a face by using a second binarized image obtained by processing a clearly photographed image with a second threshold value. 2. The feature point extracting means includes a first binarized image obtained by processing the difference image with a first threshold value and a second binarized image obtained by processing a bright image with a second threshold value. DOO using the image processing apparatus according to claim 1, wherein the reflected image from the eyeball and extracts as a feature point on the face. 3. An optical input means for inputting an image of a predetermined area including the face of the detection target person, an illuminating means for illuminating the predetermined area including the face of the detection target person, and the output from the light input means. In a feature point extracting means for extracting feature points on a face from a photographed image of a person to be detected, an illumination light control means for controlling intensity of illumination light or an input gain of the light input means, illumination light being strong or input A bright-photographed image output means for outputting an image of a detection target person with a large gain, a dark-photographed image output means for outputting an image of a detection target person with weak illumination light or a low input gain, and a bright-photographed image and a dark-photographed image. A differential image output means for outputting a differential image, wherein the feature point extraction means is configured to extract a reflection image from the eyeball of the detection target person, and a wearing object worn by the detection target person. Extract the reflection image of And a second threshold to emit,
First binarization of the difference image processed by a first threshold
Image processing characterized by extracting a reflection image from an eyeball as a feature point on a face using a processed image and a second binarized image obtained by processing the dark-captured image with a second threshold value. apparatus. 4. The feature point extracting means includes an i-th dark-photographed image, an (i-1) th bright-photographed image, and an i-
A first difference image obtained by performing a difference process on the i-th imaged bright-photographed image to the i-th imaged dark-photographed image using the second image-photographed dark-photographed image; A first difference image and a second difference image are obtained by obtaining a second difference image obtained by performing a difference process on the i−2nd photographed dark image from the i−1th photographed bright image. The image processing apparatus according to claim 3 , wherein the reflected image from the eyeball is extracted as a feature point on the face according to. 5. The feature point extracting means includes an i-th dark-photographed image, an i−1-th bright-photographed image, and an i-
Using the second dark-photographed image, the i-th bright-photographed image, the i-th dark-photographed image, and the i-2nd dark-photographed image are taken. The image processing apparatus according to claim 3 , wherein a difference image obtained by the difference processing is obtained, and the reflected image from the eyeball is extracted as a feature point on the face by the difference image. 6. A light input means for inputting an image of a predetermined area including the face of the detection target person, a coaxial illumination means for illuminating a predetermined area including the face of the detection target person coaxially with the light input means, and the detection. Non-coaxial illumination means for illuminating a predetermined area including the face of the target person non-coaxially with the light input means, and a feature for extracting feature points on the face from the photographed image of the detection target person output from the light input means An illumination light control means for controlling the presence or absence of light emission of the coaxial illumination means and the non-coaxial illumination means, a coaxial image output means for outputting an image of a detection target person by the coaxial illumination light, and a non-coaxial illumination. Non-coaxial image output means for outputting the image of the detection target person by light, comprising a difference image output means for outputting the difference image between the coaxial image and the non-coaxial image, the feature point extraction means, from the eyeball of the detection target person Anti A first threshold value for extracting the Izo, and a second threshold for the detection subject to extract reflection images from the mounting material for mounting, the difference image first
The first binarized image processed by the threshold of
An image processing apparatus characterized by extracting a reflection image from an eyeball as a feature point on a face by using a second binarized image obtained by processing a coaxial image with a second threshold value. 7. The feature point extracting means includes an i-th captured non-coaxial image, an (i-1) -th captured coaxial image, and an i-2.
A first difference image obtained by performing a difference process on the i-th imaged non-coaxial image from the i-1th imaged coaxial image using the second imaged non-coaxial image;
A second difference image obtained by performing difference processing on the i−2nd non-coaxial image obtained from the first taken coaxial image is obtained, and the eyeball is obtained by the first difference image and the second difference image. 7. The image processing apparatus according to claim 6 , wherein the reflection image from is extracted as a feature point on the face. 8. The feature point extracting means includes an i-th captured non-coaxial image, an (i-1) -th captured coaxial image, and an i-2.
The non-coaxial image captured at the i-th position is used to perform a difference process between the non-coaxial image captured at the i-th position and the non-coaxial image captured at the i-th second position using the non-coaxial image captured at the i-th position. Obtain the difference image obtained by, by the difference image,
The image processing apparatus according to claim 6 , wherein the reflected image from the eyeball is extracted as a feature point on the face. 9. The feature point extracting means includes a first binarized image obtained by processing the difference image with a first threshold value and a second binarized image obtained by processing a dark image or a non-coaxial image with a second threshold value.
By using the binarization image, claim 3 or a reflection image from the eyeball and extracting as feature points on the face
6. The image processing device according to item 6 . 10. The feature point extracting means includes a first difference processed image obtained by processing the first difference image with a first threshold value, and a second difference processed image.
The difference image by using the second differential processing images processed by the first threshold value, the image processing according to claim 4 or 7, wherein the reflected image and extracting a characteristic point on the face from the eye apparatus. 11. A light input means for inputting an image of a predetermined area including the face of the detection target person, an illuminating means for illuminating the predetermined area including the face of the detection target person, and the output from the light input means. In a feature point extracting means for extracting feature points on a face from a photographed image of a person to be detected, an illumination light control means for controlling intensity of illumination light or an input gain of the light input means, illumination light being strong or input The bright point image output means for outputting the image of the person to be detected with a large gain, the dark point image output means for outputting the image of the person to be detected with weak illumination light or a small input gain, and the feature point extracting means are provided. A first threshold value for extracting a reflection image from the eyeball of the detection target person, and a second threshold value for extracting a reflection image from a wearing object worn by the detection target person, The captured image according to the first threshold The reflected image from the eyeball is extracted as a feature point on the face using the first binarized image subjected to the image processing and the second binarized image obtained by processing the dark image with the second threshold value. An image processing device characterized by: 12. The feature point extracting means includes a binarized image of the i-th captured dark image, a binarized image of the (i-1) th captured bright image, and an (i-2) th image. Using the binarized image of the captured dark photographed image, the binarized image of the i-1th photographed bright photographed image is a binarized image of the i-th photographed dark photographed image. The first mask image obtained by the mask processing and the binarized image of the i-1th bright photographed image are binarized images of the i-2nd dark photographed image. The second mask image obtained by the mask processing is obtained, and the reflection image from the eyeball is extracted as a feature point on the face by the first mask image and the second mask image. 11. The image processing device according to item 11 . 13. The feature point extracting means includes a binarized image of a dark photographed image captured i-th, a binarized image of a bright photographed image captured i−1 th, and an i−2 th The binarized image of the i-th captured bright image is used to convert the binarized image of the captured dark-captured image into the binarized image of the i-th captured dark-captured image. And a binarized image of the i-2nd dark-photographed image, a mask image obtained by masking is obtained, and a reflection image from the eyeball is used as a feature point on the face by the mask image. The image processing device according to claim 11 , wherein the image processing device is extracted as. 14. A light input means for inputting an image of a predetermined area including the face of the detection target person, a coaxial illumination means for illuminating a predetermined area including the face of the detection target person coaxially with the light input means, and the detection. Non-coaxial illumination means for illuminating a predetermined area including the face of the target person non-coaxially with the light input means, and feature for extracting feature points on the face from the photographed image of the detection target person output from the light input means An illumination light control means for controlling the presence or absence of light emission of the coaxial illumination means and the non-coaxial illumination means, a coaxial image output means for outputting an image of a detection target person by the coaxial illumination light, and a non-coaxial illumination. A non-coaxial image output unit that outputs an image of a detection target person by light is provided, and the feature point extraction unit includes a first threshold value for extracting a reflection image from the eyeball of the detection target person, and the detection target person. Is installed And a second threshold for extracting a reflection image from the mounted object, the first binarized image obtained by processing the coaxial image with the first threshold, and the non-coaxial image by the second threshold. Using the processed second binarized image,
An image processing device characterized by extracting a reflection image from an eyeball as a feature point on a face. 15. The feature point extracting means includes a binarized image of the i-th captured non-coaxial image, a binarized image of the i−1th captured coaxial image, and an i−2nd captured image. Using the binarized image of the non-coaxial image that has been processed, the masking process of the binarized image of the i-1 th captured coaxial image with the binarized image of the i-th captured non-coaxial image The first mask image thus obtained and the binarized image of the i-1th captured coaxial image are masked with the i-2nd captured non-coaxial image of the binarized image. And a second mask image obtained by
15. The image processing apparatus according to claim 14, wherein the reflected image from the eyeball is extracted as a feature point on the face by the mask image of. 16. The feature point extracting means includes a binarized image of the i-th captured non-coaxial image, a binarized image of the i−1th captured coaxial image, and an i−2nd captured image. The binarized image of the i-th imaged coaxial image is used as the binarized image of the i-th imaged non-coaxial image and the binarized image of the i-th imaged non-coaxial image. i-The mask image obtained by performing the mask processing with the binarized image of the non-coaxial image captured secondly is obtained, and by the mask image,
The image processing apparatus according to claim 14 , wherein the reflection image from the eyeball is extracted as a feature point on the face. 17. The feature point extracting means degenerates an image to be subjected to a difference process or an image to be masked, to thereby reduce the image, according to any one of claims 1 to 8 , 12 , 13 , 15 , and 16 . Image processing device. 18. feature point extracting means, the image processing according to any one of claims 1 to 8,12,13,15,16, characterized in that enlarges an image of the image or mask processing difference processing apparatus. 19. feature point extracting means, the image processing apparatus according to claim 17 or 18, characterized in that to provide the directionality to the enlargement or reduction process corresponding to the position relationship between the coaxial illumination and non-coaxial illumination . 20. has a mounting object determining means determines the presence or absence of attachment of the detection subject, the feature point extracting means, depending on the determination result of the mounting object determining means, our first binarization image
And image processing for mounting using the second binarized image and
The image processing apparatus according to any one of switching the image processing for the non-wearing of using the first binarization image, or claims 1 and selects the processing result by the image processing 19 . 21. The image processing apparatus according to claim 20, wherein the mounted object determining means detects the presence or absence of the mounted object based on the number of reflection images or the maximum brightness of the illumination light. 22. The feature point extracting means is used when there is an attached object.
Claim 20 or, characterized in that to strengthen the illumination light to the
21. The image processing device according to item 21 . 23. The feature point extracting means is used when there is an attached object.
Is characterized by increasing the input gain of the optical input means
The image processing device according to claim 20 or 21. 24. The feature point extracting means comprises a first binarization process.
Image processing for wearing using the image and the second binarized image and image processing for non-wearing using the first binarized image
And a physical, claims 1 and selects either one of the image processing result by the image processing result 1
9. The image processing device according to any one of 9 .