JP3185522B2 - Gaze direction measuring device for vehicles - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a gaze direction measuring device for a vehicle, which measures the gaze direction of a vehicle driver remotely without contact.

[0002]

2. Description of the Related Art A gaze direction measuring device of this kind is used as a non-contact man-machine interface for a vehicle, for example, to display visual information in a direction of interest of a vehicle driver or to specify a specific information in accordance with a gaze direction. It is expected to be used for various purposes such as operating a switch. Conventionally, as a device for measuring the line of sight of a vehicle driver necessary for configuring such an interface, a device that takes in and measures the position of a corneal reflection image of an eyeball as image information has been proposed. The corneal reflection image is an image generated by the irradiation light on the eyeball being reflected and refracted by each surface of the optical system constituting the eyeball, and is also called a Purkinje image.

[0005] However, when the vehicle driver wears spectacles, as shown in FIG. 15, illumination light is reflected from the lens surface, the back surface, or the frame. Furthermore, if these reflection images W1, W2, W3, etc. have a certain size, it becomes difficult to distinguish between the retinal reflection image M and the corneal reflection image K. The case where the reflection image has a size means that the reflection image has a size due to, for example, the effect of blurring in the optical system and the effect of the blooming in the imaging system. Therefore, it is necessary that the eye gaze direction can be accurately measured even by a spectacle wearer by discriminating between these and the reflected image of the eyeball.

In order to measure the gaze direction of a spectacle wearer, Japanese Patent Laid-Open No.
There are devices disclosed in Japanese Patent Application Laid-Open No. 138431 and Japanese Patent Application Laid-Open No. 4-138432. These measures the gaze direction of the photographer looking through the viewfinder of the camera, and the intervals between the reflection images when the eyeballs are simultaneously illuminated using two light sources, as shown in FIG. Utilizing the fact that the reflection images W and W of the spectacle lens become large and the corneal reflection images K and K become small, the reflection image by the spectacles is identified.

[0005]

However, if the subject is a vehicle driver who can freely displace his or her head in free space without any restriction, the reflected light from not only the spectacle lens but also the spectacle frame. Is also mixed in as image information, and it is difficult to identify the apparatus using only the line of sight of the photographer looking through the viewfinder of the camera as described above. That is, the reflected light from the spectacle lens is observed on the same line almost parallel to the line connecting the two light sources, as shown in FIG. 16, but the reflected light from the spectacle frame is at a random position. appear. Therefore,
In particular, if a reflection image from the spectacle frame appears near the corneal reflection image to be paired for measurement, it becomes difficult to determine which reflection image and which reflection image are paired, and it is necessary to accurately determine the corneal reflection image Can not. For this reason, the conventional gaze direction measuring device for a camera photographer has a problem that it cannot be effectively applied to a vehicle driver.

Further, as shown in FIG. 15, when a reflection image W1 from a spectacle lens has a certain area and appears near a retinal reflection image M, a retinal reflection image which is a basis for determining one corneal reflection image is specified. Will also be difficult. Accordingly, the present invention has been made in view of the above-described conventional problems, and therefore, it is possible to reliably extract a corneal reflection image even when a vehicle driver wears eyeglasses and accurately measure a gaze direction. The purpose is to provide.

[0007]

SUMMARY OF THE INVENTION For this reason, the present invention has been described with reference to FIG.
As shown in FIG. 7, a plurality of illuminations 62a and 62b for irradiating the driver's eyeball from different directions, and an image pickup device 63 for imaging a reflection image from the eyeball by the plurality of illuminations are provided. , A retinal reflection image extracting means 64 for extracting a retinal reflection image from image data and calculating a pupil center, and a corneal reflection image extracting means 65 for extracting a corneal reflection image from the image data. And a gaze direction calculating means 66 for calculating the gaze direction of the driver from the extracted corneal reflection image and the center of the pupil.
Image data having a color component specific to the retinal reflection image and image data having other color components are obtained as data, and the retinal reflection image extraction means 64 obtains an image having a color component specific to the retinal reflection image. A retinal reflection image is extracted by distinguishing from other reflection images using data, and the corneal reflection image extracting means 65 is configured to extract a corneal reflection image based on the position of the retinal reflection image. And

[0008]

According to the plurality of illuminations, a plurality of image data in which the position of the reflection image is shifted for each illumination can be obtained. Retinal reflection image extraction means 6
4. The corneal reflection image extraction means 65 extracts a retinal reflection image and a corneal reflection image from a candidate area obtained by calculating a difference between a plurality of image data and other image processing. Here, in the case of a retinal reflection image. In particular, a specific color component, such as red, which is not seen in the reflection image from the surface of the spectacle lens is included. The image input means 61 obtains image data d1 having a color component specific to the retinal reflection image and image data d2 having other color components, and the retinal reflection image extracting means 64 obtains the image data d1. Image data d having the above-mentioned color component peculiar to a retinal reflection image
Since 1 is used, even when the vehicle driver wears spectacles, the retinal reflection image can be reliably identified by identifying from other reflection images, and the retinal reflection image is extracted with high accuracy. Then, the corneal reflection image extracting means 65 specifies the corneal reflection image with reference to the extracted retinal reflection image, and is therefore easily extracted.

[0009]

DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described below with reference to the drawings.
FIG. 2 shows a configuration of an embodiment in which the present invention is applied to a line-of-sight switch system of a vehicle, and FIG. 3 shows a layout on a vehicle. In order to image the driver's eyeball 13, CC on the instrument panel
A color camera 4 using D or the like is installed. The eyeball position of the vehicle driver is, for example, JIS (Japanese Industrial Standard) D002.
As shown in FIG. 1, since the color camera 4 exists in a limited area statistically indicated as the eye range of the driver of the automobile, the imaging range of the color camera 4 can be limited to that area, and is installed relatively close to the vehicle driver. it can.

[0010] A first divergent illumination 1 of a coaxial system attached to the front surface of the lens of the color camera 4 so that the illuminating direction coincides with the optical axis of the camera is provided toward the eyeball of the driver. . In addition, a non-coaxial second divergent illumination 2 whose illumination direction does not coincide with the optical axis of the color camera is provided at a predetermined position away from the color camera 4 and whose relative relationship is set in advance, also toward the eyeball. Have been. The first and second divergent lights 1 and 2 have a white light source of the same specification, are installed on a pillar, a center console, or the like, and are arranged so as not to be directly in the driver's field of view while watching the foreground during driving. You. The first divergent light 1 and the second divergent light 2 are connected to an illumination light emission control unit 11 for controlling their light emission.

The image output from the color camera 4 is A / D
The R, G, and B signals are independently A / D converted by the converter 5 as digital image data, and stored in the image memory 6. The image memory 6 is connected to a retinal reflection image extraction unit 7 and a corneal reflection image extraction unit 8, and both of these extraction units are further connected to a gaze direction calculation unit 10. The above-mentioned illumination light emission control unit 1
1. An overall control unit 12 including an A / D converter 5 and controlling the operation of the entire apparatus is provided.

The retinal reflection image extraction unit 7 extracts a retinal reflection image from the R (red) component in the image data stored in the image memory 6, and further calculates and extracts the pupil center. In addition, the corneal reflection image extraction unit 8 specifies the position of the corneal reflection image from the image data. Then, based on the corneal reflection image extracted by the corneal reflection image extraction unit 8 and the pupil center extracted by the retinal reflection image extraction unit 7,
The gaze direction of the driver is calculated and specified as a straight line direction connecting the pupil center and the corneal sphere center.

The gaze direction calculation unit 10 includes a gaze stop determination unit 2
3, a controller switching unit 27 is connected. The controller switching unit includes an audio controller 28, an air conditioner controller 29, a radio controller 30, an ASCD (constant speed traveling device) controller 31
Are connected, and a HUD (head-up display) display control unit 24 and a steering switch 22 are connected. The line-of-sight stop determination unit 23 uses the HUD
The display control unit 24 determines the gaze stop position in the gaze switch area 26 displayed on the front windshield 41, and outputs control information to the controller switching unit 27.

When the main switch 21 is pressed, the system is turned on, and the gaze direction calculator 10 outputs the gaze direction of the driver. The line-of-sight stop determination unit 23 outputs the line-of-sight direction of the driver to the line-of-sight switch area 2 for displaying each operation target item (area) such as an air conditioner and an audio.
It is determined which area of No. 6 is pointing.

When the area on the line-of-sight switch area 26 where the driver is gazing is determined, the controller switching unit 27 selects an air conditioner controller, an audio controller, and the like corresponding to the determined area. Perform a switch. Then, the HUD display control unit 24 displays the content on the HUD display unit 25 based on the signal of each controller output through the controller switching unit 27. Then, the user operates the steering switch 22
A predetermined control target is controlled by the user input.

As shown in FIG. 3, the main switch 2
1 and the steering switch 22 are attached to the steering 42. An HUD display 25 and a line-of-sight switch area 26 are set on the windshield 41. The line-of-sight switch area 26 further displays the names of various controllers in HUD, as shown in detail in FIG. FIG. 3B shows details of the steering switch 22.

Next, the operation will be described. FIG. 5 shows a flow of processing in the eye-gaze switch system. First, in step 101, when a measurement start signal is output from the overall control unit 12, based on this, a step 102 is executed.
Then, a trigger signal is issued from the illumination light emission control unit 11,
The first divergent light 1 is turned on, and the face of the vehicle driver is illuminated. In step 103, the image of the illuminated face area is captured by the color camera 4, and the image information is A / D-converted by the A / D converter 5 for each of the R, G, and B components independently. Digital image data DR1 (x, y), DG1 (x, y), D
It is stored in the image memory 6 as B1 (x, y).

Subsequently, in step 104, the second divergent illumination 2 is turned on by the trigger signal from the illumination light emission control unit 11, and the face of the vehicle driver is illuminated. In step 105, the image of the illuminated face area is captured by the color camera 4, and the A / D converter 5 converts the R, G, and B components of the image information independently from each other in the same manner as described above. And digital image data DR2
They are stored in the image memory 6 as (x, y), DG2 (x, y), and DB2 (x, y). Step 102 to above
105 constitutes the image input means of the invention.

Thereafter, in step 106, the R (red) component image data DR
From 1 (x, y) and DR2 (x, y), the position of the retinal reflection image of the vehicle driver is measured. Here, the retinal reflection image is
When white light enters the eyeball, it returns as red reflected light. This is due to the reflection of blood through the retina, for example, as is known as the "red-eye effect" when using a strobe in photography. On the other hand, with respect to the reflection image by the spectacles, generally used spectacles are often made of glass or plastic, and in the visible light region, the reflected light has little wavelength selectivity. Therefore, the reflected image returned as red light when white light is irradiated is only a retinal reflected image. For this reason, R of the reflection image in the image
By searching for a region where the luminance level of the (red) component is high, a retinal reflection image can be separated and extracted from other images.

FIG. 6 shows details of the position measurement of the retinal reflection image in step 106. First, among the input image data by the first and second divergent illuminations 1 and 2 as shown in FIGS. 7A and 7B, image data DR1 (x, R) of the R component
y) and DR2 (x, y) are read out from the image memory 6, and in step 201, a difference operation between these red component image data DR1 (x, y) and DR2 (x, y) is performed. Image data DR3 as shown in FIG.
(X, y) is required. In step 202, the image data DR3 (x, y) is binarized with a predetermined threshold value Th1 (Th1> 0) set in advance, and FIG.
Image data R4 (x, y) from which noise has been removed
It is said. The red region originally present in the image irrespective of the presence or absence of illumination lighting is removed in steps 201 and 202.

Subsequently, in step 203, a labeling process for numbering each area in the image data DR4 (x, y) is performed. Then, in step 204, among the labeled regions, the region having the largest area is extracted as a retinal reflection image. Thereafter, in step 205, the position of the center of gravity (Xgi, Ygi) of the retinal reflection image is calculated. Step 201-2
Step 106 consisting of 05 constitutes the retinal reflection image extracting means of the present invention.

Returning to FIG. 5, in order to specify the corneal reflection image position based on the retinal reflection image barycenter position obtained above, in step 107, a search is performed to search for a corneal reflection image in a region near the retinal reflection image. An area T is set. The search area T is
As shown in FIG. 8, the range is set to p × q around the center of gravity of the retinal reflection image. If the position of the retinal reflection image can be specified, the position of the corneal reflection image by the first divergent illumination 1 and the position of the corneal reflection image by the second divergent illumination 2 are limited to the vicinity of the retinal reflection image. Because it will be.

Thereafter, in step 108, the corneal reflection image extraction unit 8 extracts a corneal reflection image. Here, since white light is used for illumination, the corneal reflection image also becomes a white bright spot. When identifying a true corneal reflection image from the observed reflected light, the reflection and transmission by the spectacle lens and the reflection on the corneal surface have little wavelength selectivity in the visible light region. Regardless of which color component is used for the embedding, there is no great difference in the calculation of the corneal reflection image position. However, if the processing is performed using the R component, the retinal reflection image may overlap with the corneal reflection image to be extracted. Therefore, the G or B component may be used.

The corneal reflection image can be classified into first to fourth Purkinje images depending on which surface of the eyeball optical system is generated. Here, the brightest corneal reflection image is generated on the corneal surface. The first Purkinje image is targeted. When the vehicle driver wears spectacles, in addition to the corneal reflection image, reflected light from the lenses and frames of the spectacles is simultaneously observed. They are,
It is observed as a bright area like the corneal reflection image.

FIG. 9 shows a detailed flow of corneal reflection image extraction in such an environment. In step 301,
First, a difference calculation of the image data DG1 (x, y) and DG2 (x, y) as shown in FIGS. The image data DG3 (x, y) after the processing is, as shown in (c), a spectacle region and a cheek region having substantially the same gray level between the images DG1 (x, y) and DG2 (x, y), and furthermore, a white eye. Regions and the like are removed, and regions such as a corneal reflection image, a spectacle lens reflection image, and a spectacle frame reflection image having greatly different grayscale values are emphasized. Here, the positive value and the negative value are represented as they are, and the shaded portion in the figure indicates the negative value.

In step 302, the image data DG3
The (x, y) p × q area is binarized by a preset threshold value Th2 (Th2> 0) to obtain image data DG4 (x, y) as shown in (d). Step 303
In step, after the binarized and extracted region is subjected to labeling processing, in step 304, area threshold processing is performed. Since the corneal reflection image is a bright spot having a small area of several dots or less, in this processing, the area of each labeled area is calculated, and by excluding the area having a certain area Sth or more, a small bright spot area is obtained. Is extracted.

In the step 305, the center of gravity of the candidate area i is calculated using the area thus extracted as the candidate area i of the corneal reflection image. In the next step 306,
The search area U of the reflection image by the second divergent illumination 2 is set. That is, if the position of the image reflected by the first divergent illumination 1 is specified, the position where the image reflected by the second divergent illumination 2 is generated is limited to the vicinity thereof. Therefore, a small area of m × n pixels is set including the positions of the centers of gravity calculated in the previous step.

The search area U is determined according to the setting state of the first and second divergent lights 1 and 2. For example, FIG.
(A), in the arrangement in which the second divergent light 2 is on the right side of the first divergent light 1 as viewed from the driver,
As shown in FIG. 2B, the corneal reflection image A by the second divergent illumination 2 is smaller than the corneal reflection image A1 by the first divergent illumination 1.
2 is located to the left on the image. As shown in FIG. 16, the corresponding reflected image appears on a line parallel to the line connecting the first divergent illumination 1 and the second divergent illumination 2. Therefore, as shown in FIG. 11C, a range extending to the left from the center of gravity position V of the candidate area i may be set as the search area U.

In the next step 307, a search for a reflection image by the second divergent illumination 2 is performed in the search area for each candidate area i as described above. Here, the image data DG3 (x, y) of each search area is set to a threshold value Th3.
Binary processing is performed at (Th3 <0), and it is searched whether there is a pixel that satisfies DG3 (x, y) <Th3.

In step 308, a corneal reflection image is specified based on the result of the binarization processing. (1) When a pixel satisfying DG3 (x, y) <Th3 is not extracted in the search area. At this time, since the bright spot area corresponding to the reflection image by the second divergent illumination corresponding to the candidate area i does not exist in the search area, that is, the candidate area i is not a corneal reflection image. (2) A case where only one region composed of pixels satisfying DG3 (x, y) <Th3 is extracted in the search region. At this time, the barycentric coordinates (Xgi, Y
gi), which is stored in memory.

(3) DG3 (x, y) <T in the search area
A case where two or more regions composed of pixels satisfying h3 are extracted. At this time, if the image is a true corneal reflection image, since the image is parallel to the corneal reflection image by the first divergent illumination 1, the difference in the y coordinate from the candidate region i is minimized. Therefore, among the extracted regions, the one having the smallest difference in the y coordinate is determined as the true corneal reflection image, and the barycentric coordinates (Xgi, Ygi) are obtained. Step 108 comprising the above steps 301 to 308
Constitute a corneal reflection image extracting means.

Returning to FIG. 5, after that, step 109
Then, the gaze direction calculating unit 10 calculates the gaze direction of the driver based on the barycentric coordinates of the two corneal reflection images and the barycentric coordinates of the pupil region obtained so far. FIG. 12 is an explanatory diagram illustrating the principle of calculating the line-of-sight direction in the line-of-sight direction calculation unit. In the case of a coaxial system in which the illumination and the optical axis of the camera coincide, the straight line connecting the camera focal point F1 (= the position of the illumination light source L1) and the corneal reflection image β on the camera C (on the CCD surface) is located at the center O of the corneal sphere. pass. Therefore, the corneal sphere center O is straight line F
It is on 1-β. The actual pupil center Q is on a straight line F1-α connecting the camera focal point (illumination position) and the pupil center α on the CCD image.

The focal length of the camera 4 is f and the focal position is F1.
(0,0,0), and considering the X-, Y-, and Z-axis world coordinate system F1-XYZ, the light source L2 for non-coaxial illumination is F2.
In (a, b, c), the illumination light is specularly reflected on the corneal surface. The specular reflection point P is the light source L1 on the CCD surface.
Straight line F1-- connecting the corneal reflection image .delta.
on δ. Then, if the temporary regular reflection point P is placed on the straight line F1-δ and the spherical surface J is drawn with a radius of 7.8 mm, which is the radius of the corneal sphere, the center O of the corneal sphere is defined as the intersection of the spherical surface J and the straight line F1-β.
The three-dimensional position of is determined. The distance between the corneal sphere center O and the pupil center Q is about 4.2 mm, and the corneal sphere center O determined above
A spherical surface K having a radius of 4.2 mm centered on a straight line F1 -α
The three-dimensional position of the pupil center Q is determined as the intersection of. As a result, a line connecting the points O and Q is determined as the line-of-sight direction. Step 109 constitutes the line-of-sight direction calculating means.

In step 110, the gaze stop determination unit 23 determines which area (control item) of the gaze switch area 26 the gaze direction calculated by the gaze direction calculation unit 10 as described above looks at. , Calculate the time. Then, at step 111, a stop determination is made.
This is performed depending on whether or not the driver views the same area continuously for a certain number of times in the processing cycle. If one cycle of gaze direction calculation requires 400 msec,
For example, it is determined that the user is gazing at the same area for two consecutive times with a gazing time of about 0.8 sec.

Here, for example, an air conditioner area (A / C) of the line of sight switch area 26 shown in FIG.
When it is determined that the user is gazing at, the process proceeds to step 112, where the control of the object, that is, the air conditioner, is performed. If there is no stop for a predetermined time or more, the process returns to step 102 and the measurement of the gaze direction is repeated. In step 112, the controller switching unit 27 switches to, for example, air conditioner control, and the HUD display control unit 24 displays the current air conditioner set temperature on the HUD display unit 25. Note that when the process proceeds to step 112, the measurement of the line-of-sight direction is stopped.

When the air conditioner set temperature is displayed on the HUD display section 25, the up / down button of the steering switch 22 functions so as to adjust the set temperature up and down. The driver operates the up / down button of the steering switch 22 while watching the HUD display section 25 to set a desired set temperature. According to the set information,
Predetermined temperature control is performed by the air conditioner controller 29. If the up / down button is not operated for a certain period of time (for example, 5 seconds), it is determined that the operation has been completed, and all the switch functions are stopped. The driver needs some operation again, and when the main switch 21 is pressed, the above processing is repeated again.

The embodiment is configured as described above, and uses the fact that the retinal reflection image has a red component when the spectacle reflection image and the corneal reflection image are separated in measuring the driver's gaze direction.
Focusing on the red component in the image, the retinal reflection image is extracted, and the corneal reflection image is specified based on the retinal reflection image position. Therefore, even if the system operator wears glasses, the gaze direction can be accurately determined. Can be requested. As described above, the color camera can be installed relatively close to the driver of the vehicle and the range of imaging can be limited, so that the driver's eyeball can be enlarged and photographed to measure the gaze direction over a wide range with relatively high accuracy. It is possible.

In the embodiment, white light is used as the light source. However, the present invention is not limited to this. If the light source can be distinguished from the superimposed red component by retinal reflection, the light source has a blue or green component. Can also be used. Further, in the above embodiment, the color camera 4 inputs all three colors of R, G, and B as the image data of each color component, and outputs the data of each color component. As shown, the incident light reflected from the eyeball portion is guided to the dichroic mirror 51 and the like, and only the wavelength including the red component of the retinal reflection image is reflected and input to the camera or the imaging device 52, and the other transmission components are reflected. To another camera or image sensor 5
3 so as to obtain image data from different systems. This allows a normal black and white camera to be used instead of an expensive color camera.

Alternatively, as shown in FIG. 14, a camera or an image pickup device 52 using a beam splitter 55 for distributing the total amount of incident light to a predetermined ratio of reflection and transmission.
53, a filter 58 that transmits only a wavelength including a red component of the retinal reflection image is installed in front of one of the cameras or the imaging device 52. This also makes it possible to obtain image data of a red component separated from other color components using an inexpensive monochrome camera or the like.

[0040]

As described above, the present invention extracts a retinal reflection image and a corneal reflection image from image data of an eyeball, and calculates a gaze direction of a driver from a pupil center and a corneal reflection image. In the measurement device, utilizing the fact that the retinal reflection image has a specific color component of red, the extraction of the retinal reflection image is performed using image data having the color component specific to the retinal reflection image, and Since the corneal reflection image is to be extracted based on the extracted retinal reflection image position, even if the driver is wearing spectacles, the retinal reflection image and the corneal reflection image are reliably obtained from the reflection images of the spectacle lenses and frames. This has the effect that the gaze direction measurement can be performed easily and can be performed with high accuracy.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of the present invention.

FIG. 2 is a diagram showing a configuration of an example of the present invention.

FIG. 3 is a diagram showing a vehicle-mounted layout according to the embodiment.

FIG. 4 is a diagram showing details of a line-of-sight switch area and a steering switch.

FIG. 5 is a main flowchart showing the flow of processing in the embodiment.

FIG. 6 is a flowchart illustrating details of position measurement of a retinal reflection image.

FIG. 7 is an explanatory diagram showing a flow of processing of image data.

FIG. 8 is a diagram showing a search area.

FIG. 9 is a flowchart showing details of corneal reflection image extraction.

FIG. 10 is an explanatory diagram showing a flow of processing of image data.

FIG. 11 is a diagram showing a setting procedure of a search area.

FIG. 12 is an explanatory diagram showing the principle of calculating the line-of-sight direction.

FIG. 13 is a diagram showing another example of inputting image data of each color component.

FIG. 14 is a diagram illustrating another example of inputting image data of each color component.

FIG. 15 is a diagram illustrating an example of a reflection image when wearing glasses.

FIG. 16 is a diagram showing an interval between a reflection image of a spectacle lens and a corneal reflection image by two illuminations.

[Explanation of symbols]

 REFERENCE SIGNS LIST 1 first divergent illumination 2 second divergent illumination 4 color camera 5 A / D converter 6 image memory 7 retinal reflection image extraction unit 8 corneal reflection image extraction unit 10 gaze direction calculation unit 11 illumination emission control unit 12 overall control unit DESCRIPTION OF SYMBOLS 13 Eyeball 21 Main switch 22 Steering switch 23 Gaze stop determination part 24 HUD display control part 25 HUD display part 26 Gaze switch area 27 Controller switching part 28 Audio controller 29 Air conditioner controller 30 Radio controller 31 ASCD controller 41 Front windshield 42 Steering Reference Signs List 51 dichroic mirror 52, 53 camera or image sensor 55 beam splitter 58 filter T, U search area

──────────────────────────────────────────────────続 き Continued on the front page (58) Field surveyed (Int. Cl. ⁷ , DB name) G01B 11/00 A61B 3/10-3/113

Claims (7)

(57) [Claims] 1. An image input device comprising: a plurality of illuminations for illuminating a driver's eyeball from different directions; and an image pickup device for capturing an image reflected from the eyeball by the plurality of illuminations to obtain image data for each illumination. Means, a retinal reflection image extracting means for extracting a retinal reflection image from the image data and calculating a pupil center, a corneal reflection image extracting means for extracting a corneal reflection image from the image data, and Gaze direction calculating means for calculating the gaze direction of the driver from the corneal reflection image and the center of the pupil, wherein the image input means includes image data having a color component specific to a retinal reflection image as the image data. And retinal reflection image extracting means,
The retinal reflection image is extracted from other reflection images by using image data having a color component unique to the retinal reflection image, and the corneal reflection image extracting means extracts the retinal reflection image based on the position of the retinal reflection image. A gaze direction measuring device for a vehicle, which extracts a corneal reflection image. 2. The gaze direction measuring device for a vehicle according to claim 1, wherein a color component specific to the retinal reflection image is red. 3. The corneal reflection image extracting means extracts a corneal reflection image by searching within a predetermined first search area set around a center of gravity of the retinal reflection image. The gaze direction measuring device for a vehicle according to claim 1 or 2, wherein: 4. The corneal reflection image extracting means,
A bright spot area having a predetermined area or less in the search area is set as a candidate area of the first corneal reflection image by the first illumination among the plurality of illuminations, and a position corresponding to the center of gravity of the candidate area is determined based on the position of the first illumination. When there is a second bright spot area corresponding to a second corneal reflection image by the second illumination within a second search area set to extend in a predetermined direction determined according to the position of the second illumination ,
The gaze direction measuring device for a vehicle according to claim 3, wherein the candidate area and the second bright spot area are corneal reflection images. 5. The gaze direction measurement for a vehicle according to claim 1, wherein the image pickup device of the image input unit is a color camera that outputs image data for each of the color components. apparatus. 6. The imaging device of the image input means separates incident light reflected from an eyeball into a wavelength component including a red component of a retinal reflection image and other color components by a dichroic mirror, and detects each color component. 5. The gaze direction measuring device for a vehicle according to claim 1, wherein the gaze direction measuring device outputs the image data for each of the color components by inputting the image data to a camera or an image pickup device. 7. An image pickup device of the image input means, wherein a beam splitter separates incident light reflected from an eyeball into two systems, one of which passes through a red transmission filter and detects each color component. 5. The gaze direction measuring device for a vehicle according to claim 1, wherein the gaze direction measuring device outputs the image data for each color component by inputting the image data to an image sensor.