JPH07229714A - Sight direction measuring equipment for vehicle - Google Patents

Abstract

PURPOSE:To make precise sight direction measurement even when a driver wears glasses. CONSTITUTION:In such a sight switching system that the steadily gazed range of a driver in a sight switch area displayed in a HUD (Head-Up Display) is judged by the sight fixing judgement part 23 to perform various controls via the control switching part 27 depending on the steadily gazed range, the image of eyeballs 13 by the first and second divergent lights 1, 2 to find a sight direction is picked up by a color camera 4. Image data output for every RGB and a retinal reflected image is obtained from R (red) image data in a retinal reflected image extracting part 7 to find the centers of pupils. A cornea reflected image is extracted from other color image data in a cornea reflected image extracting part 8, based on the position of the retinal reflected image, to find a sight direction in a sight direction calculating part 10 from the centers of pupils and the cornea reflected image. As the retinal reflected image is surely and easily identified by a red component inherent to the retinal reflected image, the sight direction can be precisely found.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a visual line direction measuring device for a vehicle for remotely measuring the visual line direction of a vehicle driver in a non-contact manner.

[0002]

2. Description of the Related Art A gaze direction measuring device of this type is used as a non-contact man-machine interface for a vehicle, for example, to display visual information in a direction in which a driver of a vehicle is paying attention or to specify a gaze direction according to a gaze direction. It is expected to be used for various purposes such as operating a switch. As a device for measuring the line-of-sight direction of a vehicle driver, which is necessary to configure such an interface, a device for measuring the position of a corneal reflection image of an eyeball as image information has been conventionally proposed. The corneal reflection image is an image generated by the irradiation light to the eyeball being reflected and refracted by each surface of the optical system forming the eyeball, and is also called a Purkinje image.

However, when the vehicle driver wears the spectacles, the illumination light is reflected from the front surface or the back surface of the lens or the frame, as shown in FIG. Further, if these reflection images W1, W2, W3, etc. have a certain size, it becomes difficult to distinguish them from the retina reflection image M and the corneal reflection image K. When the reflected image has a size, for example, the reflected image has a size due to the influence of blurring in the optical system and the influence of blooming in the imaging system. Therefore, it is necessary that even a spectacle wearer can accurately measure the line-of-sight direction by distinguishing these from the reflected image of the eyeball.

Therefore, as a conventional device for measuring the line-of-sight direction of an eyeglass wearer, for example, Japanese Patent Laid-Open No.
There are devices disclosed in Japanese Patent No. 138431 and Japanese Patent Application Laid-Open No. 4-138432. These are for measuring the line-of-sight direction of a photographer looking through the viewfinder of the camera, and when the two light sources are used to irradiate the eyeballs at the same time, the distance between the reflected images is as shown in FIG. By utilizing the fact that the reflected images W, W of the spectacle lens become large and the corneal reflected images K, K become small, the reflected image by the spectacles is identified.

[0005]

However, when the subject is a vehicle driver who can freely displace his / her head in free space without any restraint, the reflected light from not only the spectacle lens but also the spectacle frame is used. Is also included as image information, and it is difficult to identify with a device that measures only the line of sight of the photographer looking into the viewfinder of the camera as described above. That is, the reflected light from the spectacle lens is observed on the same line substantially parallel to the line connecting the two light sources, as shown in FIG. 16, but the reflected light from the spectacle frame is at random positions. Occur. Therefore,
Especially, when a reflection image by the spectacle frame appears in the vicinity of the corneal reflection image to be paired for measurement, it becomes difficult to determine which reflection image and which reflection image become the pair, and the corneal reflection image can be accurately determined. I can't. Therefore, the conventional line-of-sight direction measuring device for a camera photographer cannot be effectively applied to a vehicle driver.

Further, as shown in FIG. 15, when the reflection image W1 from the spectacle lens has a certain area and appears in the vicinity of the retinal reflection image M, the retinal reflection image serving as a basis for judging one corneal reflection image is specified. Will also be difficult. Therefore, in view of the above-mentioned conventional problems, the present invention can accurately extract the corneal reflection image even if the vehicle driver wears the eyeglasses, and can measure the gaze direction with high accuracy. The purpose is to provide.

[0007]

Therefore, the present invention is based on FIG.
As shown in FIG. 2, image data for each illumination is provided by including a plurality of illuminations 62a and 62b that illuminate the driver's eyeballs from different directions and an image pickup device 63 that captures a reflected image from the eyeballs by the plurality of illuminations. For obtaining the retina reflection image from the image data, the retina reflection image extracting means 64 for calculating the center of the pupil, and the corneal reflection image extracting means 65 for extracting the corneal reflection image from the image data. And a line-of-sight direction calculating unit 66 for calculating the line-of-sight direction of the driver from the extracted corneal reflection image and the center of the pupil, and the image input unit 61 includes the image data unit.
As image data, image data having a color component peculiar to the retina reflection image and image data having other color components are obtained, and the retina reflection image extracting means 64 is an image having a color component peculiar to the retina reflection image. The retina reflection image is extracted by distinguishing it from other reflection images using data, and the corneal reflection image extracting means 65 is configured to extract the corneal reflection image based on the position of the retina reflection image. And

[0008]

With a plurality of illuminations, a plurality of image data in which the position of the reflected image is shifted for each illumination can be obtained. Retinal reflection image extraction means 6
4. The corneal reflection image extraction means 65 extracts the retinal reflection image and the corneal reflection image from the candidate regions obtained by the difference calculation of plural image data and other image processing. Here, in the case of the retina reflection image. In particular, it contains a unique color component such as red that is not seen in the reflection image from the surface of the spectacle lens. The image input means 61 is adapted to obtain the image data d1 having a color component peculiar to the retina reflection image and the image data d2 having other color components. Image data d having the above-mentioned color components peculiar to the retina reflection image
Since 1 is used, the retinal reflection image can be reliably identified by distinguishing it from other reflection images even when the vehicle driver wears eyeglasses, and the retinal reflection image can be extracted with high accuracy. Then, the corneal reflection image extraction means 65 specifies the corneal reflection image with reference to the extracted retina reflection image, and therefore is easily extracted.

[0009]

DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described below with reference to the drawings.
FIG. 2 shows the configuration of an embodiment in which the present invention is applied to a visual line switching system of a vehicle, and FIG. 3 shows its vehicle-mounted layout. CC on the instrument panel to capture the driver's eye 13
A color camera 4 using D or the like is installed. The eyeball position of the vehicle driver is, for example, JIS (Japanese Industrial Standard) D002.
As shown in FIG. 1, since it exists in a limited area which is statistically shown as the eye range of the driver of the automobile, the imaging range of the color camera 4 can be limited to that area, and the color camera 4 can be installed relatively close to the vehicle driver. it can.

Then, the first divergent illumination 1 of the coaxial system, which is attached to the front surface of the lens of the color camera 4 so that the optical axis of the camera coincides with the irradiation direction, is provided toward the eyeball portion of the driver. . Further, a second non-coaxial system divergent illumination 2 whose irradiation direction does not match the optical axis of the color camera is provided at a predetermined position where the relative relationship is preset apart from the color camera 4 toward the eyeball portion. Has been. The first and second divergent lights 1 and 2 have a white light source of the same specifications and are installed in pillars, center consoles, etc., and are arranged so that they do not come directly into the driver's field of vision when gazing at the foreground during driving. It The first divergent illumination 1 and the second divergent illumination 2 are connected to an illumination light emission control unit 11 that controls their light emission.

The image pickup output from the color camera 4 is A / D.
The converter 5 independently A / D-converts the R, G, and B signals as digital image data and stores the digital image data in the image memory 6. The image memory 6 is connected to a retina reflection image extraction unit 7 and a corneal reflection image extraction unit 8, both of which are further connected to a line-of-sight direction calculation unit 10. The illumination light emission control unit 1 described above
1. An overall control unit 12 including the A / D converter 5 and controlling the operation of the entire apparatus is provided.

The retina reflection image extraction unit 7 extracts the retina reflection image from the R (red) component in the image data stored in the image memory 6, and further calculates and extracts the center of the pupil. The corneal reflection image extraction unit 8 specifies the position of the corneal reflection image from the image data. Then, in the gaze direction calculation unit 10, based on the corneal reflection image extracted by the corneal reflection image extraction unit 8 and the pupil center extracted by the retina reflection image extraction unit 7,
The line-of-sight direction of the driver is calculated and specified as the straight line direction connecting the center of the pupil and the center of the cornea.

The line-of-sight direction determining unit 2 is included in the line-of-sight direction calculating unit 10.
A controller switching unit 27 is connected via 3. The controller switching unit includes an audio controller 28, an air conditioner controller 29, a radio controller 30, and an ASCD (constant speed traveling device) controller 31.
Etc. are connected, and a HUD (head-up display) display control unit 24 and a steering switch 22 are connected. The line-of-sight stop determination unit 23 uses the HUD
The display control unit 24 determines the line-of-sight stop position within the line-of-sight switch area 26 displayed on the front windshield 41, and outputs control information to the controller switching unit 27.

Now, when the main switch 21 is pressed, the system is turned on, and the line-of-sight direction calculation unit 10 outputs the line-of-sight direction of the driver. The line-of-sight stop determination unit 23 displays a line-of-sight switch area 2 in which the output line-of-sight of the driver displays each operation target item (area) such as an air conditioner and an audio device.
Judge which area of 6 is pointing.

When the area on the line-of-sight switch area 26 to which the driver is gazing is determined, the controller switching unit 27 selects an air conditioner controller, an audio controller, etc. corresponding to the determined area. Switch over. Then, the HUD display control unit 24 displays the content on the HUD display unit 25 based on the signal of each controller output through the controller switching unit 27. Then, the user from the steering switch 22
A predetermined controlled object is controlled by the input.

As shown in FIG. 3, the main switch 2
1 and the steering switch 22 are attached to the steering 42. A HUD display section 25 and a line-of-sight switch area 26 are set on the windshield 41. Further, in the line-of-sight switch area 26, the names of various controllers are HUD-displayed, as shown in detail in FIG. The steering switch 22 is shown in detail in FIG.

Next, the operation will be described. FIG. 5 shows a flow of processing in the eye-gaze switch system. First, in step 101, when a measurement start signal is output from the overall control unit 12, based on this, a step 102 is executed.
Then, the light emission control unit 11 outputs a real Riga signal,
The first divergent light 1 is turned on and the face of the vehicle driver is illuminated. Then, in step 103, the image of the illuminated face area is captured by the color camera 4, and the image information is independently A / D converted by the A / D converter 5 for each of the R, G, and B components. Digital image data DR1 (x, y), DG1 (x, y), D
It is stored in the image memory 6 as B1 (x, y).

Subsequently, in step 104, the second divergent illumination 2 is turned on by the trigger signal from the illumination light emission control unit 11 to illuminate the face of the vehicle driver. Then, in step 105, the image of the illuminated face area is captured by the color camera 4, and the image information is independently A / D converted by the A / D converter 5 as described above. And digital image data DR2
It is stored in the image memory 6 as (x, y), DG2 (x, y), and DB2 (x, y). Step 102-
Reference numeral 105 constitutes the image input means of the invention.

Thereafter, in step 106, the retinal reflection image extraction unit 7 causes the R (red) component image data DR
The position of the retina reflection image of the vehicle driver is measured from 1 (x, y) and DR2 (x, y). Here, the retina reflection image is
When white light enters the eyeball, it returns as red reflected light. This is due to the reflection of blood passing through the retina, as is known as the "red eye phenomenon" when using a strobe in photography, for example. On the other hand, regarding the reflection image from the spectacles, generally used spectacles are often made of glass or plastic, and the wavelength selectivity of reflected light is low in the visible light region. Therefore, the reflected image returned as red light when the white light is emitted is only the retina reflected image. Therefore, the R of the reflection image in the image
By searching for a region where the luminance level of the (red) component is high, the retina reflection image can be extracted separately from the other images.

FIG. 6 shows details of the position measurement of the retina reflection image in step 106. First, of the input image data from the first and second divergent illuminations 1 and 2 as shown in FIGS. 7A and 7B, the R component image data DR1 (x,
y) and DR2 (x, y) are read out from the image memory 6, and in step 201, the difference calculation between these red component image data DRl (x, y) and DR2 (x, y) is performed, Image data DR3 like (c) of 7
(X, y) is required. In step 202, the image data DR3 (x, y) is binarized with a predetermined threshold value Th1 (Th1> 0) set in advance, and (d) in FIG.
Image data R4 (x, y) from which noise has been removed as in
It is said that It should be noted that the red area originally existing in the image regardless of whether or not the illumination is turned on is removed in step 201 and step 202.

Subsequently, in step 203, a labeling process for numbering each area in the image data DR4 (x, y) is performed. Then, in step 204, of the labeled regions, the one having the largest area is extracted as a retinal reflection image. Then, in step 205, the barycentric position (Xgi, Ygi) of the retina reflection image is calculated. Steps 201-2
The step 106 consisting of 05 constitutes the retinal reflection image extracting means of the invention.

Returning to FIG. 5, in order to specify the position of the corneal reflection image based on the position of the center of gravity of the retina reflection image obtained above, in step 107, a search for searching the corneal reflection image in the vicinity of the retina reflection image. Area T is set. The search area T is
As shown in FIG. 8, the center of gravity of the retina reflection image is set as the center, and the range is set to p × q. This means that if the position of the retina reflection image can be specified, the position of the corneal reflection image by the first divergent illumination 1 and the position of the corneal reflection image by the second divergent illumination 2 are limited to the vicinity of the retina reflection image. Because it will be.

Then, in step 108, the corneal reflection image extraction unit 8 extracts the corneal reflection image. Here, since white light is used for illumination, the corneal reflection image is also a white bright spot. When specifying a true corneal reflection image from the observed reflected light, the wavelength selectivity in the visible light region is small in reflection and transmission by the spectacle lens and reflection on the corneal surface. There is no great difference in the calculation of the corneal reflection image position regardless of which color component is used for the embedding. However, if the R component is used for the embedding, the retina reflection image may overlap with the corneal reflection image to be extracted. Therefore, the G or B component may be used.

The corneal reflection image can be classified into first to fourth Purkinje images depending on which surface of the optical system of the eyeball part is generated, but here, it is generated on the corneal surface, which is the brightest among them. The first Purkinje image is targeted. When the vehicle driver wears the spectacles, the reflected light from the lenses and the frame of the spectacles is simultaneously observed in addition to the corneal reflection image. They are,
It is observed as a bright area like the corneal reflection image.

FIG. 9 shows a detailed flow of corneal reflection image extraction in such an environment. In step 301,
First, the difference calculation of the image data DG1 (x, y) and DG2 (x, y) as shown in (a) and (b) of FIG. 10 is performed. The image data DG3 (x, y) after the embedding is, as shown in (c), a spectacle region or a cheek region having the same gray value between the images DG1 (x, y) and DG2 (x, y), and a white eye. Areas and the like are removed, and areas such as a corneal reflection image, a spectacle lens reflection image, and a spectacle frame reflection image having large gray values are emphasized. Here, positive values and negative values are represented as they are, and the filled portions in the figure show negative values.

In step 302, the image data DG3
The p × q region of (x, y) is binarized with a preset threshold value Th2 (Th2> 0), and image data DG4 (x, y) as shown in (d) is obtained. Step 303
At step 304, after the binarized and extracted region is subjected to labeling embedding, area threshold value embedding is performed at step 304. Since the corneal reflection image is a bright spot having a small area of several dots or less, the area of each labeled region is calculated in this embedding process, and a region having a certain area Sth or more is excluded to obtain a small bright spot region. Is extracted.

In step 305, the position of the center of gravity of the candidate region i is calculated by using the region thus extracted as the candidate region i of the corneal reflection image. In the next step 306,
The search area U for the reflected image by the second divergent illumination 2 is set. That is, if the position of the reflected image by the first divergent illumination 1 is specified, the position where the reflected image by the second divergent illumination 2 is generated is limited to the vicinity thereof. Therefore, a small area of m × n pixels is set including each barycentric position calculated in the previous step.

The search area U is determined according to the setting states of the first and second divergent illuminations 1 and 2. For example, in FIG.
In the arrangement in which the second divergent illumination 2 is on the right side of the first divergent illumination 1 as seen from the driver, as shown in (a) of
As shown in (b) of the same figure, rather than the corneal reflection image A1 by the first divergent illumination 1, the corneal reflection image A by the second divergent illumination 2
2 is located on the left side of the image. Further, as shown in FIG. 16, the corresponding reflection image appears on a line parallel to the line segment connecting the first divergent illumination 1 and the second divergent illumination 2. Therefore, as the search area U, as shown in (c) of FIG. 11, a range extending from the center of gravity V of the candidate area i to the left may be set.

In the next step 307, the reflection image by the second divergent illumination 2 is searched in the search area as described above for each candidate area i. Here, the image data DG3 (x, y) of each search area is set to the threshold value Th3.
Binarization processing is performed with (Th3 <0), and it is searched whether there is a pixel satisfying DG3 (x, y) <Th3.

In step 308, the corneal reflection image is specified according to the result of the binarization process. (1) When pixels satisfying DG3 (x, y) <Th3 are not extracted in the search area. At this time, the bright spot region corresponding to the reflection image by the second divergent illumination corresponding to the candidate region i does not exist in the search region, that is, the candidate region i is not the corneal reflection image. (2) When only one region composed of pixels satisfying DG3 (x, y) <Th3 is extracted in the search region. At this time, the barycentric coordinates (Xgi, Y
gi), which is stored in memory.

(3) DG3 (x, y) <T in the search area
When two or more regions formed by pixels satisfying h3 are extracted. At this time, if the image is a true corneal reflection image, since it is parallel to the corneal reflection image by the first divergent illumination 1, the difference in y coordinate from the candidate region i is minimized. Therefore, the barycentric coordinates (Xgi, Ygi) of the extracted region having the smallest difference in y coordinates are determined as a true corneal reflection image. Step 108 including the above steps 301 to 308
Form the corneal reflection image extraction means.

Returning to FIG. 5, after this, step 109
Then, in the line-of-sight direction calculation unit 10, the line-of-sight direction of the driver is calculated based on the barycentric coordinates of the two corneal reflection images and the barycentric coordinates of the pupil region obtained so far. FIG. 12 is an explanatory diagram showing the principle of calculating the line-of-sight direction in the line-of-sight direction calculation unit. In the case of a coaxial system in which the illumination and the optical axis of the camera coincide with each other, the straight line connecting the camera focus F1 (= illumination light source L1 position) and the corneal reflection image β on the camera C (on the CCD surface) is the corneal sphere center O. pass. Therefore, the center O of the corneal sphere is the straight line F.
It is on 1-β. The actual pupil center Q is on the straight line F1 -α connecting the camera focus (illumination position) and the pupil center α on the CCD image.

The focal length of the camera 4 is f, and the focal position is F1.
Assuming that (0, 0, 0), the X-axis, Y-axis, and Z-axis world coordinate system F1-XYZ is considered, and the light source L2 for non-coaxial illumination is F2.
In (a, b, c), the illumination light is specularly reflected on the corneal surface. The specular reflection point P is the light source L1 on the CCD surface.
A straight line F1− connecting the corneal reflection image δ and the camera focus F1
It is on δ. Then, if the provisional regular reflection point P is placed on the straight line F1 -δ and the spherical surface J is drawn with a radius of 7.8 mm, which is the radius of the corneal sphere, the center of the corneal sphere O is defined as the intersection of the spherical surface J and the straight line F1 -β.
The three-dimensional position of is determined. The distance between the corneal sphere center O and the pupil center Q is about 4.2 mm, and the corneal sphere center O determined above is determined.
Spherical surface K with a radius of 4.2 mm and a straight line F1 -α
The three-dimensional position of the pupil center Q is determined as the intersection point of. Thus, the line connecting the points O and Q is obtained as the line-of-sight direction. Step 109 constitutes the gaze direction calculating means.

In step 110, the line-of-sight stop determination unit 23 determines which area (control item) of the line-of-sight switch area 26 is viewed by the line-of-sight direction calculated by the line-of-sight direction calculation unit 10 as described above. , Calculate the time. Then, in step 111, a stop determination is made.
This is done depending on whether the driver is looking at the same area a certain number of times in the processing cycle. If 400 msec is required for one cycle of gaze direction calculation,
For example, it is determined that the user is gazing at the same area with a gazing time of about 0.8 sec for two consecutive times.

Here, the driver selects, for example, the air conditioner area (A / C) in the line-of-sight switch area 26 shown in FIG. 4 (a).
If it is determined that the user is gazing at, the process proceeds to step 112, the object, that is, the air conditioner is controlled, and if there is no stop for a certain time or more, the process returns to step 102 and the measurement of the line-of-sight direction is repeated. In step 112, the controller switching unit 27 switches to, for example, air conditioner control, and the HUD display control unit 24 displays the current air conditioner set temperature on the HUD display unit 25. It should be noted that when moving to step 112, the measurement of the line-of-sight direction is stopped.

When the air conditioner set temperature is displayed on the HUD display section 25, the up / down button of the steering switch 22 functions to adjust the set temperature up and down. While looking at the HUD display portion 25, the driver operates the up / down button of the steering switch 22 to set the desired set temperature. According to the set information,
A predetermined temperature control is performed by the air conditioner controller 29. If the up / down button is not operated for a certain period of time (for example, 5 seconds), it is determined that the operation is completed, and all the switch functions are stopped. The driver again needs to perform some operation, and when the main switch 21 is pressed, the above processing is repeated again.

The embodiment is configured as described above, and when the eyeglass reflex image is separated from the corneal reflex image in measuring the driver's line-of-sight direction, the fact that the retina reflex image has a red component is utilized.
The retinal reflection image was extracted by focusing on the red component in the image, and the corneal reflection image was specified based on the position of this retinal reflection image. Can be asked. As described above, since the color camera can be installed relatively close to the vehicle driver and the imaging range can be limited, the driver's eyeball is enlarged and photographed to measure the gaze direction over a wide range with relatively high accuracy. It is possible.

In addition, although white light is used as the light source in the embodiment, the light source is not limited to this, and if it can be discriminated from the red component superimposed by the retinal reflection, the light source having a component such as blue or green. Can also be used. Further, in the above embodiment, the color camera 4 inputs all three colors of R, G, and B as image data of each color component and outputs the data of each color component. As shown, the incident light reflected from the eyeball portion is guided to the dichroic mirror 51 and the like, only the wavelength including the red component of the retina reflection image is reflected, and this is input to the camera or the image pickup element 52, and the other transmitted components. Other camera or image sensor 5
The image data can be obtained from different systems by inputting the data to the third system. This allows a normal black and white camera to be used instead of an expensive color camera.

Alternatively, as shown in FIG. 14, a camera or an image pickup device 52 is provided by using a beam splitter 55 or the like which distributes the total amount of incident light into reflection and transmission at a predetermined ratio.
A filter 58 for transmitting only the wavelength including the red component of the retina reflection image is installed in front of one of the cameras or the image pickup device 52. This also makes it possible to obtain image data of the red component separated from the other color components by using an inexpensive black-and-white camera or the like.

[0040]

As described above, the present invention extracts the retinal reflection image and the corneal reflection image from the image data of the eyeball portion, and calculates the driver's gaze direction from the pupil center and the corneal reflection image. In the measuring device, the fact that the retinal reflection image has a unique color component of red is used to extract the retinal reflection image by using image data having a unique color component of the retinal reflection image, and Since the corneal reflection image is extracted based on the extracted retinal reflection image position, even if the driver wears spectacles, the retinal reflection image and the corneal reflection image are surely obtained from the reflection image of the spectacle lens or the frame. It has an effect that it can be easily specified and highly accurate gaze direction measurement can be performed.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of the present invention.

FIG. 2 is a diagram showing a configuration of an exemplary embodiment of the present invention.

FIG. 3 is a diagram showing an in-vehicle layout of the embodiment.

FIG. 4 is a diagram showing details of a line-of-sight switch area and a steering switch.

FIG. 5 is a main flowchart showing the flow of processing in the embodiment.

FIG. 6 is a flowchart showing details of position measurement of a retina reflection image.

FIG. 7 is an explanatory diagram showing a flow of processing of image data.

FIG. 8 is a diagram showing a search area.

FIG. 9 is a flowchart showing details of corneal reflection image extraction.

FIG. 10 is an explanatory diagram showing a flow of processing of image data.

FIG. 11 is a diagram showing a procedure for setting a search area.

FIG. 12 is an explanatory diagram showing a principle of calculating a line-of-sight direction.

FIG. 13 is a diagram showing another example of inputting image data of each color component.

FIG. 14 is a diagram showing another example of inputting image data of each color component.

FIG. 15 is a diagram showing an example of a reflected image when wearing glasses.

FIG. 16 is a diagram showing a gap between a reflection image of a spectacle lens and a corneal reflection image by two illuminations.

[Explanation of symbols]

 1 First divergent illumination 2 Second divergent illumination 4 Color camera 5 A / D converter 6 Image memory 7 Retinal reflection image extraction unit 8 Corneal reflection image extraction unit 10 Eye-gaze direction calculation unit 11 Illumination emission control unit 12 Overall control unit 13 Eyeball 21 Main switch 22 Steering switch 23 Line-of-sight stop determination unit 24 HUD display control unit 25 HUD display unit 26 Line-of-sight switch area 27 Controller switching unit 28 Audio controller 29 Air conditioner controller 30 Radio controller 31 ASCD controller 41 Front windshield 42 Steering 51 dichroic mirror 52, 53 camera or image sensor 55 beam splitter 58 filter T, U search area

Claims (7)

[Claims] 1. An image input for obtaining image data for each illumination, comprising a plurality of illuminations for illuminating a driver's eyeballs from different directions and an imaging device for capturing a reflected image from the eyeballs by the plurality of illuminations. Means, a retinal reflection image extracting means for extracting a retinal reflection image from the image data to calculate a pupil center, and a corneal reflection image extracting means for extracting a corneal reflection image from the image data, A corneal reflection image and a line-of-sight direction calculating means for calculating the driver's line-of-sight direction from the center of the pupil, and the image input means includes image data having a color component specific to the retina reflection image as the image data. Image data having a color component and other color components, the retinal reflection image extraction means,
The retinal reflection image is extracted based on the position of the retinal reflection image based on the position of the retinal reflection image by distinguishing from other reflection images by using image data having a color component specific to the retinal reflection image to extract the retinal reflection image. A visual line direction measuring device for a vehicle, which extracts a corneal reflection image. 2. The gaze direction measuring device for a vehicle according to claim 1, wherein the color component peculiar to the retina reflection image is red. 3. The corneal reflection image extracting means extracts a corneal reflection image by searching within a predetermined first search area centered on the center of gravity of the retina reflection image. The visual line direction measuring device for a vehicle according to claim 1 or 2. 4. The corneal reflection image extraction means is the first
A bright spot region having a predetermined area or less in the search region is set as a candidate region for the first corneal reflection image by the first illumination of the plurality of illuminations, and the first illumination relative to the first illumination is selected from the center of gravity of the candidate region. When there is a second bright spot region corresponding to the second corneal reflection image by the second illumination in the second search region set to extend in the predetermined direction determined corresponding to the position of the second illumination. ,
The visual line direction measuring device for a vehicle according to claim 3, wherein the candidate region and the second bright spot region are formed as a corneal reflection image. 5. The vehicle gaze direction measurement according to claim 1, wherein the image pickup device of the image input means is a color camera that outputs image data for each color component. apparatus. 6. The image pickup device of the image input means separates incident light reflected from an eyeball into a wavelength component including a red component of a retina reflection image and other color components by a dichroic mirror, and detects each color component. 5. The vehicular line-of-sight direction measuring device according to claim 1, wherein the image data for each color component is output by inputting the image data into a camera or an image sensor. 7. An image pickup device of the image input means separates incident light reflected from an eyeball into two systems by a beam splitter, one system passing through a red transmission filter, and a camera detecting each color component. 5. The vehicular line-of-sight direction measuring device according to claim 1, wherein the image data is input to an image pickup device to output image data for each color component.