JPH0265834A - Visual axis detector - Google Patents

Abstract

PURPOSE:To accurately detect a visual axis by detecting the boundary of the iris and the pupil or the part of the anulus iridis with good accuracy by controlling the accumulation time of the image accumulated on a solid-state imaging element to obtain a signal wherein the part of the anulus iridis is magnified. CONSTITUTION:The data determined on the basis of the ratio of the reflecting characteristics of the cornea and the white of the eye is inputted to the first data part 47 and the data determined on the basis of the ratio of the reflecting characteristics of the cornea and the iris is inputted to the second data part 48. The accumulation of image data on a solid-state imaging element 11 is started and the time up to the reversal of a comparator 44 is magnified by a definite time ratio on the basis of the data from the first data part 47 or the second data part 48 to output image data. The image data from the output terminal C of the solid-state imaging element 11 is converted to digital signal by an A/D converter circuit 53 wherein upper and lower limit voltage levels V1, V2 are set and an image signal V3 always enters the optimum level with respect to the upper and lower limit voltage levels of A/D conversion. A visual axis operational processing circuit 58 detects the visual axis direction of a cameraman on the basis of the data of the reflected image of the cornea, the boundary of the iris and the pupil and the part of the anulus iridis.

Description

DETAILED DESCRIPTION OF THE INVENTION [Industrial Field of Application] The present invention relates to a line-of-sight detection device for detecting the line-of-sight position of an observer.

[Prior Art] Conventionally, Japanese Patent Application Laid-open No. 172552/1984 is a line-of-sight detection device that optically detects the line-of-sight (visual axis) of an observer.

This is a method that detects the first Purkinje image, which is a reflected image from the front surface of the cornea when the observer's eyeball is irradiated with parallel light, and the position of the center of the pupil.This is explained based on Figure 6. do.

In the figure, 501 is the cornea, 502 is the ice film, 503 is the iris, and 5
04 is a light source, 506 is a light projecting lens, 507 is a light receiving lens, 509 is an image sensor, and 510 is a half mirror. Oo is the center of rotation of the eyeball, 0 is the center of curvature of the cornea 50,
a and b are the ends of the iris 503, C is the center of the iris, and d is the first
This is the Purkinje image generation position. A indicates the optical axis of the light receiving lens 507, which coincides with the X axis in the figure. A is the optical axis of the eyeball.

The light source 504 is an infrared emitting diode that is insensitive to the viewer, and is placed in the focal plane of the projection lens 506. The infrared light emitted from the light source 504 is turned into parallel light by the projection lens 506 and reflected by the half mirror 510, and is reflected by the cornea 501.
to illuminate. A part of the infrared light reflected on the surface of corner 1] 1501 passes through half mirror 510 and is imaged by light receiving lens 507 at position d° on image sensor 509. Also, the ends a and b of the iris 503 are connected to a position a° on the image sensor 509 via the half mirror 510 and the light receiving lens 507.
.

Image is formed on bo. With respect to the optical axis A of the light receiving lens 507.

When the rotation angle θ of the optical axis i of the eyeball is small, if the Z coordinates of the ends a and b of the iris 503 are Zll+zb, the iris 50
The coordinate 10 of the center position C of 3 is expressed as Zc#-□.

Further, if the Z coordinate of the first Purkinje image generation position d is zd, and the distance from the center of curvature 0 of the cornea 501 to the center C of the iris 503 is i, the rotation angle θ of the eyeball optical axis I is oυsinθ#zc−z, ”'
The relational expression (1) is approximately satisfied. Therefore, each singular point (first Purkinje image zd
The rotation angle θ of the eyeball optical axis I becomes clear by detecting the positions of the iris end portions z, ′, Zb′). At this time, equation (1) can be rearranged. However, β is the distance intersection between the first Purkinje image generation position and the light receiving lens 507, and the distance between the light receiving lens 507 and the first Purkinje image generation position.
The magnification is determined by the distance 11112 degrees between the image sensor 509 and the image sensor 509, and normally takes a nearly constant value.

The principle as described above makes it possible to detect the direction of the line of sight.

[Problems to be Solved by the Invention] However, in this type of conventional line of sight detection device, the reflectance of the cornea is approximately 2.5!6, and as shown in FIG. 2, for example, the light intensity of the corneal reflected image is insufficient. However, the reflectance of the iris is extremely low, and it is actually quite difficult to accurately detect the boundary between the iris and pupil to determine the center position of the pupil.

An object of the present invention is to provide a line of sight detection device that can accurately detect the iris limbus, which is the boundary between the iris and the pupil, and the boundary between the R membrane (white of the eye) and the iris (black of the eye), and accurately detect the line of sight. It is something to do.

[Means for Solving the Problems] The gist of the present invention is to provide an illumination means for illuminating the eye, and to determine the position of the Purkinje image and the eye using the light reflected from the eye illuminated by the illumination means. an image detection means consisting of a solid-state imaging device that detects the image position of other tissues in the eye; and a line of sight that detects the line of sight direction from the relative relationship between the Purkinje image position detected by the image detection means and the image position of other tissues of the eye. The image detection means includes a calculation means and an accumulation time control means for controlling the accumulation time of the solid-state image sensor, and the image detection means has an overflow train function for discarding image information accumulated over a certain amount, and an overflow train function for discarding image information accumulated over a certain amount. The structure has a peak value output function of outputting a peak value, and the accumulation time control means controls the time from the start of image accumulation of the image detection means until the peak value of the image information reaches a certain value between the cornea and the iris or the electromembrane. The line of sight detection device is characterized in that the accumulation time is a value multiplied by a certain value based on the ratio of the reflection characteristics, and when the accumulation time is reached, the accumulated image information is sequentially output.

[Function] The line of sight detection device configured as described above can multiply the image information to be detected by controlling the image storage time and extract each image as enlarged information.

[Example] Hereinafter, the present invention will be described in detail based on an example shown in the drawings.

FIG. 1 is an embodiment of an optical block of a camera having a line of sight detection device according to the present invention.

1 is a photographing lens, 2 is a quick return mirror, 3 is a focusing plate, 4 is a condenser lens, and 5 is a pentaprism, forming a normal finder optical system.

6 is an eyepiece lens that has a beam splitter inside that transmits visible light and reflects infrared light, and 7 is a beam splitter.
8 is a light projecting lens, 9 is a light receiving lens, and 10 is an infrared light projection lens.
ED, 11 is a photoelectric conversion element such as a linear or area type CCD, and forms a line of sight detection device. 12 is the photographer's eye.

The light projected from the infrared LEDIO is converted into a parallel beam by the projection lens 8 and is irradiated to the eye 12. The reflected light from the cornea and iris of the eye is reflected by a beam splitter 7 and transmitted through a light receiving lens 9 to a photoelectric conversion element (solid-state image sensor) 11.
It is configured to be imaged onto the image.

This solid-state image sensor 11 has an overflow train function that does not give an adverse image 5 to adjacent pixels even if the signal of some pixels is saturated, and a floating function that non-destructively stores image information stored in each pixel. It has a function that allows direct monitoring in real time, and a so-called real-time peak value output function that monitors the maximum value of the monitor output. Its configuration is shown in FIG.

FIG. 3 is a block diagram showing an example of the configuration of the solid-state image sensor 11. As shown in FIG.

In the figure, 31 is a photoelectric conversion storage unit for storing image information;
Reference numeral 2 denotes a charge transfer gate for transferring the information of the photoelectric conversion storage section 31 to the analog shift register 33 for reading, and when a charge 8 sending pulse is input from the BOS element,
Transfers information to the analog shift register 33,
The information is output from terminal C. 34 is an overflow drain that discards any more information when the information in the photoelectric conversion storage unit 31 reaches a certain value, and 35 non-destructively detects each piece of information in the photoelectric conversion storage unit 31 and outputs the maximum value to a terminal. This is a real-time peak output circuit that outputs from A.

By the way, the horizontal scanning signal B corresponding to the position of the eyeball when the eyeball 12 is irradiated with light from an infrared LEDIO, its reflected image is formed on the solid-state image sensor 11, and the center of the eyeball is scanned horizontally is The result is as shown in FIG.

As is clear from the figure, the corneal reflection image is very strong and can be detected accurately, but the contrast at the boundaries of other tissues is low.
As mentioned above, it is quite difficult to accurately detect the iris limbus, which is the boundary between the iris and the pupil, or the boundary between the white of the eye and the iris.

In this embodiment, the iris limbus, which is the boundary between the iris and the pupil, and the boundary between the white and the iris of the eyes, in the horizontal scanning signal B can be detected with high precision by the fourth accumulation time of images accumulated in the solid-state image sensor 11.
This is achieved by controlling with the control device shown in FIG. 5(A) and obtaining a signal that enlarges the iris limbus as shown in FIG. 5, for example.

FIG. 4 is a block diagram of a control device that controls the information storage time in the solid-state image sensor 11 and processes the output information.

Before explaining the block diagram of this control device, the basic principle of control will be explained based on the structural characteristics of the solid-state image sensor 11.

The solid-state image sensor 11 is an infrared LEDIO reflected by the eyeball 12.
When the reflected image from the When the peak value reaches a certain level near the saturation level, if a charge transfer pulse is manually applied to the B terminal and the accumulated image information is output from the output terminal C, only the horizontal scanning signal B shown in Fig. 2 will be obtained. . Therefore, even if the peak value of the corneal reflection image reaches a certain level near the saturation level, if the charge transfer pulse is not applied manually and the reflection image is accumulated as it is, the photoelectric conversion storage section 31 of the solid-state image sensor 11 Information on the corneal reflection image, the boundary between the iris and pupil, and the iris limbus, which is the boundary between the white of the eye and the iris, accumulates, and their values increase until eventually the information on the corneal reflection image becomes saturated and overflows. However, since it has an overflow drain function, there is no adverse effect on adjacent pixels.

Next, information that reaches a certain level near the saturation level is
As is clear from the figure, the information on the iris limbus at this point has a value multiplied by its accumulation time.

Therefore, if you need accurate information about the iris limbus,
By inputting a charge 9 pulse after a certain period of time based on the ratio of the reflection characteristics of the cornea and the white of the eye from the time required for the information of the corneal reflection image to reach a certain level near the saturation level, the enlarged iris limbus can be Information can then be taken out from output terminal C.

In addition, when accurate information on the boundary between the iris and the pupil is required, information on the corneal reflection image is based on the ratio of the reflection characteristics of the cornea and the iris based on the time required to reach a constant level near the saturation level. After a certain period of time has elapsed, it is sufficient to manually send nine charge pulses.

Next, the control device will be explained.

42.43 is a resistor for generating reference voltage v0, 44
is a comparator, 45 is a latch circuit, 46 is a counter,
47 is a first information section, 48 is a second information section, 49 is a selection switch, 50 is a multiplier, 51 is a memory, 52 is a magnitude comparator, 53 is an A/D conversion circuit, 54.5
5 is a resistor for generating the upper limit level voltage V1, 56.57 is a resistor for generating the lower limit level voltage V2, 58 is a line of sight arithmetic processing circuit, and the first information section 47 contains the ratio of the reflex characteristics of the cornea and the white of the eye;
That is, in FIG. 2, the peak level a of the corneal reflection image
, and the peak level a2 of the iris limbus, and the second information section 48 contains the information determined based on the ratio of the reflection characteristics of the cornea and the iris, that is, the peak level a of the corneal reflection image in FIG. , and the iris peak level a3 is inputted. Note that the details of this line-of-sight calculation processing circuit 58 will be described later.

The control device configured in this way is configured so that A of the solid-state image sensor 11
The peak output of each pixel from the terminal is input to the comparator 44 in real time, and the value is set as the reference voltage V.
When it reaches 0, the output of the comparator 44 is inverted, and the latch circuit 45 latches the count value of the counter 46 which has started counting at the same time as the accumulation of image information of the solid-state image sensor 11 starts.

On the other hand, which information is required, the information on the iris limbus or the information on the boundary between the iris and the pupil, is selected in advance by the selection switch 49, and the information latched by the latch circuit 45 and the information selected by the selection switch 49 are selected in advance. is multiplied by the multiplier 50 and stored in the memory 51. Here, the magnitude comparator 52 compares the count value of the counter 46 and the data stored in the memory 51, and generates a match signal when the count value of the counter 46 matches the data in the memory 51.2. A charge 9 sending pulse is applied to the input terminal B of the solid-state image sensor 11, and image information is output from the output terminal C.

That is, after the time from when the accumulation of image information to the solid-state image sensor 11 is started until the comparator 44 is inverted is multiplied by a certain time based on the information from the first information section 47 or the second information section 48, the image is displayed. Information will be output.

The image information from the output terminal C of the solid-state image sensor 11 is processed by the line-of-sight calculation processing circuit 58, so that the direction of the photographer's line of sight is processed by the line-of-sight arithmetic processing circuit 58.
It will be A/D converted by the /D conversion circuit 53,
For example, when selecting the first information section 47 to detect the iris limbus, as shown in FIG. It will always be at the optimal level. Note that the lower limit voltage level (2) may be set, for example, near the dark current signal level of the solid-state image sensor 11.

The line of sight calculation processing circuit 58 detects the direction of the photographer's line of sight based on the corneal reflection image, the boundary between the iris and the pupil, information on the iris limbus, etc., and based on the detected information, an exposure control circuit of the camera (not shown), It controls the focus detection circuit, etc., so that the photographer can adjust the exposure and focus on what he or she wants to photograph.

This line of sight arithmetic processing circuit 58 is configured as shown in FIG. 4<8).

101 is a pupil edge detection unit that detects the edge of the pupil based on the output signal from the A/D conversion circuit 53, and 102 is a pupil center detection unit that detects the center of the pupil from the information output from the pupil edge detection unit 101. , 103 is a corneal reflection image position detection unit that detects the corneal reflection image position based on the output signal from the A/D conversion circuit 53, and 104 is a corneal reflection image position (first Purkinje image) from the corneal reflection image position detection unit 103. )
This is a visual axis calculation unit that calculates the direction of the line of sight using the method shown in FIG. 6 based on the pupil center information from the pupil center detection unit 102.

In the above embodiment, among the output image signals from the solid-state image sensor 11, the signal of the corneal reflection image is a saturated signal, and if the blur of the corneal reflection image is known in advance, the position of the corneal reflection image can be determined from the saturated signal. can also be detected with high accuracy.

Furthermore, the ratio of the reflection characteristics of the cornea and the iris or the whites of the eyes differs depending on the race, such as Westerners and Asians, but the ratio of the reflection characteristics of the cornea and the iris or the whites of the eyes is different from that shown in Figure 4 (A). It is good to set the information corresponding to the second dimension Aojuen part 47 and 48.

[Effects of the Invention] As described above, when the present invention is used, the boundary between the pupil and the iris and the iris limbus can be detected with a good S/N ratio, so there is a remarkable effect that highly accurate line of sight detection is possible.

In addition, even if the optical power of the illumination means such as a floodlight LED changes, the storage time of the element is automatically corrected via the peak output circuit of the solid-state image sensor, so the iris and whites of the eyes are always at a TiM output level. This has the effect of making it possible to obtain highly accurate line-of-sight signals.

[Brief explanation of the drawing]

Fig. 1 is an optical block diagram of a camera having an embodiment of the line of sight detection device according to the present invention, Fig. 2 is a diagram showing horizontal scanning signals of a solid-state image sensor corresponding to the position of the eyeball, and Fig. 3 is a diagram showing the solid-state image sensor. 4(A) is a system block diagram showing an example of the control device of the visual line detection device, FIG. 4(B) is a block diagram showing details of the visual line calculation processing circuit, Figure 5 shows one of the output waveforms from the solid-state image sensor.
An example diagram, FIG. 6, is a schematic diagram of a conventional line of sight detection device that detects the line of sight using a corneal reflection image and the center of the pupil. 7... Beam splitter 8... Emitter lens, 9... Light receiving lens, 10... Infrared LED. 12...Eyes (eyeballs). 4 others Figure 2: Iris: Figure 4 (B) Figure 5 11111: -- :Rainbow 'f/.

Claims (1)

[Scope of Claims] 1. Image detection comprising an illumination means that illuminates the eye, and a solid-state image sensor that detects the Purkinje image position and the image position of other tissues of the eye using the light reflected from the eye illuminated by the illumination means. means, line-of-sight calculation means for detecting a line-of-sight direction from the relative relationship between the Purkinje image position detected by the image detection means and the image position of other tissues of the eye, and an accumulation time for controlling the accumulation time of the solid-state image sensor. the image detection means has a structure having an overflow drain function for discarding image information accumulated over a certain amount and a peak value output function for outputting the peak value of image information of each pixel; The time control means sets the accumulation time to a value obtained by multiplying the time from the start of image accumulation by the image detection means until the peak value of the image information reaches a certain value based on the ratio of the reflection characteristics of the cornea and the iris or the sclera. A line-of-sight detection device characterized by outputting accumulated image information sequentially when an accumulation time is reached.