JPH0265835A - Visual axis detector - Google Patents

Abstract

PURPOSE:To accurate detect a visual axis by detecting the part of the anulus iridis with good accuracy by detecting the data shortest in the accumulation time of image data in a plurality of line sensors as Purkinje's figures and accumulating the other image data until the peak value of the image data becomes a definite value. CONSTITUTION:A solid-state imaging element detects the reflected image of the cornea on the basis of the output signal from the second C-terminal C2, for example, when the reflected image of the cornea to the solid-state imaging element is present at the position 43 of the second photoelectric converting accumulation part 32 by parallelly arranging photoelectric converting accumulation parts in three stages in an up-and-down direction in the direction crossing the reflected light axis from an eyeball at a right angle. When the peak of the integrated value in either one of the first and third photoelectric converting accumulation parts 31, 33 reaches reference voltage Vo, either one of the first and third comparators 54, 56 corresponding thereto is reversed and, for example, when the third comparator 56 is reversed, an one-shot pulse is generated from a circuit 63 and a charge transfer pulse is applied to the third B-terminal B3 and image data is outputted from the third C-terminal C3 and the part of the anulus iridis is magnified by integrating the image data.

Description

DETAILED DESCRIPTION OF THE INVENTION [Industrial Field of Application] The present invention relates to a line-of-sight detection device for detecting the line-of-sight position of an observer.

[Prior Art] Conventionally, Japanese Patent Application Laid-open No. 172552/1984 is a line-of-sight detection device that optically detects the line-of-sight (visual axis) of an observer.

This is a method that detects the first Purkinje image, which is a reflected image from the front surface of the cornea generated by illuminating the observer's eyeball with parallel light, and the position of the center of the O1 hole.
This will be explained based on the diagram.

In the figure, 501 is the cornea, 502 is the R membrane, 503 is the iris, and 5
04 is a light source, 506 is a light projecting lens, 507 is a light receiving lens, 509 is an image sensor, and 1510 is a half mirror. 0'' is the center of rotation of the eyeball, 0 is the center of curvature of the cornea 501, a and b are the ends of the iris 503, C is the center of the iris, and d is the position where the first Purkinje image is generated. A is the light receiving lens 507
The optical axis coincides with the X axis in the figure. A is the optical axis of the eyeball.

The light source 504 is an infrared emitting diode that is insensitive to the viewer, and is placed in the focal plane of the projection lens 506. The infrared light emitted from the light source 504 is turned into parallel light by the projection lens 506 and reflected by the half mirror 510, and is reflected by the cornea 501.
to illuminate. A portion of the infrared light reflected on the surface of the cornea 501 passes through the half mirror 510 and is imaged by the light receiving lens 507 at a position d° on the image sensor 509. Also, the ends a and b of the iris 503 are a half mirror 510 and a light receiving lens 5.
07 on the image sensor 509.

Image is formed on bo. With respect to the optical axis A of the light receiving lens 507.

When the rotation angle θ of the optical axis I of the eyeball is small, and the Z coordinates of the ends a and b of the iris 503 are za, Zl), the iris 50
The coordinate zC of the center position C of 3 is expressed as Za+Zh zc cape□.

In addition, the Z coordinate of the first Purkinje image generation position d is Zd, and the distance m from the center of curvature 0 of the cornea 501 to the center C of the iris 503 is
If toc, the rotation angle O of the optical axis of the eyeball is θ#Zc-Z6
... The relational expression (1) is approximately satisfied. Therefore, by detecting the position of each singular point (first Purkinje image Zd' and iris end Z1°, Zb') projected onto the image sensor 509, the rotation angle θ of the eyeball optical axis I becomes clear. At this time, equation (1) can be replaced with β·oc-sinθ~-Zd' ++ (2). However, β is the distance between the first Purkinje image generation position and the light receiving lens 507, and the distance between the first Purkinje image generation position and the light receiving lens 507.
This is a magnification determined by the distance 11iffR8 between the image sensor 509 and the image sensor 509, and normally takes a nearly constant value.

The principle as described above makes it possible to detect the direction of the line of sight.

[Problems to be Solved by the Invention] By the way, in this type of conventional line of sight detection device, the reflectance of the cornea is about 2.5, so the eyeball is irradiated with infrared light and the reflected image is converted into a linear or area type. Figure 2 shows the horizontal scanning signal B corresponding to the position of the eyeball when the image is formed on the solid-state image sensor and the center of the eyeball is horizontally scanned, and the corneal reflection image is detected very strongly and accurately. However, the contrast at the borders of other tissues is low, making it quite difficult to accurately detect the border between the iris and the pupil, or the iris limbus, which is the border between the white of the eye and the iris.

An object of the present invention is to provide a line of sight detection device that can accurately detect the iris limbus, which is the boundary between the iris and the pupil, and the boundary between the electromembrane (white of the eye) and the iris (black of the eye), and accurately detect the line of sight. It is something to do.

[Means for Solving the Problems] The gist of the present invention is to provide an illumination means for illuminating the eye, and to determine the position of the Purkinje image and the eye using the light reflected from the eye illuminated by the illumination means. an image detection means consisting of a solid-state imaging device that detects the image position of other tissues in the eye; and a line of sight that detects the line of sight direction from the relative relationship between the Purkinje image position detected by the image detection means and the image position of other tissues of the eye. The image detection means includes a calculation means and an accumulation time control means for controlling the accumulation time of the solid-state image sensing device, and the image detection means includes a line sensor that outputs the peak value of image information in real time relative to the optical axis of the reflected light. A plurality of them are arranged in parallel in orthogonal directions, and the accumulation time control means controls the image for each line sensor from the start of image accumulation in each line sensor of the image detection means until the peak value of image information reaches a certain value. The line of sight detection device is characterized in that it performs accumulation, detects the accumulation time for each line sensor, and selects image information to be detected by each line sensor based on the detected accumulation time.

[Function] The line-of-sight detection device that has been developed as described above uses an image detection means consisting of a solid-state image sensor to detect information with the shortest accumulation time of image information as a Purkinje image among a plurality of line sensors, and to Image information is obtained from other line sensors, and other image information is accumulated until the peak value of the image information reaches a constant value, and is detected with high accuracy.

[Example] Embodiments of the present invention will be described in detail below with reference to the drawings.

FIG. 1 is an embodiment of an optical block of a camera having a line of sight detection device according to the present invention.

1 is a photographing lens, 2 is a quick return mirror, 3 is a focusing plate, 4 is a condenser lens, and 5 is a pentaprism, forming a normal finder optical system.

6 is an eyepiece lens that has a beam splitter inside that transmits visible light and reflects infrared light, and 7 is a beam splitter.
8 is a light projecting lens, 9 is a light receiving lens, and 10 is an infrared light projection lens.
ED, 11 is a photoelectric conversion element such as a linear or area type DDD, and forms a line of sight detection device. Reference numeral 12 indicates the eyes of three people who take pictures.

The light projected from the infrared LEDIO is converted into a parallel beam by the projection lens 8 and is irradiated to the eye 12. The reflected light from the cornea and iris of the eye is reflected by the beam splitter 7 and transmitted through the light receiving lens 9 to the photoelectric conversion element (fixed image sensor) 11.
It is configured to be imaged onto the image.

This solid-state image sensor 11 has a function that can directly monitor image information stored in each pixel such as a floating gate in real time in a non-destructive manner, and a so-called real-time peak value output function that outputs the maximum value of the monitor output. The configuration is shown in FIG.

FIG. 3 is a block diagram showing an example of the configuration of the solid-state image sensor 11. As shown in FIG.

In the figure, 31, 32, and 33 are the first
The photoelectric conversion storage section, the second ninth electric conversion storage section, and the third photoelectric conversion storage section are arranged in parallel in the vertical direction. 34, 35, and 36 are for transferring the information of the first to third photoelectric conversion storage units 31 to 33 to the first analog shift register 37, second analog shift register 38, and third analog shift register 39 for reading, respectively. These are the first charge transfer gate, the second charge transfer gate, and the third charge transfer gate, and when a charge transfer pulse is input from the first B terminal B, the information is transferred to the first analog shift register 37, and the information is transferred to the first analog shift register 37. When information is output from the first C terminal C1 and charge transfer pulses are similarly input from the second B terminal B2 and the third B terminal B, the second analog shift register 38 and the third analog shift register 3
9, and the information is output from the 20th terminal 02% 3rd C4 child C3, respectively.

40.41.42 are the first to third photoelectric conversion storage units 31 to 3
3. Non-destructively detect each piece of information, and calculate their maximum value as the first
A terminal A1 2nd A terminal A2. This is a real-time peak output circuit that outputs from the third Ai element A3.

In the solid-state image sensor 11 of this embodiment, the photoelectric conversion and storage sections are arranged vertically in three stages in a direction perpendicular to the optical axis of reflection from the eyeball. When the photoelectric conversion storage unit 32 is at position 43, the corneal reflection image is detected based on the output signal from the 20th terminal C2, and the corneal reflection image is detected based on the output signal from the 1st C terminal C3 or the 30th terminal C3. Then, the boundary between the iris and the pupil and the iris limbus are detected, and at this time, the first photoelectric conversion storage section 31. The second photoelectric conversion storage section 32 or the third photoelectric conversion storage section 33 independently controls the storage time to increase the S/N of the output signals from the first C terminal CI, the 20th terminal C2, and the 30th terminal C8. That's what I do. Further, for example, the corneal reflection image on the solid-state image sensor 11 is
1 and the second photoelectric conversion storage section 32, the photoelectric conversion storage section 31 and the second photoelectric conversion storage section 32, whichever has the shorter storage time, The corneal reflection image is detected based on the output signal from the third photoelectric conversion storage section 33, and the border between the iris and the pupil and the iris limbus are detected based on the output signal from the third photoelectric conversion storage section 33.
Such control is performed by the control device shown in FIG. 4(A), and for example, the signal of the corneal reflection image as shown in FIG. 5(A) is extracted, and the iris signal as shown in FIG. I am trying to get a signal that magnifies the limbus.

Note that when the corneal reflection image is at position 43 in Figure 3, in order to detect the boundary between the iris and pupil and the iris limbus,
The output signal from the first photoelectric conversion storage section 31 may be used, or the output signal from the third photoelectric conversion storage section 33 may be used.

However, when the corneal reflection image is at position 44 in FIG. 3, it straddles two photoelectric conversion and accumulation sections, so
It is necessary to detect the boundary between the iris and pupil and the iris limbus using the output signal from the photoelectric conversion storage unit 33, but since the position of the corneal reflection image is not actually known, three photoelectric conversion storage units are used. When detecting the line of sight using a solid-state image sensor, the corneal reflection image position is detected based on the output signal from the photoelectric conversion storage unit with the shortest storage time, and the position of the corneal reflected image is detected based on the output signal from the photoelectric conversion storage unit with the longest storage time. It is most rational to detect the border between the iris and the pupil or the iris limbus based on the signal, and it is possible to detect the direction of the line of sight with high precision.

FIG. 4(8) is a block diagram of the control device.

51 is a solid-state image sensor (same structure as the solid-state image sensor 11 shown in FIG. 3), and 52 and 53 are reference voltages V. Resistors for generation, 54, 55, 56 are first to third comparators, 5
7, 58, 59 are first to third latch circuits, 60 is a counter, 61 to 63 are first to third one shot circuits, 646
5 is a resistor for generating the upper limit level voltage v1, 66°67 is a resistor for generating the lower limit level voltage v2, 68°69.70 is the 1st to 38th /D conversion circuits, 71 is the line of sight calculation processing circuit,
Details of this line of sight calculation processing circuit 71 will be described later.

In the control device configured in this way, the peak output of each pixel is manually input from the first to third A terminals A+ to A of the solid-state image sensor 51 to the comparators 54 to 56 in real time, and the value reaches the reference voltage vo. Then, the corresponding 1st~
When the outputs of the third comparators 54 to 56 are inverted and the accumulation of image information of the solid-state image sensor 51 is started, the count value of the counter 60 which has started counting is changed to each of the first to third comparators corresponding to the time when the output of each comparator is inverted. 3 latch circuit 57
Latch at ~59.

On the other hand, in synchronization with the inversion of the outputs of the first to third comparators 54 to 56, one-shot pulses are generated from the first to third one-shot circuits 61 to 63 corresponding to each comparator, and the corresponding pulses of the solid-state image pickup device 51 are generated. 1 to 3rd B terminal B +
, B2. B.

A charge transfer pulse is applied to the charge transfer gate, and the image information of the photoelectric conversion storage section corresponding to the charge transfer gate to which the charge transfer pulse is applied is transferred to the analog shift register.
Output begins from C terminals C1 to C5.

That is, in the reflected image from the eyeball that is received by the solid-state image sensor 51, the corneal reflected image, the pupil reflected image, the iris reflected image, and the white of the eye are not all focused on one line. A reflected image is formed, for example, at a position indicated by reference numerals 43 and 44 in FIG. 3, and a pupil reflection image, an iris reflection image, and a reflection image of the white of the eye are also formed on other photoelectric conversion storage units. Therefore, for example, when a corneal reflection image is formed at the position indicated by reference numeral 43 in FIG.
The peak output value from the second real-time peak output circuit 41 corresponding to the second real-time peak output circuit 41 is larger than the peak output value from the other 1.3 real-time peak output circuits 40 and 42.

Therefore, when the peak output value from the second real-time peak output circuit 41 reaches the reference voltage V0, the second comparator 55 is first inverted, and at the same time, the second latch circuit 58 stores the count value up to that point. It will be latched. The second one-shot circuit 62 generates a one-shot pulse due to the inversion of the second comparator 55, and a charge transfer pulse is applied to the second B terminal B2.
Image information is output from terminal C2, and as shown in FIG. 5(A), the corneal reflection image is large, as in the case of FIG.
A horizontal scanning signal is obtained in which it is difficult to distinguish other information such as the iris limbus. However, since only the information on the corneal reflection image is required here, the accuracy of other information is irrelevant.

On the other hand, in the other first and third photoelectric conversion storage units 31 and 33, the peak value of image information other than corneal reflection images such as reflection images from the pupil, iris, and white of the eye is at the first and third A terminals A + ,
A3, but since the value has not yet reached the reference voltage V0, image information is stored in the first and third photoelectric conversion storage sections 31 and 33 even after the second comparator 55 is inverted. . In this case, since the iris has a higher reflectance than the pupil, and the white of the eye has a higher reflectance than the iris, the peak value of the white of the eye next reaches the reference voltage v0.

Eventually, the peak of the integrated value in the first photoelectric conversion storage section 31 or the third photoelectric conversion storage section 33 reaches the reference voltage V. When the first comparator 54 or the third comparator 56 corresponding thereto is inverted, for example, when the third comparator 56 is inverted, the third one-shot time 1i863
h) A one-shot pulse is generated and the 38i-th child B,
A charge transfer pulse is applied to the 30th Mizuko C3, and image information is output from the 30th Mizuko C3.As shown in FIG. This results in a horizontal scanning signal that is not

Further, in synchronization with the inversion of the third comparator 56, the count value up to that point is transferred to the third latch circuit 59.

Then, the last remaining photoelectric conversion storage unit, in this case the first
The peak value of the information in the photoelectric conversion storage section 31 is the reference voltage V. , the corresponding first comparator 5
4 is inverted, a one-shot pulse is generated from the first one-shot circuit 61, a charge transfer pulse is applied to the first B-terminal B1, image information is output from the first C-terminal C3, and at the same time, the first latch circuit 57 receives a charge transfer pulse. The count value up to is latched.

The image information from the first to third C terminals C1 to C3 of the solid-state image sensor 51 is set to upper and lower voltage levels V, .
The first to third A/D conversion circuits 6 where V2 is set
A/D conversion is performed by 8 to 70. Figure 5 (
As shown in A) and (B), the image signal is A/D
The voltage levels V + and V 2 at the upper and lower limits of conversion are always at the mA level.

The line-of-sight calculation processing circuit 71 detects the line-of-sight direction based on the corneal reflection image position, the boundary between the iris and the pupil, and the iris limbus. Although it is not possible to determine what type of information the information is, as shown in Figure 2, based on the reflex characteristics of the eyeballs, first the reference voltage ■. The first to third latch circuits 57 to 3 reach the corneal reflection image, and then the white of the eye reflection image.
The image information with the smallest value among the count values latched in step 59 is the corneal reflection image, the next count value is information on the iris limbus where a part of the corneal reflection image may enter, and the last is the information on the iris limbus and pupil. For example, by detecting the line-of-sight direction according to the principle shown in Figure 2, and introducing the obtained line-of-sight information into the camera's exposure control circuit and focus detection circuit (not shown), the photographer can determine the subject he or she wants to photograph. You can adjust the exposure and focus.

This line-of-sight calculation processing port i71 is configured as shown in FIG. 4(B).

101 is a pupil edge detection unit that detects the edge of the pupil;
02 is a pupil center detection unit that detects the center of the pupil from the information output from the pupil edge detection unit 101, 103 is a corneal reflection image position detection unit that detects the corneal reflection image position, and 104 is from the corneal reflection image position detection unit 103. Corneal reflection image position (first
Purkinje image) and pupil center information from the pupil center detection unit 102, a visual axis calculation unit 105 calculates the direction of the line of sight using the method shown in FIG. The signals from ~70 are sent to the pupil edge detection unit 101,
A signal selection unit 106 selectively outputs to the corneal reflected image position detection unit 103 determines the storage time length of each photoelectric conversion storage unit of the solid-state image sensor 51 from the count values from the first to third latch circuits 57 to 59. This is an accumulation time length determining section.

That is, the accumulation time length determination unit 106 determines the accumulation time length based on the count values latched in the first to third latch circuits 57 to 59, and determines the accumulation time length of the longest accumulation time in each line of the solid-state image sensor. The signal selection unit 105 is controlled to input the signal from the short line to the corneal reflection image position detection unit 103 to detect the corneal reflection image position, and the signal from the line with the long integration time is input to the pupil edge detection unit 101. The signal selection unit 105 is controlled to detect the center position of the pupil.

[Effects of the Invention] As described above, according to the present invention, reflected images from the eye are formed on a plurality of line sensors made of solid-state image sensors, and the corneal reflected image position, Information on the boundary between the iris and pupil and the iris limbus is detected, and since the information on the boundary between the iris and the pupil and the iris limbus that is detected is a high S/N value that is integrated over a long period of time, This has the effect of allowing highly accurate detection.

[Brief explanation of the drawing]

Fig. 1 is an optical block diagram of a camera having an embodiment of the line of sight detection device according to the present invention, Fig. 2 is a diagram showing horizontal scanning signals of a solid-state image sensor corresponding to the position of the eyeball, and Fig. 3 is a diagram showing the solid-state image sensor. 4(A) is a system block diagram showing an example of the control device of the visual line detection device, FIG. 4(B) is a block diagram showing details of the visual line calculation processing circuit, Figure 5 (8), CB) is a diagram showing an example of an output waveform from a solid-state image sensor, and Figure 6 is a schematic diagram of a conventional gaze detection device that detects gaze using a corneal reflection image and the center of the pupil. It is. 7... Beam splitter 8... Light emitting lens, 9... Light receiving lens, 10... Infrared LED, 12... Eye (eyeball). Figure 1 Ground sequence〉 Figure 3

Claims (1)

[Scope of Claims] 1. Image detection comprising an illumination means that illuminates the eye, and a solid-state image sensor that detects the Purkinje image position and the image position of other tissues of the eye using the light reflected from the eye illuminated by the illumination means. means, a line-of-sight calculation means for detecting a line-of-sight direction from the relative relationship between the Purkinje image position detected by the image detection means and the image position of other tissues of the eye, and an accumulation time control unit for controlling the accumulation time of the solid-state image sensor. The image detection means has a structure in which a plurality of line sensors for outputting the peak value of image information in real time are arranged in parallel in a direction perpendicular to the optical axis of the reflected light, and the accumulation time control means includes: , causing each line sensor to accumulate images from the start of image accumulation in each line sensor of the image detection means until a peak value of image information reaches a certain value, and detecting the accumulation time for each line sensor; A line of sight detection device that selects image information to be detected by each line sensor based on the detected accumulation time.