JPH0627365A - Line of sight detection device - Google Patents

Abstract

PURPOSE:To provide an appropriate pupil edge detection corresponding to the change in the diameter of a pupil by changing the position of an irradiating light based on the pupil diameter change. CONSTITUTION:An irradiating section 6a irradiates an infrared beam to an eyeball 100 of an observer who observes an object through an optical observation system 1 in an approximate frontal direction and an irradiating section 6b irradiates an infrared beam with a prescribed angle. A photometry section 4 outputs a luminance information which shows the luminance value of the object being observed and when the luminance value is less than a prescribed level, the irradiating section 6a emits a beam for a pupil edge detection of an eyeball picture and when the value is larger than the prescribed level, the irradiating section 6b emits a beam for the same purpose under the control of a control section 5. A photodetector 2 receives a reflected light and outputs an eyeball picture information and a detection computation section 3 detects the line of sight direction of the observer based on the eyeball picture information from the photodetector 2.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a device for detecting the line-of-sight direction of an observer in an optical device, such as a camera, through which an observer looks into a viewfinder, and in particular, irradiates an eye with an infrared ray and reflects it. The present invention relates to a line-of-sight detection device that detects a line-of-sight direction from a first Purkinje image obtained as light and a pupil image.

[0002]

2. Description of the Related Art Conventionally, an apparatus for detecting the line-of-sight direction of an eye to be inspected by using a first Purkinje image has been developed. For example, in Japanese Patent Laid-Open No. 2-5, a first Purkinje image is detected from a pupil edge. An apparatus for detecting a rotation angle of an optometry is disclosed.

There is also an apparatus for detecting the rotation angle of the eye to be inspected from the first Purkinje image and the fourth Purkinje image. However, in this apparatus, the light amount of the fourth Purkinje image is very small.
It is difficult to detect and is not suitable for downsizing of the device. Hereinafter, the case where the edges of the iris and the pupil are detected in the device disclosed in Japanese Patent Laid-Open No. 2-5 will be specifically described.

First, as shown in FIG. 7A, when it is dark, that is, when the amount of light incident on the eye is small, the reflected light of the fundus is relatively high with respect to infrared light, and the pupil diameter becomes large. Since the amount of light reaching the fundus increases, the reflected light level of the fundus becomes higher than the reflected light level of the iris. Therefore, at this time, it is relatively easy to distinguish the pupil image.

On the other hand, as shown in FIGS. 7B and 7C, when the light is bright, that is, when the amount of light incident on the eye is large, the pupil diameter becomes small and the amount of light reaching the fundus decreases. The difference in the reflected light level of the fundus is smaller than that of the reflected light level of the iris. Therefore, at this time, it becomes relatively difficult to distinguish the pupil image. However, this level difference varies depending on the irradiation light irradiation position, the positional relationship between the eye axis and the light receiving system, individual differences, and the like.

As described above, the pupil diameter changes in accordance with the change in the amount of light incident on the eye, and when adopting illumination light from the front, it becomes difficult to detect the pupil edge depending on the brightness condition. .

On the other hand, in Japanese Patent Laid-Open No. 2-209125, for example, by illuminating illumination light from the periphery so that corneal reflected light does not return and the apparent size of the iris is measured, the eyeball is measured. A technique for detecting a rotation angle is disclosed. In this technique, the rotation angle is detected without using the Purkinje image, focusing on the fact that the size of the iris looks small when the eyeball rotates.

In this technique, when the illumination light is emitted from the periphery, the pupil edge can be easily detected even when it is bright. That is, the pupil edge can be easily detected by preventing the reflected light from the fundus from returning and increasing the level difference between the reflected light from the iris and the reflected light from the fundus. Then, as shown in FIG. 7D, in the case of adopting the peripheral illumination, the level difference of the pupil edge becomes large when it is bright.

On the other hand, as shown in FIG.
When peripheral illumination is adopted, when the light is dark, the fundus reflected light is partially high in the pupil image. Therefore, it becomes difficult to accurately detect the edges of the pupil and iris,
When the pupil position and the position of the Purkinje image are used for detection, it causes an error. Incidentally, the above-mentioned FIG. 7 (d), (e)
, The first Purkinje image is variously changed depending on the illumination state and the light receiving state, and is not shown in FIG.

In addition to this, for example, "a method of designing a pupil diameter photographing optical system for detecting a line of sight (The Institute of Electronics, Information and Communication Engineers, Vol.
4-d-II, No.6, pp.736-747, June 1991, by Banno) ”, when the illumination light from the periphery is used when the pupillary system is large, the reflected light from the fundus is kicked by the iris. It is described that there is.

[0011]

As described above, since the pupil diameter changes according to the amount of light incident on the eyeball, for example, when the amount of incident light is small, the pupil diameter becomes large and the edges of the pupil and the iris are accurately measured. It becomes difficult to detect.

The present invention has been made to solve the above problems, and its purpose is to change the irradiation position of the illumination light based on the change of the pupil diameter according to the usage conditions such as brightness. The purpose is to enable proper pupil edge detection corresponding to changes in pupil diameter, thereby improving the accuracy and reliability of line-of-sight detection.

[0013]

In order to achieve the above object, in the visual axis detection device according to the first aspect of the present invention, the visual axis detection device having an observation optical system for observing a target object is used. A first light projecting means for projecting light to the eyeball of an observer who is observing the target object through the system, substantially parallel to the optical axis of the observation optical system; Second light projecting means for projecting light to the eyeball of the observer at a predetermined elevation angle with respect to the optical axis, brightness information measuring means for measuring the brightness of the target object, and measurement by the brightness information measuring means. When the brightness value is smaller than a predetermined value, the first light projecting means is driven, and when the brightness value is larger, the second light projecting means is driven.
And control means for controlling the selection of the second light projecting means, and reflected light from the eyeball of the observer on which the light from the first or second light projecting means selected by the control means is projected. It is characterized by comprising a light receiving means for receiving the light and a calculating means for calculating the line-of-sight direction of the observer based on the eyeball image information from the light receiving means.

Further, in the line-of-sight detection device according to the second aspect, in the line-of-sight detection device having an observation optical system for observing the object of interest, the eyeball of the observer who observes the object through the observation optical system is used. On the other hand, first light projecting means for projecting light substantially parallel to the optical axis of the observation optical system, and light for projecting to the eyeball of the observer at a predetermined elevation angle with respect to the optical axis of the observation optical system. Second light projecting means, brightness information measuring means for measuring the brightness of the target object, and reflected light from the eyes of the observer to which the light from the first and second light projecting means is projected. A light receiving means for receiving light, a storage means for storing eyeball image information from the eyeball by the first light projecting means, and a computing means for computing the line-of-sight direction of the observer based on the eyeball image information of the light receiving means, Where the brightness value measured by the brightness information measuring means is predetermined When it is smaller than the value, the calculating means calculates the line-of-sight direction of the observer based on the eyeball image information stored in the storage means, and when it is larger than a predetermined value, the second light projecting means is driven. And a control means for controlling so as to calculate the line-of-sight direction of the observer based on the eyeball image information.

[0015]

That is, in the line-of-sight detection device according to the first aspect of the present invention, the first light projecting means causes the optical axis of the observation optical system with respect to the eyeball of the observer who is observing the object through the observation optical system. And the second light projecting means projects the light to the eyeball of the observer at a predetermined elevation angle with respect to the optical axis of the observation optical system.
Then, when the brightness information measuring means measures the brightness of the target object, the control means causes the first light projecting means to emit light when the measured brightness value is smaller than a predetermined value, and when the brightness value is larger than the first brightness value. The selection of the light projecting means is controlled so that the second light projecting means is caused to emit light. Further, the light receiving means receives the reflected light from the eyeball of the observer on which the light from the first or second light projecting means selected by the control means is received, and the calculating means is based on the eyeball image information. Then, the observer's line-of-sight direction is calculated.

On the other hand, in the line-of-sight detection apparatus according to the second aspect, the first light projecting means is substantially the same as the optical axis of the observation optical system with respect to the eyeball of the observer who is observing the object through the observation optical system. The light is projected in parallel, and the second light projecting means projects the light on the eyeball of the observer at a predetermined elevation angle with respect to the optical axis of the observation optical system. Then, the brightness information measuring means measures the brightness of the target object, the light receiving means receives the reflected light from the eyeball by the first and second light projecting means, and the storing means by the first light projecting means. The eyeball image information from the eyeball is stored, and the computing means computes the eyeball image information of the light receiving means. Further, when the brightness value measured by the brightness information measuring means is smaller than the predetermined value, the control means calculates the observer's line-of-sight direction based on the eyeball image information stored in the storage means. When it is larger than a predetermined value, the second light projecting means is caused to emit light, and the eye gaze direction of the observer is calculated based on the eyeball image information.

[0017]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is a view showing the arrangement of a visual axis detection device according to the first embodiment of the present invention. In FIG. 1, an observation optical system 1 is an optical system provided for observing a target object, and an illumination unit 6 a includes an eyeball 100 of an observer who observes the object through the observation optical system 1. On the other hand, infrared light is emitted from almost the front through the half mirrors 7a and 7b. Then, the illuminating section 6b is provided with the eyeball 10 of the observer.
Irradiate infrared light at a predetermined angle with respect to 0. Further, the photometric unit 4 outputs the luminance information representing the luminance value of the target object, and the control unit 5 receives the luminance information from the photometric unit 4, and when the luminance value is smaller than a predetermined level, the eyeball image The illumination unit 6a is caused to emit light for detecting a pupil edge, and the illumination unit 6b is caused to emit light for detecting a pupil edge of the eyeball image when the brightness value is higher than a predetermined level. Then, the light receiving unit 2 receives the reflected light from the eyeball 100 of the observer and outputs the eyeball image information, and the detection calculation unit 3 determines the line-of-sight direction of the observer based on the eyeball image information from the light receiving unit 2. To detect.

FIG. 2 is a diagram showing the configuration of a visual axis detection device according to the second embodiment of the present invention. As shown in the figure, the line-of-sight detection device of this embodiment is roughly divided into an observation optical system and a line-of-sight detection optical system.

First, in the observation optical system, the taking lens 101
The flip-up mirror 102 is arranged on the optical path of the light that has passed through, and the focus plate 103, the condenser lens 104, the penta roof prism 106, and the visible light are transmitted on the optical path of the reflected light of the flip-up mirror 102 to transmit the red light. Eyepiece 10 having a dichroic mirror 107a that reflects external light
7 are arranged.

On the other hand, the line-of-sight detection optical system further comprises first and second illumination sections and a light receiving section, and the first illumination section emits infrared light which is a light source insensitive to the photographer. A light emitting diode (LED) 113a and a light projecting lens 1
14a and. And the 2nd illumination part is
It is composed of an LED 113b, which is a light source insensitive to the photographer, and a light projecting lens 114b, and is arranged apart from the finder optical axis.

Further, the light receiving portion is the image sensor 11
2, a light receiving lens 111 and a half mirror 110. As the image sensor 112, a CCD (charge coupled devic) used in a video camera or the like is used.
e) An area sensor is used and the light receiving lens 111
Is arranged such that the image sensor 112 is at a position that is conjugate with a normal eye point, that is, the distance from the viewfinder exit surface to the eyeball. In such a configuration, in the observation optical system, the subject light that has passed through the taking lens 101 is reflected by the flip-up mirror 102 and is reflected by the focusing plate 1.
It is imaged at 03. The subject light diffused by the focusing plate 103 is reflected by the condenser lens 104 and the penta roof prism 106, and the dichroic mirror 107.
Infrared light is removed through the eyepiece lens 107 having a and is guided to the eyeball 100 of the photographer. On the other hand, in the line-of-sight detection optical system, in the first illumination unit, the infrared light emitted from the LED 113a is reflected by the dichroic mirror 107a via the light projecting lens 114a and emitted from the eyepiece lens 107 as substantially parallel light. , The eyeball 100 of the photographer is illuminated.

Further, in the second illumination section, the LED 113
The infrared light emitted from b illuminates the eyeball 100 of the photographer through the light projecting lens 114b. Since the illumination light from the second illumination unit does not emerge in parallel from the eyepiece lens 107, under the condition of this illumination light, the fundus reflection light level when it is bright can be reduced as shown in FIG. .

Then, in the light receiving portion, the eyeball 100 of the photographer is taken.
The eye-reflected light from is reflected by the mirror 107a, and the eye image by the detected infrared light is formed on the image sensor 112 via the light receiving lens 111 and the dichroic mirror 110. Further, the subject light diffused by the focusing plate 103 is guided to the photometric sensor 115 via the condenser lens 104 and the prism 106.

FIG. 3 shows a central processing unit (CPU; Central P).
It is a figure which shows the detailed connection relation of rocessing Unit) 120. In the figure, the CPU 120 has a photometric sensor 115.
The charge accumulation signal and the reading of the image sensor 112 are controlled on the basis of the output of. Here, the image sensor 11
A CCD area sensor used in a video camera or the like is used for 2 and is driven by a drive control unit (not shown) to sequentially read image signals.

Then, by the image sensor 112,
The photoelectrically converted image signal is digitized by a signal processing circuit and an analog / digital (A / D) converter 117, and is stored in the memory 118 as image data. Then, the captured image data can be accessed from the CPU 120.

Further, the photometric sensor 115 is a sensor such as a silicon photodiode, the output of which is transmitted through the interface circuit 116 including a logarithmic compression circuit and the like to the CP.
It is digitized by the A / D converter built in U120 and taken into CPU120. Besides this, CPU
120 includes LEDs 113a, 113b for eyeball illumination,
An operation unit 119 such as a camera switch is connected, and each unit is controlled based on a sequence shown later.

The operation of the visual axis detection device according to the second embodiment will be described below with reference to FIG. When the line-of-sight detection and automatic focusing sequence starts due to interruption such as switch-on (step S100), the photometric sensor 11
5 measures the subject brightness BV (step S101).
Next, this subject brightness BV is compared with a predetermined value BV0,
If BV> BV0, the process proceeds to step S103, and if not, the process proceeds to step S110 (step S102).

In step S103, the infrared light emitting diodes LED113a, b of the illumination light from the surroundings are turned on.
Subsequently, the image sensor 112 is reset to start the light amount integration (step S104).

Since this integration time is substantially determined by the amount of illumination light, the CPU 120 measures the time for a predetermined time by the counter and waits for the completion of the sensor integration (step S105). When the integration is completed, the LEDs 113a and 113b are turned off (step S106).
The sensor signal is read out and the eyeball image signal from the sensor is read
D / D-converted and stored in the memory 118 (step S10).
7). An eyeball image obtained by this ambient light is disclosed in
The configuration is almost the same as that disclosed in Japanese Patent No. 109028,
It is as shown in FIG.

Next, the center position of the first Purkinje image is calculated from the obtained eyeball image (step S108). This first Purkinje image becomes two images by two illumination lights as described in JP-A-3-109028, so the position of each first Purkinje image is obtained and the center position thereof is calculated. This is the Purkinje image center position. Then, the pupil-iris edge position is detected from the image data to obtain the pupil center position, and the process proceeds to step S117 (step S109).

On the other hand, in step S110, the LEDs 113a and b for the illumination light from the optical axis are turned on. Then, the image sensor 112 is reset to start the light amount integration (step S111).

Since this integration time is substantially determined by the amount of illumination light, the CPU 120 counts a predetermined time with a counter,
Wait for the sensor integration to end (step S112), and turn off the LEDs 113a and 113b (step S113).
The sensor signal is read out and the eyeball image signal from the sensor is read
/ D convert and store in memory 118 (step S11)
4).

Next, the center position of the first Purkinje image is calculated from the obtained eyeball image (step S115). At this time, the first Purkinje image obtained is as shown in FIG.
One for each illumination light. Then, the pupil-iris edge position is detected from the image data to find the pupil center position, and the process proceeds to step S117 (step S116).

In step S117, it is determined whether or not the detection is performed, and if it is not detected, the step S1 is performed.
If 01 is detected, if detected, step S118
Proceed to.

Then, in step S118, the eyeball position and the rotation angle are calculated from the obtained first Purkinje image position and the pupil center position, and the line-of-sight direction is calculated (step S11).
8) Thus, one detection is completed (step S11).
9).

In the above operation, as a concrete method for calculating the position of the first Purkinje image, the position of the center of the pupil, and the calculation of the direction of the line of sight, there are specific methods. , And the method disclosed in Japanese Patent Application No. 4-65697 by the present applicant, and the like, the description thereof is omitted.

As described above, in the first embodiment, since the illumination light is switched depending on the brightness, even when the pupil diameter changes depending on the brightness, the above-mentioned problems do not occur and appropriate detection is performed. be able to.

FIG. 5 is a view showing the arrangement of a visual axis detection device according to the third embodiment of the present invention. As shown in the figure, in the third embodiment, the first and second illuminators are LED 11
3a and 3b and light projecting lenses 114a and 114b, which are characterized in that they are arranged at positions apart from the finder optical axis. Then, the LEDs 113a, b
The infrared light emitted from the camera illuminates the photographer's eyeball 100 from a position having a predetermined angle with respect to the finder optical axis via the light projecting lenses 114a and 114b.

Hereinafter, with reference to the flowchart of FIG.
The operation of the third embodiment will be described. When the sequence of line-of-sight detection and automatic focusing is started by interruption such as switch-on (step S200), first, the subject brightness BV is measured by the photometric sensor 115 (step S2).
01).

Then, the infrared light emitting diode LED108 for the illumination light from the optical axis is turned on (step S20).
2) The image sensor 112 is reset to start the light amount integration (step S203).

Since this integration time is substantially determined by the amount of illumination light, the CPU 120 counts a predetermined time with a counter,
Waiting for the sensor integration to end (step S204), and when the sensor integration has ended, turn off the LED 108 (step S205), then read the signal from the sensor, and detect the eyeball image signal from the sensor.
A / D converted and stored in the memory 118 (step S2
06). This data is I1.

Next, the first Purkinje image position is calculated from the obtained eyeball image (step S207). Then, the contrast value of the image data 11 is calculated, and when the contrast value is lower than the predetermined level, it is determined that the time is, for example, fluttering, and the process returns to step S201 (step S201).
208).

Then, the subject brightness BV is compared with a predetermined value BV0 (step S209). If BV> BV0, the process proceeds to step S210, and if not, step S21.
Go to 5.

Next, the infrared light emitting diodes LED113a, b of the illumination light from the surroundings are turned on (step S21).
0), reset the image sensor 112 and start the light amount integration (step S211).

Since this integration time is almost determined by the amount of illumination light, the CPU 120 waits for the completion of sensor integration by counting the time with a counter for a predetermined time (step S212), and turns off the LEDs 113a and 113b (step S21).
3), reading the eyeball image signal from the image sensor 112, A / D converting the eyeball image signal from the sensor, and executing the memory 1
It stores in 18 (step S214). This data I
Set to 2. At this time, the addresses of the data I1 and I2 on the memory are set to be the same, and BV> BV
When it is 0, the image data is rewritten from I1 to I2.

Next, the pupil-iris edge is extracted from the image data on the memory to obtain the pupil center position (step S215). Then, it is determined whether or not the line-of-sight direction has been calculated (step S216). If the line-of-sight direction has not been detected, the process returns to step S201, and if it has been detected, the process proceeds to the next step S217.

Then, the eyeball position and the rotation angle are calculated from the obtained first Purkinje image position and the pupil center position, and the line-of-sight direction is calculated (step S217). In this way, one detection is completed (step S218).

In the above operation, as a specific method for calculating the first Purkinje image position, the pupil center position, and the line-of-sight direction, Japanese Patent Laid-Open Nos. 1-242029 and 2-5 can be used. And JP-A-4-65697.
The description is omitted because it can be realized by using the method disclosed in the publication.

As described above in detail, the visual axis detecting device of the present invention has the illuminating means for illuminating the eyeball from the front and the illuminating means for illuminating the surroundings, and the illuminating means according to the brightness of the ambient light. By switching between, it is possible to accurately detect the line-of-sight direction of the observer regardless of the level of ambient light and brightness.

[0051]

According to the present invention, by changing the irradiation position of the illumination light based on the change of the pupil diameter according to the use condition such as the brightness, it is possible to detect the pupil edge appropriately corresponding to the change of the pupil diameter. As a result, it is possible to provide a line-of-sight detection device with improved line-of-sight detection accuracy and reliability.

[Brief description of drawings]

FIG. 1 is a diagram showing a configuration of a visual line detection device according to a first exemplary embodiment of the present invention.

FIG. 2 is a diagram showing a configuration of a visual line detection device according to a second exemplary embodiment of the present invention.

FIG. 3 is a diagram showing a detailed connection relationship of a CPU 120.

FIG. 4 is a flowchart showing the operation of the second embodiment.

FIG. 5 is a diagram showing a configuration of a visual line detection device according to a third exemplary embodiment of the present invention.

FIG. 6 is a flowchart showing the operation of the third embodiment.

7A to 7E are diagrams showing a state of reflected light from an eyeball of an observer.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Observation optical system, 2 ... Light receiving part, 3 ... Detection calculation part, 4 ... Photometric part, 5 ... Control part, 6a, 6b ... Illumination part, 7a, 7b ...
Half mirror.

Claims (2)

[Claims] 1. A line-of-sight detection apparatus having an observation optical system for observing a target object, wherein the observation optical system is for an eyeball of an observer who is observing the target object via the observation optical system. And a second light projecting means for projecting light to the eyeball of the observer at a predetermined elevation angle with respect to the optical axis of the observation optical system. A luminance information measuring means for measuring the luminance of the target object, and driving the first light projecting means when the luminance value measured by the luminance information measuring means is smaller than a predetermined value. Control means for controlling selection of the first and second light projecting means so as to drive the second light projecting means when large, and the first or second light projecting means selected by the control means. From the eyeball of the above observer when the light from Light receiving means for receiving light, the line-of-sight detection apparatus characterized by comprising a calculating means for calculating a viewing direction of the observer based on the eyeball image information from said light receiving means. 2. A line-of-sight detection apparatus having an observation optical system for observing a target object, wherein an optical axis of the observation optical system is provided with respect to an eyeball of an observer who observes the object through the observation optical system. A first light projecting means for projecting light substantially in parallel, a second light projecting means for projecting light onto an eyeball of an observer at a predetermined elevation angle with respect to the optical axis of the observation optical system, and the above object Brightness information measuring means for measuring the brightness of the object, and light receiving means for receiving the reflected light from the eyeball of the observer on which the light from the first and second light projecting means is projected; The storage means for storing the eyeball image information from the eyeball by the light projecting means, the computing means for computing the line-of-sight direction of the observer based on the eyeball image information of the light receiving means, and the brightness measured by the brightness information measuring means. When the value is smaller than the predetermined value, the storage means The eye gaze direction of the observer is calculated by the calculating means based on the stored eyeball image information, and when it is larger than a predetermined value, the second light projecting means is driven, and the observer is obtained by the eyeball image information. And a control means for controlling so as to calculate the line-of-sight direction of the line-of-sight detection device.