JP2003339642A - Visual line detector and optical apparatus having the same - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To detect a visual line by preventing glass reflection light by eyeball illumination light from disturbing the eyeball image of a photographing person though the photographing person wears various glasses or looks into a finder system in various manners. <P>SOLUTION: This detector comprises: an illumination means for illuminating the eyeball of an observing person; a means for receiving the reflected light of the illuminated eyeball; and a means for detecting the line of sight from obtained eyeball information. The illumination means comprises a plurality of sets of light emitting elements for emitting light by a pair of the elements and has an illumination selection means for selecting an optimal pair of light emitting elements from the plurality of pairs of the light emitting elements of the illumination means from a positional relation of an eyeball position obtained from the eyeball information and the position of reflected light from glasses which the observing person wears. <P>COPYRIGHT: (C)2004,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an optical device having a line-of-sight detection device for detecting the line of sight of a photographer who is observing a subject, and for example, the observer looks into the finder field of an optical device and observes it. It is suitable for selecting a symbol representing the function of the optical device displayed in the finder using the line-of-sight information of the person and executing the function by selecting the symbol with the line-of-sight information. Specifically, the focus detection may be performed by selecting a plurality of focus detection areas displayed in the viewfinder field of the camera using the line-of-sight information of the photographer.

[0002]

2. Description of the Related Art Conventionally, a photographer's (observer's) line-of-sight direction is detected to determine which region (position) the photographer's view field is within.
Or the so-called gaze direction of the photographer is detected by the line-of-sight detection means provided in a part of the camera, and various shooting functions such as automatic focus adjustment and automatic exposure are controlled based on the signal from the line-of-sight detection means. Various cameras designed to do so have been proposed.

[0003] For example, the applicant of the present invention has filed Japanese Patent Application Laid-Open No. 1-24151
Japanese Patent No. 1 has a line-of-sight detection unit for detecting the gaze direction of the photographer and an automatic exposure control unit having a plurality of photometric sensitivity distributions, and at this time, a focus detection unit based on an output signal from the line-of-sight detection unit. And a camera adapted to control the drive of the automatic exposure control means.

A major problem in actually detecting the line of sight is when the photographer wears glasses.

In order to detect the line of sight, a photographer's eyes are illuminated by a light emitting element. The illumination light is reflected by the surface of the glasses worn by the photographer and taken into the camera. There is a problem in that the Purkinje image and the image of the pupil, which are originally necessary for line-of-sight detection, overlap with each other on the photographed eyeball image of the photographer, and as a result the line-of-sight cannot be detected considerably for non-spectacle wearers. It was

On the other hand, in Japanese Unexamined Patent Publication No. 6-86758, an eyeball of a photographer who looks into the finder system is detected based on a signal from a spectacle detecting means for detecting whether or not the photographer is wearing spectacles. There has been proposed a visual axis detection device characterized in that a predetermined light emitting element among a plurality of light emitting elements for illuminating the eye is emitted.

Specifically, as shown in FIG. 11, light emitting elements 13p, 13q, 13r for illuminating an eyeball are provided below an eyepiece lens through which a photographer of a camera looks into a finder for photographing.
13s is arranged to detect whether or not the photographer wears glasses from the eyeglass reflected light of the light emitting element light in the eyeball image information of the photographer, and when the photographer does not wear the glasses, A pair of light emitting elements of 13p and 13q with a narrow interval between
If the photographer wears glasses, the
The light emitting elements of the set of r and 13s are used.

12 (a) to 12 (c) show the effect of switching the position of the light emitting element.

FIG. 12 (a) shows an eyeball image of the photographer who normally illuminates the light emitting elements 13p and 13q with a narrow interval while the photographer of the camera is not wearing glasses.
Therefore, it is possible to clearly detect the corneal reflection image of the pupil and the light emitting element, which is necessary for detecting the line of sight of the photographer, and the so-called Purkinje image.

On the other hand, FIG. 12 (b) is an eyeball image obtained by illuminating the light emitting elements 13p and 13q when the photographer wears spectacles, and the spectacle surface in front of the photographer's eyes. 2 shows a state in which the light of the light emitting element reflected by is overlapped with the pupil and the Purkinje image, and the line of sight of the photographer cannot be detected or malfunctioned.

Next, the light emitting elements are arranged at wide intervals 13r and 13s.
FIG. 12C shows an eyeball image when the eyeballs are illuminated by changing to, and the position where the spectacle reflected light is generated is widened due to the widening of the light emitting element spacing. This indicates that the photographer's line of sight can be detected without hindering the eyeball image of the photographer by this action.

[0012]

DISCLOSURE OF THE INVENTION Problems to be Solved by the Invention JP-A-6-867
When the photographer detects a naked eye state based on the signal from the eyeglasses detecting means as proposed in Japanese Patent Laid-Open No. 58-58, the photographer uses a pair of light emitting elements with a narrow interval to wear the glasses. When the wearing state is detected, the suggestion to use a set of light emitting elements with a wide interval is that when the spectacle wearer directly faces the finder optical system of the camera, the generation position of the spectacle reflected light is changed from the eyeball image position. It had the effect of keeping it away, and it was very effective.

However, due to uncertain causes such as the type of spectacles (lens shape) of the photographer, how to wear the spectacles, and how to look into the finder system vary from person to person, the above-mentioned proposal alone is enough to obtain the eyeball image of the photographer. It was not possible to reliably separate the reflected light from the glasses.

Therefore, an object of the present invention is to prevent a photographer from wearing various spectacles or looking into the finder system without causing the reflected light of the spectacles to disturb the eyeball image of the photographer. An object of the present invention is to provide a line-of-sight detection device that can accurately detect the line-of-sight and an optical device including the same.

[0015]

In order to solve the above-mentioned problems, the present invention according to claim 1 provides an illumination means for illuminating the vicinity of an eyeball of an observer and a reflected light of the eyeball illuminated by the illumination means. It has a light receiving means for receiving light and a visual axis detecting means for calculating the visual axis position of the observer from the eyeball image information obtained by the light receiving means, and the illuminating means illuminates the eyeball of the observer in pairs. A plurality of light emitting elements for performing light emission for, the plurality of the illumination means from the positional relationship between the eyeball image position obtained from the eyeball image information of the photographer and the reflected light position from the glasses worn by the photographer. In order to solve the above problems, the present invention provides a line-of-sight detection device characterized by having an illumination selection means for selecting an optimum set of light-emitting elements from the set of light-emitting elements and an optical device having the same. is there.

[0016]

BEST MODE FOR CARRYING OUT THE INVENTION The present invention will now be described in detail based on the illustrated embodiments.

FIG. 1 is a schematic diagram of a main part showing a first embodiment of the present invention when applied to a single-lens reflex camera, and FIG.
(A), (B) is a figure which shows the upper surface and back surface of the single lens reflex camera of FIG. 1, and FIG. 3 is a figure which shows the inside of the viewfinder of the single lens reflex camera of FIG.

In these figures, 1 is a taking lens,
Although shown as two lenses for convenience, it is actually composed of a larger number of lenses. Reference numeral 2 denotes a main mirror, which is obliquely installed or retreated in the photographing optical path according to the observation state of the subject image by the finder system and the photographing state of the subject image. Reference numeral 3 denotes a sub-mirror, which reflects the light flux transmitted through the main mirror 2 toward a focus detection device 6 described below below the camera body.

A shutter 4 and a photosensitive member 5 are formed of a silver salt film, a solid-state image pickup device such as CCD or MOS type, or an image pickup tube such as a vidicon.

Reference numeral 6 denotes a focus detection device, which includes a field lens 6a and reflection mirrors 6b and 6 arranged near the image plane.
c, a secondary imaging lens 6d, a diaphragm 6e, a line sensor 6f including a plurality of CCDs, and the like.

The focus detection device 6 in this example performs focus detection by a well-known phase difference method, and as shown in FIG. 3, a plurality of regions (focus detection region marks 70) in the object scene are detected.
5 to 74 are set as focus detection areas (focus detection areas), and the focus detection areas can be detected.

Reference numeral 7 is a focusing plate arranged on the planned image forming surface of the taking lens 1, and 8 is a pentaprism for changing the finder optical path. Reference numerals 9 and 10 respectively denote an imaging lens and a photometric sensor for measuring the brightness of the subject within the observation screen, and the imaging lens 9 conjugates the focusing plate 7 and the photometric sensor 10 via the reflection optical path in the pentaprism 8. Have a relationship.

Reference numeral 11 is an eyepiece lens 11 provided behind the exit of the pentaprism 8 and having a light splitter 11a, and is used for observing the focusing plate 7 by the eye 15 of the photographer. The light splitter 11a includes, for example, a dichroic mirror that transmits visible light and reflects infrared light.

A finder optical system is constituted by the main mirror 2, the focusing plate 7, the penta prism 8 and the eyepiece lens 11 described above.

Reference numeral 12 is an image forming lens, and 14 is an image sensor (CCD-EYE) in which a photoelectric conversion element array such as a CCD is arranged two-dimensionally with vertical and horizontal 80 pixels and 100 pixels, respectively.
The imaging lens 12 and the image sensor 14 (CCD-EYE) are arranged so as to be conjugate with the vicinity of the pupil of the eyeball 15 of the photographer at a predetermined position with respect to the imaging lens 12.
Constitutes a light receiving means for detecting the line of sight. 1
Reference numerals 3a to 13h denote illumination means each including a light emitting element for illuminating the vicinity of the pupil of the eyeball 15 of the photographer.

Infrared light emitting diodes (hereinafter referred to as IREDs) are used for these light emitting elements, which are arranged around the eyepiece lens 11 as shown in FIG. It is a set of two of 13a to 13h that emits light.

As described above, the line-of-sight detecting device is constituted by the light receiving means, the illuminating means, and the diloic mirror 11a.

Reference numeral 21 is a high-intensity superimposing LED that can be visually recognized even in a bright subject. The light emitted from this LED is reflected by the main mirror 2 through the projection prism 22 and is displayed on the display portion of the focus plate 7. It is bent in the vertical direction by the provided micro prism array 7a, passes through the penta prism 8 and the eyepiece lens 11, and reaches the eye 15 of the photographer.

That is, as can be seen from the viewfinder field shown in FIG. 3, each of the focus detection area marks 70 to 74 shines in the viewfinder field and the focus detection area can be displayed (hereinafter, this will be described). Is called the superimpose display).

In FIG. 3, dot marks 70 'and 74' are engraved inside the left and right focus detection area marks 70 and 74, which are for correcting the line-of-sight detection error due to individual differences of the eyeball. It shows an optotype at the time of collecting line-of-sight correction data (referred to as calibration).

Here, 51 is a shutter speed display, 52 is a segment of aperture value display, 50 is a line-of-sight input mark indicating that the line-of-sight is being input, and 53 is a focus mark indicating the focus state of the taking lens 1. . Reference numeral 24 denotes an LCD in the finder (hereinafter,
F-LCD), and is illuminated by the LED 25 for illumination.

The light transmitted through the F-LCD 24 is guided by the triangular prism 26 to the outside of the field of view of the finder as shown at 24 in FIG. 3, and the photographer can know various photographing information.

Returning to FIG. 1, 31 is an aperture provided in the taking lens 1, 32 is an aperture drive device including the aperture drive circuit 111, 33 is a lens drive motor, and 34 is a lens drive member including a drive gear and the like. is there. Reference numeral 35 is a photocoupler, which detects the rotation of the pulse plate 36 interlocked with the lens driving member 34 and transmits it to the lens focus adjustment circuit 110. The focus adjustment circuit 110 uses this information and the lens drive amount from the camera side. The lens driving motor 33 is driven by a predetermined amount based on the above information to move the taking lens 1 to the in-focus position. A mount contact 37 serves as an interface between a known camera and lens.

Reference numeral 27 denotes an attitude detection switch such as a mercury switch for detecting whether the camera is held in the horizontal position or the vertical position.

In FIGS. 2A and 2B, reference numeral 41 is a release button. Reference numeral 42 denotes a monitor LCD as an external monitor display device, which comprises a fixed segment display section for displaying a predetermined pattern and a 7-segment display section for displaying variable numerical values.

Reference numeral 44 is a mode dial for selecting the photographing mode and the like. The shooting mode is set according to the display content by matching the index 43 engraved on the camera body with the display. For example, a lock position that deactivates the camera, a position in an automatic shooting mode controlled by a preset shooting program, a manual shooting mode in which the photographer can recognize the shooting content, and program AE, shutter priority AE, aperture priority AE Shooting modes of subject depth priority AE and manual exposure can be set. Further, the "CAL" position for line-of-sight input is also in the mode dial 44, and by setting the "CAL" position and operating the electronic dial 45 described later, ON / OFF of line-of-sight input, and execution and selection of calibration. It can be performed.

Reference numeral 45 is an electronic dial for selecting a set value that can be further selected from among the modes selected by the mode dial 44 by rotating and generating a click pulse. Is. For example, when the shutter priority shooting mode is selected by the mode dial 44, the currently set shutter speed is displayed on the in-viewfinder LCD 24 and the monitor LCD 42. Looking at this display, the photographer uses the electronic dial 45
When is rotated, the shutter speed is sequentially changed from the currently set shutter speed according to the rotation direction.

The other operating members are not directly related to the present invention, and therefore their explanations are omitted.

FIG. 4 is a block diagram showing an electrical structure incorporated in the single-lens reflex camera having the above-mentioned structure. The same parts as those in FIG. 1 are designated by the same reference numerals.

A central processing unit (hereinafter referred to as CPU) 100 of a microcomputer incorporated in the camera body includes a visual axis detection circuit 101, a photometric circuit 102, an automatic focus detection circuit 103, a signal input circuit 104, an LCD drive circuit 1
05, LED drive circuit 106, IRED drive circuit 10
7, a shutter control circuit 108, and a motor control circuit 109 are connected. Further, signals are transmitted to the focus adjustment circuit 110 and the diaphragm drive circuit 111 arranged in the photographing lens 1 through the mount contact 37 shown in FIG.

EEPROM 10 attached to CPU 100
Reference numeral 0a has a function of storing the line-of-sight correction data as a storage unit for correcting individual differences in line-of-sight.

The line-of-sight detection circuit 101 A / D receives an eyeball image signal from the image sensor 14 (CCD-EYE).
The image information is converted and transmitted to the CPU 100. CPU
Reference numeral 100 extracts each feature point of the eyeball image necessary for detecting the line of sight according to a predetermined algorithm, and further calculates the line of sight of the photographer from the position of each feature point.

The photometric circuit 102, after amplifying the signal from the photometric sensor 10, performs logarithmic compression and A / D conversion, and transmits it to the CPU 100 as luminance information of each sensor. The photometric sensor 10 is an SP for photometry of six areas in the viewfinder screen.
C-L2, SPC-L1, SPC-C, SPC-R1,
It is composed of six photodiodes consisting of SPC-R2 and SPC-M, and so-called split photometry is possible.

The line sensor 6f includes five sets of line sensors CCD-L2, CCD-L1, CC corresponding to the five focus detection areas 70 to 74 in the screen shown in FIG.
It is a known CCD line sensor composed of DC, CCD-R1, and CCD-R2.

The automatic focus detection circuit 103 A / D converts the voltage obtained from the line sensor 6f, and the CPU 1
Send to 00O.

SW1 is a switch that is turned on by the first stroke of the release button 41 to start photometry, AF, line-of-sight detection operation, etc., and SW2 is a release switch that is turned on by the second stroke of the release button 41.

SW-DIAL and SW-DIAL2 are
The dial switch provided in the electronic dial 45, which has already been described, is input to the up / down counter of the signal input circuit 104 to count the rotation click amount of the electronic dial 45. SW-HV1 and SW-HV2 are line-of-sight detection switches corresponding to the attitude detection switch 27, and the attitude state of the camera is detected based on this signal.

State signal input circuit 104 for these switches
To the CPU 100 via the data bus.

The LCD driving circuit 105 has a known structure for driving the LCD which is a liquid crystal display element, and according to a signal from the CPU 100, an aperture value,
At the shutter speed, the display of the set shooting mode and the like can be simultaneously displayed on both the monitor LCD 42 and the in-viewfinder LCD (F-LCD) 24.

The LED driving circuit 106 is a lighting LE.
D (F-LED) 25 and LED2 for superimposing
1 is turned on and off. The IRED drive circuit 107
Is the infrared light emitting diode 1 according to the instruction of the CPU 100.
Lighting power is controlled by selectively lighting 3a to 13h or changing the output current value (or pulse number) to the infrared light emitting diodes 13a to 13h.

The shutter control circuit 108 controls the magnet MG-1 that runs the front curtain and the magnet MG-2 that runs the rear curtain when energized to expose the photosensitive member with a predetermined amount of light. The motor control circuit 109 includes a motor Ml for winding and rewinding the film and the main mirror 2
And for controlling the motor M2 that charges the shutter 4.

The shutter control circuit 108 and the motor control circuit 109 perform a series of camera release sequence operations.

Next, the operation of the camera having the visual axis detection device will be described with reference to the flowchart of FIG.

When the mode dial 44 is rotated to set the camera from the inoperative state to the predetermined photographing mode (this embodiment will be described based on the case where the shutter priority AE is set), the power of the camera is turned off. ON (step #
100), variables used for visual axis detection other than the visual axis calibration data stored in the EEPROM 100a of the CPU 100 are reset (step # 10).
1). Then, the camera waits until the release button 41 is pressed and the switch SW1 is turned on (step #
102).

When the release button 41 is pressed, the switch S
When the signal input circuit 104 detects that the W1 is turned on, the CPU 100 confirms with the visual axis detection circuit 101 which calibration data to use when performing visual axis detection (step # 103). At this time, if the calibration data of the confirmed calibration data number remains unchanged from the initial value or is set to the line-of-sight prohibition mode (step # 1
03), without performing the visual axis detection, that is, without using the visual axis information, a specific focus detection area is selected by the focus detection area automatic selection subroutine (step # 115). Then, in this focus detection area, the automatic focus detection circuit 103
Performs focus detection operation (step # 107).

Although several methods are conceivable as the algorithm for automatically selecting the focus detection area, the near point priority algorithm which weights the central focus detection area, which is well known in the multi-point AF camera, is effective.

When it is recognized that the sight line calibration data corresponding to the calibration data number is set to a predetermined value and the data is input by the photographer (step # 10).
3) The line-of-sight detection circuit 101 executes line-of-sight detection according to the calibration data, and the line-of-sight is converted into the gazing point coordinates on the focusing plate 7 (step # 104).

Next, the line-of-sight information detected in the above step # 104 is subjected to a judgment as to whether or not it is successful (step #
105). The determination conditions here are the reliability of the Purkinje image, which is a corneal reflection image, the center position of the pupil, the rotation angle of the eyeball, and the like. As a result, if unsuccessful, the process proceeds to the "focus detection region automatic selection subroutine" in step # 115. Also,
If the line-of-sight detection is successful, the CPU 100 selects a focus detection area close to the gaze point coordinates (step # 10).
6). Then, the automatic focus detection circuit 103 also executes focus detection in the selected focus detection area using the detected line-of-sight information (step # 107).

Next, it is determined whether or not the focus detection area selected is incapable of focus detection (step # 108). If not, the CPU 100 sends a signal to the LCD drive circuit 105 to bring the LCD 24 in the finder into focus. The mark 53 blinks to warn the photographer that the focus detection is NG (step # 117), and this warning is continued until the switch SW1 is released (step # 118 → # 117 → # 118).
......).

On the other hand, when focus detection is possible and the focus adjustment state of the focus detection area selected by the predetermined algorithm is not in focus (step # 109), the CPU 100 sends a signal to the lens focus adjustment circuit 110. The photographing lens 1 is driven by a predetermined amount (step # 116). After driving the lens, the automatic focus detection circuit 103 performs focus detection again (step # 107), and determines whether or not the taking lens 1 is in focus (step # 109).

Photographing lens 1 in a predetermined focus detection area
If the focus is on, the CPU 100 sends a signal to the LCD drive circuit 105 to turn on the focus mark 53 of the LCD 24 in the finder and the LED drive circuit 106.
Also, by sending a signal, the superimposing LED 21 corresponding to the focus detection area in focus is turned on, and the focus detection area is illuminated to display the focus (step # 1).
10).

When the photographer sees that the focused focus detection area is displayed in the viewfinder, the photographer sees that the focus detection area is not correct or determines that the shooting is to be stopped and the release button 4 is pressed.
When the switch SW1 is turned off (step # 111) by releasing the hand from 1, the camera continues to turn on the switch SW1.
The process returns to step # 102 where the process waits until N is given.

If the photographer looks at the in-focus displayed focus detection area and continues to turn on the switch SW1 (step # 111), the CPU 100 causes the photometry circuit 102 to operate.
To send a signal to perform photometry (step # 11
2).

At this time, the exposure value is calculated by weighting the photometric area including the focused focus detection area. In the case of the present embodiment, the aperture value (F5.6 shown by 52 in FIG. 3) is displayed using the segment and the decimal point as the photometric calculation result.

Further, it is determined whether or not the switch SW2 is turned on by pressing the release button 41 (step # 113), and if the switch SW2 is in the off state, the state of the switch SW1 is checked again (step # 113).
111).

When the switch SW2 is turned on, the CPU 100 sends signals to the shutter control circuit 108, the motor control circuit 109, and the diaphragm drive circuit 111 to perform the known shutter release operation (step # 114). ).

Specifically, first, the motor control circuit 109
Then, the motor M2 is energized to raise the main mirror 2 and the diaphragm 31 is narrowed down, and then the magnet MG-1 is energized to open the front curtain of the shutter 4. The aperture value of the aperture 31 and the shutter speed of the shutter 4 are determined by the photometric circuit 1 described above.
It is determined from the exposure value detected in 02 and the sensitivity of the film 5. After a predetermined shutter time (1/125 seconds),
The magnet MG-2 is energized to close the rear curtain of the shutter 4. When the exposure on the film 5 is completed, the motor M2 is energized again to perform mirror down and shutter charge, and the film feeding motor Ml is energized to advance the film frame, and a series of shutter release sequence operations. Ends.

After that, the switch SW1 of the camera is turned on again.
Wait until it is done (step # 102).

FIG. 6 is a flow chart showing the visual axis detection operation algorithm.

In FIG. 6, the line-of-sight detection circuit 101 executes line-of-sight detection upon receiving a signal from the CPU 100 as described above (step # 104). Line-of-sight detection circuit 101
Determines whether the sight line is detected in the photographing mode or the sight line is detected in the sight line calibration mode (step # 200).

In practice, when the mode dial 44 of FIG. 2 is set to the calibration mode for the visual axis detection operation, the calibration (CA
L) The operation (step # 300) is executed.

There is a line-of-sight detection mode setting on the mode dial 44, and calibration data numbers 1, 2, and 3 and line-of-sight detection that enable registration and execution of calibration data for three persons are executed at this set position. Not O
The FF has a total of four positions set by the electronic dial 4 in FIG.
It is possible by the operation of 5.

Therefore, when the camera is not set to the calibration mode, the line-of-sight detection circuit 101 recognizes which calibration data number the camera is currently set to.

Subsequently, in the case of detecting the line of sight in the photographing mode, the line-of-sight detection circuit 101 first detects the posture of the camera through the signal input circuit 104 (step # 201). The signal input circuit 104 is shown in FIG.
The output signal of the inside mercury switch 27 (SW-ANG) is processed to determine whether the camera is in the horizontal position or the vertical position, and in the case of the vertical position, for example, whether the release button 41 is in the top direction or the ground (surface). ) Judge whether it is in the direction.

Next, an infrared light emitting diode (hereinafter referred to as IRED) 13 is obtained from the previously detected camera posture information and the photographer's eyeglass information included in the calibration data.
A to 13h are selected (step # 202).

That is, if the camera is held in the horizontal position and the photographer does not wear glasses, the IREDs 13a and 13b shown in FIG. 2B are selected. If the camera is in the horizontal position and the photographer wears glasses, the IRED 13a,
IREDs 13c and 13d having a wider spacing than the spacing of 13b are selected.

Further, if the camera is, for example, the release button 41 being in the top direction or in the vertical direction in the ground (plane) direction, a combination of IREDs for illuminating the photographer's eyes from below, that is, photographing If the person does not wear glasses, 13c and 13g are selected, and if the photographer wears glasses, the combination of 13a and 13e is selected. The IRED selection operation performed here is referred to as an IRED selection operation 1 and is distinguished from an IRED selection operation 2 performed during a calibration operation described later, which is the content of the present invention.

Next, the image sensor 14 (hereinafter, CCD-
(Referred to as EYE) and the illumination power of the IRED is set based on the image signal output of the preliminary accumulation performed prior to the main accumulation of the CCD-EYE 14 or the information as to whether or not the eyeglasses are attached (step # 203). ).

CCD-EYE14 preliminary storage means
A stable eye image can be obtained by controlling the accumulation time of the actual eyeball image capture depending on the strength of the signal level by always capturing the image signal for a fixed time, for example, 1 ms, just before the main accumulation. It becomes possible to obtain a signal.

CCD-EYE 14 accumulation time and IRE
When the illumination power of D is set, the CPU 100
The IRED 13 is turned on with a predetermined power via the D drive circuit 107, and the line-of-sight detection circuit 101 is a CCD-
The accumulation of EYE14 is started (step # 204). In addition, the accumulation here is completed according to the previously set accumulation time of the CCD-EYE 14, and along with that, the IRED
13 is also turned off.

Next, the image signals accumulated in the CCD-EYE 14 are sequentially read out, and the line-of-sight detection circuit 101 performs A / D conversion.
After the conversion, it is stored in the CPU 100 (step # 205).

FIG. 13 shows an image of the eyeball image signal of the CCD-EYE 14, and the cornea 16 of the eyeball 15 in FIG.
In this case, if the IREDs 13a and 13b emit light, corneal reflection images (hereinafter referred to as Purkinje images) 19a and 19b shown in FIGS. 13 and 14 are generated. Further, 17 is an iris and 18 is a pupil.

A known line-of-sight detection process is performed on these image signals (step # 206).

That is, the CPU 100 detects the positions of the Purkinje images 19a and 19b, which are virtual images of the pair of IREDs 13a and 13b used to illuminate the eyeball.

As described above, Purkinje images 19a and 19b
Appears as a bright spot having a high light intensity, and can be detected by providing a predetermined threshold value for the light intensity and making a light intensity exceeding the threshold value a Purkinje image. Further, the center position of the pupil is calculated by detecting a plurality of boundary points between the pupil 18 and the iris 17 and performing least square approximation of a circle based on each boundary point. The rotation angle θ of the eyeball is obtained from the Purkinje image position and the pupil center position, and the eyepiece 1 of the camera is calculated from the distance between the two Purkinje images 19.
1 and the distance between the photographer's eyeball 15 are calculated, and the CCD-E
The imaging magnification β of the eyeball image projected on the YE 14 can be obtained.

From the above, the position coordinates on the focusing plate 7 in the line-of-sight direction of the photographer can be obtained using the rotation angle θ of the eyeball, the imaging magnification β, and the individual difference correction information obtained by the calibration. You can

Returning to FIG. 5, next, the reliability of the line-of-sight information is determined based on the line-of-sight position of the photographer (step # 105).

Next, the calibration operation algorithm will be described with reference to the flowchart of FIG.

As described above, the calibration is performed by the photographer holding the right end 74 'and the left end 70' of the dot mark in the focus detection area in the finder field for a certain period of time, and the eyeball image obtained therefrom. The line-of-sight correction data is collected from the data, and the calibration operation is started by setting the mode dial 44 to the "CAL" position (step # 30).
0).

First, the posture of the camera is detected through the output signal of the mercury switch 27 (SW-ANG) and the signal input circuit 104 (step #
301). This is step # 20 in the eye gaze detection flow of FIG.
The detection process is the same as 1.

Next, the right end 741 of the dot mark in the focus detection area within the finder field is displayed in a blinking manner to display the target to be held by the photographer (step # 302).

At the same time, the calibration data stored in the CPU 100 is confirmed from the currently set calibration number, and if already registered, the "CAL" display on the monitor LCD 42 in FIG. If it is not registered, the "CAL" display blinks.

Subsequently, the camera selects the IRED which illuminates the eyeball of the photographer during the calibration operation 2
Is performed (step # 303). This routine will be described later.

In step # 303, I which should emit light
The RED is determined, the camera waits for the ON signal of the photographer's SW1, and when the SW1-ON signal is detected, the same eyeball image capturing operation as in steps # 203 to # 205 described in the flowchart of FIG. 6 is performed.

That is, the accumulation time of the CCD-EYE 14 and the illumination power of the IRED 13 are set (step # 30).
4) Actually accumulation of CCD-EYE14 and IRED13
Illumination is performed (step # 305), and CCD-EY
The image signals accumulated at E14 are sequentially read out and AD-converted and then stored in the CPU 100 (step # 30).
6). Next, the CPU 100 calculates the rotation angle θ of the eyeball of the photographer according to a predetermined calculation formula for the AD-converted image signal on the memory (step # 307).

During the eyeball image capturing operation, the blinking of the right end 74 'of the dot mark in the focus detection area in the finder field changes to a lighting display, indicating that the photographer is in the middle of performing the eyeball image capturing operation. I am informed.

The rotation angle θ calculated subsequently is used to judge whether the value is proper (step # 30).
8).

Since the deviation between the optical axis of the eyeball and the visual axis rarely shifts by several tens of degrees in the biological sense, the threshold for determination is set to ± 10 degrees here. In step # 308,
Whether or not the detected rotation angles OK and NG of the eyeball are determined, and whether the result is OK or NG, the next step #
If the total number of eyeball rotation angle detections is less than 10 times, the process returns to step # 304, the eyeball image capturing operation is performed again, and the total number of eyeball rotation angle detections is 10 times.
When the number of times reaches 10, the calibration (CAL) success or failure is determined depending on how many times OK occurs out of the 10 times (step # 310).

Here, the rotation angle detection succeeds six times or more, and the CAL at the right end 74 'succeeds (step # 311).
Subsequently, this time, the calibration operation at the left end 70 'is started. When the calibration at the left end 70 'is similarly successful, the "CAL" display on the monitor LCD 42 is turned on and the calibration data is stored in the CPU 10.
It is stored in 0. If the calibration data has already been registered, the newly acquired calibration data is integrated with the stored past data. If the number of successful rotation angle detections is less than 6, the CAL fails (step # 312), the "CAL" display on the monitor LCD 42 is replaced by the blinking display, and the photographer is informed that the calibration has failed. .

Next, the IRED selection operation 2 (step # 303) in the calibration operation flowchart of FIG. 7 will be described with reference to the flowchart of FIG.

Here, in order to simplify the description, a case where the calibration operation is performed for the first time when the camera is held at the normal position will be described as an example. The arrangement of the IREDs 13a to 13h, which are the illumination means used in this description, is as shown in FIG.

In the IRED selection operation 2, first, in order to perform the first light emission operation of the IRED 13 for illuminating the photographer's eyeball, the IRED corresponding to the normal position and orientation of the camera
13a and 13b are selectively emitted (step # 40
1). The image signal obtained by the CCD-EYE 14 at the time of this light emission is AD-converted as described above, and the CPU 10
Signal processing is performed at 0. Here, as described in the section of the conventional example, if the photographer does not wear glasses, the pupil 1 of the photographer as shown in FIG. 13 or FIG.
8, the iris 17, and Purkinje images 19a and 19b can be easily detected. However, when the photographer wears spectacles, the illumination light emitted by the IRED 13 is reflected by the spectacles front surface (including the back surface), and a large and strong signal is included in the eyeball image signal as shown in FIG. 12B. Spawn a level glasses ghost.

This ghost is the IRED 13a, 13b
If the distance between the two is relatively small as shown in Fig. 3, the probability that the position of occurrence is near the photographer's pupil and Purkinje's image is high, and this obstructs the eyeball image information, resulting in detection of the line of sight (eyeball rotation angle). (Including detection) will be the cause.

Therefore, the reflected light of the glasses is detected from the image signal in the state of FIG. 12B, and it is determined whether or not the photographer wears the glasses (step # 402). This determination can be easily performed because the signal corresponding to the eyeglass reflected light is extremely strong and its range is large compared to other pixel signals.

In step # 402, there is no light reflected from the glasses,
That is, when it is determined that the photographer does not wear glasses, the process proceeds to step # 409, and the illumination I that is the initial setting is set.
RED 13a, 13b determines.

On the other hand, when it is determined that the photographer wears spectacles because there is reflected light from the spectacles, the selective emission of the IREDs 13c and 13d, which is wider than the currently selected mutual spacing of the IREDs 13a and 13b. Is performed (step # 403). The image signal received by the CCD-EYE 14 at this time is shown in FIG. In FIG. 9A, IRED
13c 'and 13c are reflected by the glasses 13c and 13d.
It can be seen that d'occurs, and particularly 13c 'obstructs the image of the pupil and the Purkinje image necessary for calculating the eyeball rotation angle. If the IREDs 13c and 13d emit light, FIG.
If the image signal of (c) is obtained, there is no problem in calculating the eyeball rotation angle. However, depending on how the photographer looks through the viewfinder, the type of eyeglasses worn, especially the shape, The state of 9 (a) frequently occurs.

Now, returning to FIG. 8 again, in the state of FIG. 9A, the distance between the pupil and the reflected light of the spectacles is measured from the image signal by CP.
U100 calculates (step # 404). Actually, it is highly possible that the boundary edge between the pupil and the iris cannot be accurately detected due to the influence of the reflected light from the eyeglasses, so here it is possible to consider the lowest value of all pixel signals as the pupil center. Becomes In addition, all pixels of the image signal in the preliminary accumulation performed before the main accumulation of the CCD-EYE 14 are 80 × 10 in all pixels.
By converting 0 into a block pixel signal of 4 × 4 pixels of 20 × 25 pixels in the vertical and horizontal directions, it is possible to obtain the pupil center position which is the lowest luminance signal block pixel and the eyeglass reflected light position which is the highest luminance signal block pixel. It will be easy. The pupil center and the spectacle reflected light 13c 'of FIG.
Is s (pixel).

Next, the s value is compared with a predetermined value (step # 405). If the s value is a predetermined value or more, it can be considered that there is no influence on the detection of the center of the pupil and the Purkinje image. For example, a value of 30 pixels is set here. If the determination result shows that the s value is 30 pixels or more, the process proceeds to step # 409, and the currently selected IRED13
c and 13d are determined.

On the other hand, as shown in FIG. 9A, the s value is 3
When the number of pixels is less than 0, the IRED1s are arranged at the same intervals as the IREDs 13c and 13d and at different positions.
3g and 13h are selectively emitted (step # 406).

FIG. 9 shows image signals received by the CCD-EYE 14 when the IREDs 13g and 13h are selectively emitted.
It shows in (b).

Subsequently, in step # 404, the distance between the pupil and the reflected light of the spectacles is determined from the image signal by the CPU 1 as performed.
00 is calculated (step # 404). The distance calculated here is defined as t (pixel).

Next, step # 404 and step # 40
The distance between each pupil and the reflected light from the eyeglasses obtained in 7.
s and t are compared (step # 408), and the combination of IREDs, which is the illumination condition having the larger value (pixel), is selected (step # 409). The values s and t from the schematic FIGS. 9A and 9B used in this description are s.
Since <t, the IRED to be determined is 13 g, 1
It will be 3h.

Further, in this description, the flow in the case where the calibration operation is performed for the first time has been shown.
In practice, in order to obtain the calibration correction values under various environments, a method of performing the calibration with the same calibration number and performing weighted addition averaging of the obtained correction values in a new order is adopted. In this case, the illumination (IRED) that should be emitted in the already registered calibration data, specifically, the illumination for the naked eye (IRED 13a and 13b in the normal position) or the illumination for glasses (13c and 13d in the normal position, or 13g and 13h)
Since the data regarding whether or not to use is included, the process starts from step # 401 for the naked eye illumination and step # 403 for the eyeglass illumination.

(Second Embodiment) A second embodiment of the present invention will be described with reference to the flowchart of FIG.

This description is to improve the visual axis detection algorithm of step # 104 in the flowchart of FIG.

Here, from step # 500 to step #
Up to 506, step # 20 in the flowchart of FIG.
Since the processing is exactly the same as that from 0 to step # 206, the description will be omitted.

After the line-of-sight detection process is performed in step # 506, it is judged whether or not the calculated line-of-sight information is successful (step # 507). If it is determined that the visual axis detection is successful, the focus detection area is selected in step # 106 of FIG. 5 based on the calculated visual axis information. On the other hand, when it is determined that the line-of-sight detection is unsuccessful, it is determined how many times the present unsuccessfulness is (step # 508).

If the previous time was unsuccessful and the current time is also unsuccessful, the line-of-sight cannot be detected completely, and the camera shifts to the focus detection area automatic selection operation of step # 115 in FIG. On the other hand, if the line-of-sight detection is unsuccessful this time for the first time, the type of illumination performed in step # 504 is first determined (step # 509). If this is an illumination for the naked eye, the previous step # 50 is performed in order to change the condition for line-of-sight detection again.
The accumulation time of the CCD-EYE 14 performed in step 4 is extended 1.5 times and set (step # 510). Then, the process returns to step # 504 again to execute the line-of-sight detection operation.
This is an effective measure when the photographer looks away from the camera and looks into the viewfinder for some reason.

On the other hand, if the illumination for the glasses is currently being performed in step # 509, then in step # 511, I
The RED selection operation 3 is executed.

Since this IRED selection operation 3 is almost the same as the IRED selection operation 2 described in the flowchart of FIG. 8, the illustration thereof is omitted, but steps # 401 to # 403 of the flowchart of FIG. 8 are omitted. The flow chart shows that the eyeglass reflected light distance measurement performed in step # 404 is performed on the image information obtained in step # 504 in FIG. 10, and the eyeglass illumination 2 light emission performed in step # 406 in FIG. Is step # 504 in FIG.
A set of IREDs that are arranged at the same intervals as the set of IREDs that emits light at 1 and different positions are used. That is, the light emitted in step # 504 is 13c, 13
If it is the set of d, the spectacle illumination 2 light emission performed in step # 406 of FIG. 8 will be the set of 13g and 13h. Emission of these IRED13, CCD-EYE14
The image signal processing obtained by accumulating can be performed at high speed by processing the block signal of 4 × 4 pixels.

In this way, the reflected light from the glasses of the two types of IRED illuminations is compared, and the result is shown in step # of FIG.
A determination is made at 512.

That is, when it is determined that the effect is not obtained even if the illumination for glasses performed in step # 504 is changed, it is determined that it is not due to the influence of the reflected light from the glasses, and step #
The process proceeds to 510 and the accumulation time is changed. On the other hand, if it is determined that changing the spectacle lighting is effective, then IRED
Group is changed, the process proceeds to step # 504 again, and the second line-of-sight detection is performed.

(Modification) The present invention has been described as applied to a single-lens reflex camera, but can also be applied to cameras such as a lens shutter camera and a video camera. Furthermore, it can be applied to other optical devices, other devices, and constituent units.

[0124]

According to the present invention, the illumination means for illuminating the vicinity of the eyeball of the observer, the light receiving means for receiving the reflected light of the eyeball illuminated by the illumination means, and the eyeball image information obtained by the light receiving means are used. There is a line-of-sight detection means for calculating the line-of-sight position of the observer, the illumination means is composed of a plurality of sets of light-emitting elements for emitting light to illuminate the eyes of the observer in pairs, the eyeball of the photographer From the positional relationship between the eyeball image position obtained from the image information and the position of the reflected light from the spectacles worn by the photographer, the optimum light emitting element set is selected from the plurality of light emitting element sets of the illumination means. By providing a line-of-sight detection device having an illumination selection unit and an optical device having the same, when an observer wears eyeglasses and looks through a viewfinder of an optical device such as a camera, A light to illuminate the eyeball Light is reflected by the observer's eyeglasses can the reflected light can be prevented from interfering with the eyeball image for detecting the line of sight of the observer, enabling accurate sight line detection of the observer.

[Brief description of drawings]

FIG. 1 is a schematic view of a single-lens reflex camera.

FIG. 2 is a top view of the single-lens reflex camera and a rear view of the single-lens reflex camera.

FIG. 3 is a view showing shooting information in a viewfinder field.

FIG. 4 is an electric circuit diagram of the camera.

FIG. 5 is a flowchart of camera operation.

FIG. 6 is a flowchart of a line-of-sight detection operation.

FIG. 7 is a flowchart of eye-gaze detection calibration.

FIG. 8 is a flowchart of IRED selection operation 2.

FIG. 9 is an explanatory diagram of effects of the present invention.

FIG. 10 is a flowchart for explaining the second embodiment.

FIG. 11 is an explanatory diagram of a conventional IRED layout.

FIG. 12 is an explanatory diagram of an eyeball image of a conventional example.

FIG. 13 is an explanatory diagram of a gaze detection principle.

FIG. 14 is an explanatory diagram of a gaze detection principle.

[Explanation of symbols]

1 Shooting lens 2 primary mirror 6 Focus detection device 6f image sensor 7 focus plate 10 Photometric sensor 11 eyepiece 13 Infrared light emitting diode (IRED) 14 Image sensor (CCD-EYE) 15 eyeballs 16 cornea 17 Iris 18 pupil 27 Mercury switch 31 aperture 41 Release button 44 Mode dial 45 electronic dial 50 Line-of-sight input mark 53 Focus mark 70-74 Focus detection area mark 100 CPU 100 line-of-sight detection circuit 103 Focus detection circuit 104 signal input circuit 105 LCD drive circuit 106 LED drive circuit 107 IRED drive circuit 110 Focus adjustment circuit

Claims (3)

[Claims] 1. An illuminating means for illuminating the vicinity of an observer's eyeball, a light receiving means for receiving reflected light of the eyeball illuminated by the illuminating means, and an eye line of the observer from eyeball image information obtained by the light receiving means. It has a line-of-sight detecting means for calculating the position, the illuminating means is composed of a plurality of light emitting elements for emitting light to illuminate the eyeball of the observer in pairs, and is obtained from eyeball image information of the observer. Illumination selection means for selecting an optimal set of light emitting elements from a set of a plurality of light sources of the illumination means from the positional relationship between the eyeball image position and the position of reflected light from the eyeglasses worn by the observer. An eye-gaze detecting device having the same and an optical instrument having the same. 2. The illumination selection means operates during a correction value sampling operation for correcting the difference between the optical axis of the eyeball and the visual axis of the observer, so-called calibration, and the light emission selected by the illumination selection means. The line-of-sight detection device according to claim 1, wherein the position of the set of elements is stored together with the correction value of the observer, and an optical apparatus having the same. 3. The illumination selecting means operates when it is determined that the line-of-sight detecting operation for calculating the line-of-sight position of the observer is unsuccessful, and in the set of light-emitting elements newly selected by the illumination selecting means, The line-of-sight detection device according to claim 1, wherein the line-of-sight detection is performed again, and an optical apparatus including the line-of-sight detection device.