JP2002301033A - Sight line detecter - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a sight line detecter enhancing the accuracy of by using a proper sight-line correcting means to automatically correct the mis-detection of the sight lines caused by differences in eyeball size among individuals. SOLUTION: The sight line detecter includes a determining means for determining if an examee is wearing eyeglasses, and an illuminating means for illuminating the examee eyeballs from a first position if the determining means determines that the examee is wearing eyeglasses and for illuminating the eyeballs from a second position if the determining means determines that the examee is not wearing eyeglasses. A corneal reflex image formed on the eyeball illuminated by the illuminating means is used to detect the examee's sight line.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a gaze detecting apparatus for detecting a gaze direction of an observer from a reflected image obtained by illuminating an observer's eyeball.

[0002]

2. Description of the Related Art Conventionally, various devices (for example, eye cameras) for detecting a so-called line of sight (a visual axis) for detecting which position on an observation surface the observer is observing have been provided.

For example, in Japanese Patent Application Laid-Open No. 1-274736, a parallel light beam from a light source is projected to the anterior segment of an observer's eyeball, and a corneal reflection image formed by light reflected from the cornea and an image forming position of a pupil are used. Looking for the visual axis.

[0004] The present applicant has proposed an optical device having a visual axis detection device in which Japanese Patent Application No. 3-11492 has calibrated the visual axis to correct individual differences in the visual axis of the observer.

FIG. 25 is a view for explaining the principle of a known line-of-sight detection method. In the figure, reference numerals 13a and 13b denote light sources such as light-emitting diodes which emit infrared light insensitive to an observer, and the light sources 13a and 13b are substantially symmetric in the x direction with respect to the optical axis of the light receiving lens 12. It is arranged and divergently illuminates the eyeball 15 of the observer. A part of the illumination light reflected by the eyeball 15 is condensed on the image sensor 14 by the light receiving lens 12.

FIG. 24A is a schematic diagram of an eyeball image projected on the image sensor 14, and FIG. 24B is a diagram showing the intensity of an output signal from the image sensor 14 in FIG. The gaze detection method will be described below with reference to the drawings.

[0007] The infrared light emitted from the light source 13b illuminates the cornea 16 of the eyeball 15 of the observer. At this time, a corneal reflection image d (virtual image) formed by a part of the infrared light reflected on the surface of the cornea 16 is condensed by the light receiving lens 12 and forms an image at a position d ′ on the image sensor 14.

Similarly, infrared light emitted from the light source 13a illuminates the cornea 16 of the eyeball 15. At this time, a corneal reflection image e formed by a part of the infrared light reflected on the surface of the cornea 16 is condensed by the light receiving lens 12 and forms an image at a position e ′ on the image sensor 14. Also, the end a of the iris 17,
The light beam from b forms an image of the ends a and b at positions a ′ and b ′ on the image sensor 14 via the light receiving lens 12.

If the rotation angle θ of the optical axis of the eyeball 15 with respect to the optical axis of the light receiving lens 12 is small, the x coordinates of the ends a and b of the iris 17 are xa and xb. Is expressed as xc ≒ (xa, xb) / 2.

The x coordinate of the midpoint between the corneal reflection images d and e substantially coincides with the x coordinate xo of the curvature center O of the cornea 16.
For this reason, the x coordinate of the corneal reflection image generation positions d and e is x
Assuming that a standard distance between d, xe and the center of curvature O of the cornea 16 and the center C of the pupil 19 is OC, and that a coefficient (gaze correction coefficient) considering individual differences with respect to the distance OC is A,
The rotation angle θ of the optical axis 15a substantially satisfies the relational expression of (A * OC) * SIN θ ≒ xc- (xd + x) / 2 ‥‥‥ (1).

Therefore, as shown in FIG. 24, the position of each characteristic point (corneal reflection images d and e and the end portions a and b of the iris) of the eye 15 projected on the image sensor 14 is detected to detect the position of the eye. The rotation angle θ of the 15 optical axes 15a can be obtained. At this time, the expression (1) can be rewritten as β * (A * OC) * SINθ ≒ (xa ′ + xb ′) / 2− (xd ′ + xe ′) / 2 (2) Here, β is a magnification determined by the position of the eyeball 15 with respect to the light receiving lens 12, and is substantially obtained as a function of the interval | xd'-xe '| of the corneal reflection image.

The rotation angle θ of the optical axis of the eyeball 15 is rewritten as follows: θ {ARCSIN} (xc′−xf ′) / β / (A * OC)} (3) However, xc ′ ≒ (xa ′ + xb ′) / 2 xf ′ ≒ (xd ′ + xe ′) / 2 Since the optical axis 15a of the observer's eye 15 does not coincide with the visual axis, the optical axis of the observer's eyeball Horizontal rotation angle θ
Is calculated, the horizontal line of sight θx of the photographer is obtained by correcting the angle difference α between the optical axis of the eyeball and the visual axis.

[0013] Assuming that a coefficient (gaze correction coefficient) considering individual differences with respect to the correction angle α between the optical axis and the visual axis of the eyeball is B, the horizontal gaze θx of the observer is θx = θ ± (B * α) ‥‥‥ (4) is required. Here, as for the sign ±, if the rotation angle to the right with respect to the observer is positive, the sign + is selected when the observer's eyes are left eyes, except for the observation apparatus (finder system), and the sign-is selected when the observer's eyes are right eyes.

In FIG. 1, the eyeball of the observer is z-
Although an example in which the eye rotates in an x plane (for example, a horizontal plane) is shown, the detection can be similarly performed when the eyeball of the observer rotates in a yz plane (for example, a vertical plane). However,
Since the vertical component of the observer's line of sight coincides with the vertical component θ 'of the optical axis of the eyeball, the vertical line of sight θy is θy = θ'.

Further, when a single-lens reflex camera is used as the optical device, the position (xn, yn) on the focusing plate viewed by the observer from the line-of-sight data θx, θy is xn ≒ m * θx ≒ m * [ ARCSIN {(xc′−xf ′) / β / (A * OC)} ± (B * α)] {(5) ynεm * θy

Here, m is a constant determined by the finder optical system of the camera. Here, since there are two coefficients A and B for correcting the individual difference of the line of sight, for example, when the observer sees two optotypes at different positions, the rotation angle of the observer's eyeball calculated at that time is calculated. The coefficients A and B can be obtained.

Also, coefficients A and B for correcting individual differences in the line of sight
Usually corresponds to the horizontal rotation of the observer's eyeball, so that the two optotypes arranged in the viewfinder of the camera are set to be horizontal to the observer.

When the coefficients A and B for correcting the individual difference of the line of sight are obtained, and the position of the line of sight of the observer looking into the finder system of the camera is calculated using the equation (5), the line of sight information is obtained. It can be used for adjusting the focus of a lens or controlling exposure.

[0019]

The camera having the line-of-sight detection function described in the above-mentioned conventional example has the following problems.

When the observer wears spectacles and illuminates the eyeball from the same position as that of the naked eye, light is reflected on the surface of the spectacles, and a reflection image (ghost image) different from the corneal reflection image appears on the image sensor. ) Occurs, which may hinder gaze detection.

[0021]

In order to solve the above-mentioned problems, the invention according to claim 1 of the present application is a judging means for judging whether or not the user wears spectacles, and wearing the spectacles by the judging means. If it is determined that the eyeball is illuminated from the first position, and if the determination unit determines that the eyeglasses are not worn, the illumination illuminates the eyeball from a second position different from the first position. Means for detecting a line of sight using a corneal reflection image generated in an eyeball when illuminated by the illuminating unit.

According to the above configuration, when the observer wears eyeglasses, the eyeball is illuminated from a position different from that of the naked eye, so that it can be proved from a position where light is not reflected on the surface of the eyeglasses. Ghosts that hinder detection do not appear on the image sensor.

[0023]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS FIG. 1 is a schematic view of a main part of an embodiment when the present invention is applied to a single-lens reflex camera, and FIGS.
(B) is a top external view and a rear view of the single-lens reflex camera of FIG. 1,
FIG. 3 is a view of the viewfinder in FIG.

In FIG. 1, reference numeral 1 denotes a photographing lens, which is shown by two lenses 1a and 1b for convenience in FIG. 1, but is actually composed of a large number of lenses. Reference numeral 2 denotes a main mirror which is inclined or retreated to a photographing optical path according to an observation state and a photographing state. A sub-mirror 3 reflects a light beam transmitted through the main mirror 2 toward the lower side of the camera body. 4 is a shutter,
5 is a photosensitive member, which is a silver halide film or a CCD or MOS.
It consists of a solid-state imaging device such as a mold or an imaging tube such as a vidicon.

Reference numeral 6 denotes a focus detection device, which is a field lens 6a, reflection mirrors 6b and 6 disposed near the image plane.
c, a well-known phase difference system including a secondary imaging lens 6d, an aperture 6e, a line sensor 6f including a plurality of CCDs, and the like. As shown in FIG. 3, the focus detection device 6 in FIG. 3 is configured to be able to perform focus detection on a plurality of regions (five distance measurement point marks 200 to 204) in the observation screen 213.

Reference numeral 7 denotes a focusing plate disposed on a predetermined image forming plane of the photographing lens 1, 8 denotes a pentaprism for changing the optical path of a finder, 9 and 10 denote an image forming lens and a photometer for measuring the luminance of a subject in an observation screen. In the sensor, the imaging lens 9 conjugately connects the focusing plate 7 and the photometric sensor 10 via a reflection optical path in the penta roof prism 8.

An eyepiece 11 having a light splitter 11a is arranged behind the exit surface of the penta roof prism 8, and is used for observing the focus plate 7 with the eye 15 of the photographer. The light splitter 11a is composed of, for example, a dichroic mirror that transmits visible light and reflects infrared light. Reference numeral 12 denotes a light receiving lens, and reference numeral 14 denotes an image sensor having a two-dimensional array of photoelectric elements such as CCDs, which are arranged so as to be conjugate with the vicinity of the pupil of the photographer's eye 15 at a predetermined position with respect to the light receiving lens 12. .

The image sensor 14 and the light receiving lens 12 constitute one element of the light receiving means. 13, 13a-13
f is an infrared light emitting diode, which is an illumination light source (light emitting means) for the photographer's eye 15, and is arranged around the eyepiece 11 as shown in FIG.

Reference numeral 21 denotes a high-brightness superimposing LED that can be visually recognized even in a bright subject. The light emitted from the LED for superimposition is the light projection prism 2
2. The light is reflected by the main mirror 2 and vertically bent by the micro prism array 7a provided on the display unit of the focus plate 7, and reaches the photographer's eye 15 through the pentaprism 8 and the eyepiece 11. Therefore, the micro prism array 7a is formed in a frame shape at a position corresponding to the focus detection area of the focus plate 7, and the super prism array 7a is provided with five superimposing LEDs 2 corresponding to each.
1 (each LED-L1, LED-L2, LED-C,
LED-R1, LED-R2).

Accordingly, as can be seen from the viewfinder view shown in FIG.
01, 202, 203, 204 are within the viewfinder field 2
13 illuminates and a focus detection area (distance measurement point) is displayed (hereinafter, this is referred to as a superimposed display).

Here, distance measuring point marks 200, 20 at the left and right ends
4, dot marks 205 and 206 are engraved, and collect eye-gaze correction data (eye-gaze correction coefficients) A and B for correcting gaze detection errors due to individual differences of the eyeballs as described later. (Hereinafter, this operation is referred to as calibration).

Reference numeral 23 denotes a field mask for forming a field of view in the viewfinder, and 24 denotes an LCD in the viewfinder for displaying photographing information outside the field of view of the viewfinder.
-LED) 25. The light transmitted through the LCD 24 in the finder is guided into the finder by the triangular prism 26, and is outside the finder field of view 2 in FIG.
07, the photographer is observing the photographing information.
Reference numeral 27 denotes a posture detecting unit, which is a mercury switch for detecting the posture of the camera.

Reference numeral 31 denotes an aperture provided in the taking lens 1;
A diaphragm driving device including a diaphragm driving circuit 111 described later;
Reference numeral 3 denotes a lens driving motor, reference numeral 34 denotes a lens driving member including a driving gear and the like, and reference numeral 35 denotes a photocoupler which detects rotation of a pulse plate 36 interlocked with the lens driving member 34 and transmits the rotation to a lens focus adjustment circuit 110. The lens focus adjustment circuit 110 drives the lens drive motor by a predetermined amount based on this information and the information on the lens drive amount from the camera side,
The focusing lens 1a of the taking lens 1 is moved to a focusing position. Reference numeral 37 denotes a mount contact which serves as an interface between a known camera and a lens.

In FIG. 2, reference numeral 41 denotes a release button, and reference numeral 42 denotes a fixed segment display unit 4 for displaying a predetermined pattern on a monitor LCD as an external monitor display device.
2a and a 7-segment display section 42b for displaying variable numerical values. Reference numeral 43 denotes an AE lock button for holding a photometric value, and reference numeral 44 denotes a mode dial for selecting a photographing mode or the like. Other operation members are not particularly necessary for understanding the present invention, and therefore will be omitted.

FIG. 4A is a detailed explanatory view of the mode dial 44 of FIG. The mode dial 44 sets the shooting mode based on the display contents by matching the display with the index 55 marked on the camera body. 44a is a lock position for disabling the camera, 44b is a position for an automatic photographing mode controlled by a photographing program preset by the camera, and 44c is a manual photographing mode in which a photographer can set photographing contents. AE, aperture priority AE, depth of field priority AE, and manual exposure are provided. Reference numeral 44d denotes a “CAL” position in a calibration mode for performing a line-of-sight calibration described later.

FIG. 4B is an explanatory diagram of the internal structure of the mode dial 44. Reference numeral 46 denotes a flexible printed circuit board which is a switch pattern (M
11, M21, M31, M41) and the GND pattern are arranged as shown in the figure, and four contact pieces (47) of a switch contact piece 47 interlocked with the rotation of the mode dial 44.
By sliding a, 47b, 47c, 47d), 13 positions shown on the mode dial 44 can be set with 4 bits.

In FIG. 2A, reference numeral 45 denotes an electronic dial for selecting a set value which can be further selected from among the modes selected by the mode dial by rotating and generating a click pulse. For example, when a shooting mode with shutter priority is selected by the mode dial 44, the LCD 24 in the viewfinder and the LCD for monitoring are selected.
In 42, the currently set shutter speed is displayed. When the photographer rotates the electronic dial 45, the shutter speed is sequentially changed from the currently set shutter speed according to the rotation direction.

FIGS. 5A and 5B show this electronic dial 4.
FIG. 5 is a detailed view showing an internal structure of No. 5; Electronic dial 45
A click plate 48 that rotates together with the printed circuit board 49 is disposed, and a printed circuit board 49 is fixed to the click plate 48. Printed circuit board 49
Has a switch pattern 49a (SWDIAL-1), 4
9b (SWDIAL-2) and the GND pattern 49c are arranged as shown, and three sliding contact pieces 50a,
The switch contact piece 50 having 50b and 50c is a fixing member 51.
It is fixed to.

A click ball 52 that fits into a concave portion 48 a formed in the outer peripheral portion of the click plate 48 is disposed, and a coil spring 53 that biases the click ball 52 is held by a fixed member 51. In the normal position (the state where the click ball 52 is stuck in the concave portion 48a), the sliding contact pieces 50a and 50b do not contact either of the switch patterns 49a and 49b.

The electronic dial 4 thus formed
5, when the photographer rotates the electronic dial 45 clockwise in FIG. 5, the sliding contact 50b first contacts the switch pattern 49b, and then the sliding contact 50b.
The set value is counted up at this timing such that a contacts the switch pattern 49a. In the case of counterclockwise rotation, the relationship between the sliding contact and the switch pattern is exactly the opposite, and the set value is counted down at the same timing.

FIG. 5B is a timing chart showing this state, and shows pulse signals generated in the switch patterns 49a and 49b when the dial is rotated and their timings. The upper part shows the case of one click rotation in the clockwise direction, and the lower part shows the case of the rotation in the counterclockwise direction. In this way, the timing of count-up and the rotation direction are detected.

FIG. 6 is a block diagram of a main part of an electric circuit built in the camera body of the present embodiment. 5, the same components as those in FIG. 1 are denoted by the same reference numerals.

A microcomputer central processing unit (hereinafter referred to as a CP) as a line-of-sight correction means built in the camera body.
U) 100 includes a line-of-sight detection circuit 101, a photometry circuit 102,
Automatic focus detection circuit 103, signal input circuit 104, LCD
A drive circuit 105, an LED drive circuit 106, an IRED drive circuit 107, a shutter control circuit 108, and a motor control circuit 109 are connected. Further, signals are transmitted to the focus adjustment circuit 110 and the aperture driving circuit 111 arranged in the taking lens via the mount contact 37 shown in FIG.

The EEPROM 100a as storage means attached to the CPU 100 has a function of storing eye-gaze correction data for correcting individual differences in gaze. Mode dial 44
When the "CAL" position is adjusted to the index, a calibration mode for acquiring gaze correction data (hereinafter, referred to as calibration data) for correcting individual differences in gaze becomes selectable, and the calibration mode corresponding to each calibration data can be selected. The selection of the calibration number, the setting of the calibration operation “OFF”, and the setting of the line-of-sight detection prohibition mode can be performed by the electronic dial 45. Calibration data can be set multiple times, can be distinguished by the person who uses the camera, when the same user has different observation conditions, for example, when using glasses, when not using glasses, or when using a diopter correction lens This is effective for setting the time to be used separately from the time when it is not used. The calibration number selected at this time or the state of the set line-of-sight inhibition mode is also the calibration data number (1, 2, 3) as described later.
(Or 0) is stored in the EEPROM 100a.

The line-of-sight detection circuit 101 A / D converts the output of the eyeball image from the image sensor 14 (CCD-EYE), and transmits this image information to the CPU 100. CPU10
0 extracts each feature point of the eyeball image necessary for gaze detection in accordance with a predetermined algorithm, as described later, and further calculates the gaze of the photographer from the position of each feature point. The CPU 100, the line-of-sight detection circuit 101, and the image sensor 14 constitute one element of the line-of-sight detection device.

The photometric circuit 102 amplifies the output from the photometric sensor 10, performs logarithmic compression and A / D conversion, and sends the result to the CPU 100 as luminance information of each sensor. The photometry sensor 10 is located at the left ranging point 2 in the viewfinder visual field shown in FIG.
SPC for photometry of the left area 210 including 00 and 201
L, SPC-C for metering a central area 211 including a center ranging point 202, SPC-R for metering a right area 212 including right ranging points 203 and 204, and metering of these peripheral areas 213. And four photodiodes SPC-A.

As described above, the line sensor 6f includes five sets of line sensors CCD-L2, CCD-L1, CCD-C, and C corresponding to the five ranging points 200 to 204 on the screen.
A known CCD composed of a CD-R1 and a CCD-R2
It is a line sensor. The automatic focus detection circuit 103 A / D converts the voltage obtained from the line sensor 6f, and
Send to PU100.

SW-1 is turned on by the first stroke of the release button 41, and starts a photometry, AF, and line-of-sight detection operation. SW-2 is turned on by the second stroke of the release button.
N-release switch, SW-ANG is a posture detection switch detected by the mercury switch 27, S
W-AEL is turned on by pressing AE lock button 43
AE lock switches, SW-DIAL1 and SW-D
IAL2 is input to the up / down counter of the signal input circuit 104 by a dial switch provided in the electronic dial 45 described above, and counts the amount of rotation click of the electronic dial 45. SW-M11 to SW41 are also dial switches provided in the mode dial already described.

The signals of these switches are applied to the signal input circuit 10
4 and transmitted to the CPU 100 via the data bus. Reference numeral 105 denotes a well-known LCD driving circuit for driving the liquid crystal display element LCD, and displays a display of an aperture value, a shutter time, a set photographing mode, and the like according to a signal from the CPU 100 and the monitor LCD 42 and the LC in the viewfinder.
D24 is displayed at the same time.

The LED drive circuit 106 includes an illumination LED (F
(LED) 22 and the LED 21 for superimpose are controlled to be turned on and off. The IRED drive circuit 107 selectively turns on the infrared light emitting diodes (IRED1 to 6) 13a to 13f according to the situation. Shutter control circuit 108
Is a magnet MG-1 that runs the front curtain when energized,
The magnet MG-2 for moving the rear curtain is controlled to expose a predetermined amount of light to the photosensitive member.

The motor control circuit 109 controls a motor M1 for winding and rewinding the film and a motor M2 for charging the main mirror 2 and the shutter 4. The shutter control circuit 108 and the motor control circuit 109 operate a series of camera release sequences.

FIGS. 7A and 7B show a monitor LCD 4.
FIG. 2 is an explanatory diagram showing the contents of all display segments of the LCD 24 in the finder. In FIG. 7A, in addition to a known shooting mode display,
A gaze input mode display 61 is provided to indicate that gaze detection is performed and a shooting operation such as selection of a camera AF operation or a shooting mode is controlled using gaze information.

The 7-segment section 42b for displaying a variable numerical value includes a 4-digit 7-segment 62 for displaying the shutter time, a 2-digit 7-segment 63 for displaying the aperture value, and a decimal point 64;
It is composed of a limited numerical value display segment 65 for displaying the number of films and a one-digit seven segment 66.

In FIG. 7B, reference numeral 71 denotes a camera shake warning mark, reference numeral 72 denotes an AE lock mark, reference numerals 73, 74 and 75 denote the same display segments as the shutter time display and aperture value display, and reference numeral 76 denotes an exposure correction setting mark. , 77 are a flash full mark, 78 is a line-of-sight input mark indicating that the line of sight is being input, and 79 is a focus mark that indicates the state of focus of the taking lens 1.

Next, FIG. 8 shows a flowchart of the operation of the camera having the eye-gaze detecting device, and FIGS. 15 and 16 show the display state in the finder at this time. The following description is based on these drawings.

When the mode dial 44 is rotated to set the camera from a non-operation state to a predetermined shooting mode (this embodiment will be described based on the case where the shutter priority AE is set), the camera is turned on. (# 100), C
Variables used for line-of-sight detection other than line-of-sight calibration data stored in the EEPROM of the PU 100 are reset (# 101).

The camera stands by until the release button 41 is pressed and the switch SW1 is turned on (# 10).
2). Release button 41 is pushed in and switch SW1 is turned O
When the signal input circuit 104 detects that N
U100 confirms with the gaze detection circuit 101 which calibration data to use when performing gaze detection (# 103).

At this time, if the calibration data of the confirmed calibration data number is unchanged from the initial value or is set to the line-of-sight prohibition mode, the line-of-sight detection is not executed. AF point automatic selection subroutine (# 11
A specific ranging point is selected according to 6). At this distance measuring point, the automatic focus detection circuit 103 performs a focus detection operation (# 107). Several methods can be considered as an algorithm for automatic selection of a ranging point. A near-point priority algorithm in which a weight is assigned to the central ranging point is effective, and an example thereof is shown in FIG. 9 and described later.

When the line-of-sight calibration data corresponding to the calibration data number is set to a predetermined value and it is recognized that the data is input by the photographer, the line-of-sight detection circuit 101
Performs eye gaze detection in accordance with the calibration data (# 104). At this time, the LED drive circuit 106 turns on the illumination LED (F-LED) 25, the LCD drive circuit 105 turns on the line of sight input mark 78 of the LCD 24 in the finder, and the photographer detects the line of sight outside the viewfinder 207 with the camera. It is possible to confirm that the operation is being performed (see FIG. 15).
(A)).

The set shutter time is displayed in the 7 segment 73 (1/3 in this embodiment).
A case of 250 second shutter priority AE is shown).
Here, the line of sight detected by the line of sight detection circuit 101 is converted into coordinates of a gazing point on the focus plate 7. The CPU 100 selects a focus detection point close to the coordinates of the gazing point, and
6 to make the distance measuring point mark blink using the superimpose LED 21 (# 10).
5).

FIGS. 15A and 15C show a state in which the distance measuring point mark 201 is selected as an example.
At this time, when the reliability of the gazing point coordinates detected by the eye-gaze detecting circuit 101 is low, the CPU 100 sends a signal to change the number of ranging points selected according to the degree of the reliability and display the signals. Sending.

In FIG. 15B, the reliability of the gazing point is lower than in the state of FIG.
Indicates a state where is selected. The photographer sees that the focus detection point selected by the photographer's line of sight is displayed, recognizes that the focus detection point is incorrect, and releases the release button 41.
Release switch SW1 and turn off switch SW1 (# 10
6), the camera waits until the switch SW1 is turned on (# 102).

After the photographer has displayed the distance measuring point selected by his / her line of sight, the switch SW1 is subsequently turned on.
If it continues (# 106), the automatic focus detection circuit 103
Performs focus detection at one or more distance measuring points using the detected line-of-sight information (# 107). Here, it is determined whether or not the selected ranging point cannot be measured (# 108). If not, the CPU 100 sends a signal to the LCD drive circuit 105 to blink the focusing mark 79 of the LCD 24 in the finder, The photographer is warned that the distance measurement is NG (impossible) (# 118), and is continued until the switch SW1 is released (# 119).

If distance measurement is possible and the focus adjustment state of the distance measurement point selected by a predetermined algorithm is not in focus (#
109), the CPU 100 sends a signal to the lens focus adjustment circuit 110 to drive the focusing lens 1a of the photographing lens 1 by a predetermined amount (# 117). After driving the lens, the automatic focus detection circuit 103 performs focus detection again (# 107), and determines whether or not the photographing lens 1 is in focus (# 10).
9). If the photographing lens 1 is in focus at a predetermined distance measuring point, the CPU 100 sends a signal to the LCD drive circuit 105 to turn on the focus mark 79 of the LCD 24 in the finder and also sends a signal to the LED drive circuit 106. The focus point 201 is displayed in focus by sending the focus point 201 (#
110) (FIG. 16A).

At this time, the blinking display of the focusing point selected by the line of sight is turned off, but the focusing point displayed in focus often coincides with the ranging point selected by the line of sight. The focusing distance measuring point is set to a lighting state in order to make the photographer recognize that focusing has been achieved. The photographer observes that the focused ranging point is displayed in the viewfinder, recognizes that the ranging point is not correct, releases the release button 41, and turns off the switch SW1 (# 111). The camera waits until the switch SW1 is turned on (# 10
2).

If the photographer looks at the focusing point displayed in focus and continues to turn on the switch SW1 (# 1).
11), the CPU 100 transmits a signal to the photometric circuit 102 to perform photometry (# 112). At this time, a weighted exposure value is calculated for the photometric regions 210 to 213 including the focused distance measuring point.

In the case of the present embodiment, a known photometric operation weighted to the photometric area 210 including the distance measuring point 201 is performed.
Is used to display the aperture value (F5.6) (FIG. 16).
(A)).

Further, it is determined whether or not the release button 41 is depressed and the switch SW2 is turned on (# 113).
The state of the switch SW1 is confirmed again (# 11
1). If the switch SW2 is turned on, the CPU 1
00 is the shutter control circuit 108, the motor control circuit 1
09, a signal is transmitted to the aperture drive circuit 111, respectively.

First, power is supplied to the motor M2, the main mirror 2 is raised, and the aperture 31 is stopped down.
1 is turned on to open the front curtain of the shutter 4. The aperture value of the aperture 31 and the shutter speed of the shutter 4 are determined from the exposure value detected by the photometric circuit 102 and the sensitivity of the film 5. At a predetermined shutter time (1/250
After the elapse of (sec), the magnet MG2 is energized, and the rear curtain of the shutter 4 is closed. When the exposure of the film 5 is completed, the motor M2 is energized again to perform mirror down and shutter charging, and also energizes the motor M1 to perform frame advance of the film, thereby ending a series of shutter release sequence operations (#). 114). Thereafter, the camera waits until the switch SW1 is turned on again (# 10).
2).

During a series of operations other than the shutter release operation (# 114) of the camera shown in FIG. 8, the mode is changed by the mode dial 44, and the signal input circuit 104 indicates that the mode has been set to the line-of-sight calibration mode.
Is detected, the CPU 100 temporarily suspends the operation of the camera, transmits the result to the line-of-sight detection circuit 101, and sets a state in which the line-of-sight calibration (# 115) is possible. The gaze calibration method will be described later.

Here, the AF point automatic selection subroutine # 11
6 will be described with reference to FIG. This subroutine is executed when the line-of-sight detection inhibition mode, that is, the line-of-sight input mode is not set, as described above, and determines the distance measuring point from the information of the defocus amount and the absolute distance of each distance measuring point. Things.

First, it is determined whether or not there is a distance measuring point among the five distance measuring points (# 501). If none of the distance measuring points can be measured, the process returns to the main routine (# 511). .
If there is one ranging point that can be measured and there is only one (# 50)
2) One of the points is set as a distance measuring point (# 507). If there are two or more distance measuring points that can be measured, the process proceeds to the next. If there is a central distance measuring point (# 503), the central distance measuring point is a short distance (for example, 20 times or less the focal length). ) Is determined (# 504).

If the center ranging point can be measured and the distance is short, or if the center ranging point cannot be measured, # 5 is selected.
Go to 05. In # 505, if the number of short-distance ranging points is larger than the number of long-distance ranging points, it is determined that the main subject is considerably closer to the photographer, and the nearest ranging point is selected (# 506).
If the number of short distance measuring points is small, it is determined that the main subject is on the long distance side, and the nearest point among the long distance measuring points is selected in consideration of the depth of field (# 510). . If the center ranging point is far in # 504, the process proceeds to # 508.

If the number of long distance measurement points is larger than the number of short distance measurement points, it is determined that the main subject is on the long distance side including the center distance measurement point, and the center distance measurement point is selected ( # 509).
If the number of long distance measurement points is small, the closest distance measurement point is selected in the same manner as described above (# 506).

As described above, if there are distance measuring points that can be measured, one of the distance measuring points is automatically selected, the process returns to the main routine (# 511), and focus detection is again performed at this distance measuring point. The operation is performed (# 107). Note that the in-focus display when a focus detection point is selected using the above-mentioned line-of-sight information is the same as in FIG.
As shown in (B), the distance measuring point 201 and the focus mark 79 are lit, but the line-of-sight input mark 78 is naturally in a non-lighted state.

FIGS. 10 and 11 are flow charts of the line of sight detection. As described above, the line-of-sight detection circuit 101 is the CPU 1
When a signal is received from 00, eye gaze detection is executed (# 10
4). The gaze detection circuit 101 determines whether the gaze is detected in the photographing mode or the gaze is detected in the gaze calibration mode (# 201). At the same time, the line-of-sight detection circuit 101 recognizes which calibration data number the camera is set to be described later.

In the case of gaze detection in the photographing mode, the gaze detection circuit 101 first determines whether the camera is in a vertical position or a horizontal position, for example.
4 based on a signal from the posture detecting means 27 (# 202). That is, the signal input circuit 104 processes the output signal of the mercury switch 27 (SW-ANG) as the posture detecting means to determine whether the camera is in the horizontal position or the vertical position. It is determined whether 41 is in the top direction or in the ground (plane) direction. Subsequently, information on the brightness of the shooting area is obtained from the photometric circuit 102 via the CPU 100 (# 203).

Next, the infrared light emitting diode (hereinafter referred to as IRED 13) 1 is obtained from the previously detected posture information of the camera and the eyeglass information of the photographer included in the calibration data.
3a to 13f are selected (# 204). That is, if the camera is held in the horizontal position and the photographer does not wear glasses, the IREDs 13a and 13b from the finder optical axis are selected as shown in FIG. If the camera is in the horizontal position and the photographer wears glasses, the IREDs 13c and 13d that are away from the viewfinder optical axis are selected.

At this time, a part of the illumination light reflected by the photographer's eyeglasses reaches an area other than a predetermined area on the image sensor 14 on which the eyeball image is projected, so that the analysis of the eyeball image is not hindered. I have to. That is, the direction of illumination of the eyeball is changed according to the spectacle information to prevent reflected light (noise light) from the spectacles from entering the image sensor, thereby enabling highly accurate gaze detection.

Further, if the camera is held in a vertical position, an IRE that illuminates the photographer's eyeball from below.
Either the combination of D13a, 13e or the combination of IREDs 13b, 13f is selected.

Next, the image sensor 14 (hereinafter referred to as CCD-
Called EYE. ) And the illumination power of the IRED 13 are set based on the photometric information, the eyeglass information of the photographer, and the like (# 205). The accumulation time of the CCD-EYE 14 and the illumination power of the IRED 13 may be set based on values determined from the contrast of the eyeball image obtained at the time of the previous gaze detection.

Storage time and IRE of CCD-EYE 14
When the illumination power of D13 is set, the CPU 100
The IRED 13 is turned on with a predetermined power via the RED drive circuit 107, and the line-of-sight detection circuit 101
The storage of the D-EYE 14 is started (# 206). Also, the CCD-EYE 14 terminates the accumulation according to the previously set accumulation time of the CCD-EYE 14, and at the same time, the CCD-EYE 14
The RED 13 is also turned off. If it is not the line-of-sight calibration mode (# 207), a predetermined read area of the CCD-EYE 14 is set (# 208).

The read area of the CCD-EYE 14 is set on the basis of the read area of the CCD-EYE 14 at the time of the previous line-of-sight detection except for the first line-of-sight detection after the power of the camera body is turned on. When the posture of the camera changes, or when the presence or absence of glasses changes, the CCD-
The read area of the EYE 14 is set to the entire area. CC
When the read area of the D-EYE 14 is set, the CCD
-Reading of the EYE 14 is executed (# 209). At this time, the area other than the read area is blank-read, and is actually skipped.

The image output read from the CCD-EYE 14 is subjected to A / D conversion by the visual axis
100, and a calculation for extracting each feature point of the eyeball image is performed in the CPU 100 (# 21).
0). That is, the CPU 100 detects the positions (xd ', yd') and (xe ', ye') of the Purkinje image, which are virtual images of the set of IREDs 13 used for illuminating the eyeball. Since the Purkinje image appears as a bright spot having a high light intensity, a predetermined threshold value for the light intensity is provided, and a light intensity exceeding the threshold value can be detected as a Purkinje image.

The pupil center position (xc ', yc') is calculated by detecting a plurality of boundary points between the pupil 19 and the iris 17 and performing a least square approximation of a circle based on each boundary point. At this time, the pupil diameter rp is also calculated. The interval between the two Purkinje images is calculated.

The CPU 100 analyzes the eyeball image, detects the contrast of the eyeball image, and resets the accumulation time of the CCD-EYE 14 from the degree of the contrast. Further, the position of the Purkinje image and the position of the pupil (x
d ', yd'), (xe ', ye') from CCD-EYE1
4 is set.

At this time, the read-out area of the CCD-EYE 14 is set to a range including the detected pupil and capable of detecting the entire pupil even if the position of the pupil changes by a predetermined amount. And, needless to say, its size is smaller than the size of the iris.

The read-out area of the CCD-EYE 14 is set to a rectangle, and the coordinates of two diagonal points of the rectangle are set to the CCD-EYE 14.
The eye-gaze detection circuit 101 stores the read area of the EYE 14. Further, the reliability of the calculated Purkinje image and the position of the center of the pupil is determined from the contrast of the eyeball image or the size of the pupil. The reliability information at this time is
This is one of the eye-gaze correction data (calibration data).

When the analysis of the eyeball image is completed, the line-of-sight detection circuit 101 also serving as a means for confirming the calibration data is used to determine the distance between the calculated Purkinje image and the illuminated IRED1.
It is determined whether or not the eyeglasses information, which is one of the calibration data, is correct based on the combination of # 3 (# 21).
1). This is to cope with a photographer who uses or does not use glasses at each time.

That is, the spectacle information of the photographer in the calibration data is set to use, for example, spectacles, and the IRE in the IRED 13 shown in FIG.
When D13c and 13d are turned on, if the interval between the Purkinje images is larger than a predetermined size, the photographer is recognized as a spectacle wearer, and it is determined that the spectacle information is correct. On the other hand, if the interval between the Purkinje images is smaller than the predetermined size, the photographer is recognized as the naked eye or the contact lens wearer, and it is determined that the eyeglasses information is incorrect.

If it is determined that the glasses information is incorrect (#
211), the line-of-sight detection circuit 101 changes the eyeglass information as the line-of-sight correction data (# 217), and again the IRED.
No. 13 is selected (# 204), and gaze detection is executed.
However, when changing the eyeglass information, the CPU 100
The glasses information stored in the ROM is not changed.

When it is determined that the eyeglass information is correct (#
212), the distance between the eyepiece 11 of the camera and the eyeball 15 of the photographer is calculated from the distance between the Purkinje images, and the CC is calculated from the distance between the eyepiece 11 and the eyeball 15 of the photographer.
The imaging magnification β of the eyeball image projected on the D-EYE 14 is calculated (# 212). From the above calculated values, the rotation angle θ of the optical axis 15a of the eyeball 15 is obtained by modifying the equation (3), and θx {ARCSIN} (xc ′ − (xp ′ + δx) / β / OC} (6) θy {ARCSIN} (yc ′ − (yp ′ + δy) / β / OC} (7) (# 213).

Here, xp'ｘ (xd '+ xe') / 2 yp '≒ (yd' + ye ') / 2 δx, δy are correction terms for correcting the center positions of the two Purkinje images.

When the rotation angles θx and θy of the eyeball of the photographer are obtained, the position (x, y) of the line of sight on the focus plate 7 is (5)
The expression is modified to obtain x ≒ m * ax * (θx + bx) ‥‥‥ (8) y ≒ m * ax * (θy + by) ‥‥‥ (9) (# 214). Here, ax, bx, and by are parameters for correcting individual differences in the line of sight, and ax is calibration data.

Further, bx corresponding to the correction amount between the optical axis and the visual axis of the eyeball in the horizontal direction (x direction) is expressed as bx = kx * (rp−rx) + bOx ‥‥‥ (10) It is a function of the pupil diameter rp. Here, rx is a constant and bOx is calibration data.

In equation (10), the value of the proportional coefficient kx relating to the pupil diameter rp differs depending on the size of the pupil diameter. When rp ≧ rx, kx = 0, when rp <rx, kx = ｛1−k0 * k1 * (θx + bx ′) / | k0 |｝ * k0 ‥‥‥ (11)

That is, the proportionality coefficient kx takes a value of 0 if the pupil diameter rp is equal to or larger than the predetermined pupil size rx, and conversely, if the pupil diameter rp is smaller than the predetermined pupil size rx, k
x is a function of the rotation angle θx of the optical axis of the eyeball.

Bx 'is equivalent to the amount of correction of the visual axis when the photographer is looking substantially at the center of the viewfinder, and is expressed as bx' = k0 * (rp-rx) + b0x.

K0 is the calibration data and represents the ratio of the change in the visual axis correction amount bx to the change in the pupil diameter rp when the photographer looks at the approximate center of the finder. K1 is a predetermined constant.

The by value corresponding to the correction amount in the vertical direction (y direction) is represented as by = ky * rp + bOy ＋ (12), and is a function of the pupil diameter rp. Where ky, b
0y is calibration data. A method for obtaining the above-mentioned line-of-sight calibration data will be described later.

Further, the reliability of the coordinates of the line of sight calculated using the equations (8) to (12) is changed according to the reliability of the line of sight calibration data. When the coordinates of the line of sight on the focus plate 7 are obtained, a flag indicating that the line of sight has been detected is set (# 215), and the process returns to the main routine (# 218).

The flow chart of the visual axis detection shown in FIGS. 10 and 11 is also effective in the visual axis calibration mode. If it is determined in (# 201) that the gaze detection is performed in the calibration mode, then it is determined whether or not the current gaze detection is the first gaze detection in the calibration mode (# 201). 21
6). If it is determined that this gaze detection is the first gaze detection in the calibration mode, the CCD-E
Ambient brightness is measured to set the accumulation time of YE14 and the illumination power of IRED13 (# 2)
03). The subsequent operation is as described above.

If it is determined that the current line-of-sight detection is the second or more line-of-sight detection in the calibration mode (# 216), the accumulation time of the CCD-EYE 14 and the illumination power of the IRED 13 are set to the previous values. Immediately after adoption, lighting of the IRED 13 and accumulation of the CCD-EYE 14 are started (# 206). If the gaze detection mode is the second or more gaze detection mode in the gaze calibration mode (# 20)
7) Since the same area as the previous area is used as the read area of the CCD-EYE 14, the CCD-EYE 14 is immediately read out upon completion of the accumulation of the CCD-EYE 14 (# 209). The subsequent operation is as described above.

In the flow charts of gaze detection shown in FIGS. 10 and 11, the return number when returning to the main routine is the coordinates (x, y) of the gaze on the focus plate in the case of normal gaze detection. However, in the case of gaze detection in the gaze calibration mode, the rotation angle (θx, θy) of the optical axis of the eyeball of the photographer. Further, the reliability of the detection result, which is another return number, the accumulation time of the CCD-EYE 14, the CCD-EY
The readout area of E14 is common.

In this embodiment, the CCD-EYE
The photometric information detected by the photometric sensor 10 of the camera is used to set the accumulation time 14 and the illumination power of the IRED 13, but the brightness of the anterior segment of the photographer is detected near the eyepiece 11. It is also effective to newly provide a means and use the value.

FIGS. 12, 13 and 14 are flowcharts of the line-of-sight calibration, and FIGS. 17 to 22 show the display states of the LCD 24 in the finder and the monitor LCD 42 during the line-of-sight calibration.

Conventionally, gaze calibration has been performed by detecting the gaze when the photographer gazes at two or more targets, but in the present embodiment, the two targets are compared with the brightness of the viewfinder. Are gazed twice in different states, and the gaze at that time is detected to execute the gaze calibration. Thus, the calibration data of the line of sight corresponding to the pupil diameter is calculated. Hereinafter, description will be made with reference to FIG.

The photographer turns the mode dial 44 to
When the index is set to the AL position 44d, the line-of-sight calibration mode is set, and the signal input circuit 104 is set.
Transmits a signal to the LCD drive circuit 105 via the CPU 100, and the monitor LCD 42 displays a message indicating that one of the eye-gaze calibration modes described below has been entered. The CPU 100 resets variables other than the calibration data stored in the EEPROM (# 301).

FIG. 23 shows the types of calibration data stored in the EEPROM of the CPU 100 and their initial values. Actually the EEPROM of CPU100
What is stored in M is the data surrounded by the thick line in FIG.
It is a currently set calibration data number and a plurality of calibration data managed by the calibration data number. Here, the calibration data number 0 is a mode for prohibiting gaze detection. In addition, the above-mentioned line-of-sight calibration data is stored in each address on the EEPROM corresponding to the calibration data numbers 1 to 5 (in the present embodiment, five data are used for explanation). I try to remember it,
Of course, it can be set in any way depending on the capacity of the EEPROM.)

The initial value of the calibration data is set to a value such that the line of sight is calculated using standard eyeball parameters. Further, it has a flag indicating whether or not the photographer uses glasses and the degree of reliability of the calibration data. The initial value of the flag indicating the presence or absence of spectacles is set to “1” as if the spectacles are used, and the initial value of the reliability flag of the calibration data is set to “0” so as not to be reliable. ing.

Further, the monitor LCD 42 is provided as shown in FIG.
The currently set calibration mode is displayed as shown in FIG. The calibration mode includes an "ON" mode in which the calibration operation is performed and an "OFF" mode in which the calibration operation is not performed.

First, in the "ON" mode, calibration numbers CAL1 to CAL5 are prepared so as to correspond to the calibration data numbers 1 to 5, a 7-segment 62 for displaying the shutter time, and a 7-portion for displaying the aperture value. It is displayed using the segment 63, and all other fixed segment display sections 42a are turned off (this embodiment shows the state of data number 1 and only the 7-segment display section is shown in an enlarged manner).

At this time, if the calibration data of the set calibration number is the initial value, the calibration number displayed on the monitor LCD 42 flashes (FIG. 17B), while the set calibration number is If the calibration data (eye-gaze correction data) different from the initial value is included in the address of the EEPROM 100a as the storage means corresponding to the calibration number, the calibration displayed on the monitor LCD 42 is performed. The application number is fully lit (FIG. 17A).

As a result, the photographer can recognize whether or not each of the currently set calibration numbers already contains the calibration data. The initial value of the calibration data number is set to 0, so that information is not input by the line of sight unless the line of sight is calibrated.

Next, in the "OFF" mode, the seven segments 62 are displayed as "OFF" (FIG. 17 (C)), so that the calibration data number 0 is always selected and the gaze prohibition mode is set. ing. This is effective for prohibiting information input by the line of sight and taking a picture in order to prevent erroneous gaze detection position and malfunction, for example, when suddenly taking a picture by another person for commemorative photography etc. is there.

Subsequently, the timer set in the CPU 100 is started to start the line-of-sight calibration (#
302). If no operation is performed on the camera within a predetermined time after the timer starts, the gaze detection circuit 101 resets the calibration data number set at that time to 0 and changes to the gaze prohibition (OFF) mode. I do. In addition, if a visual target or the like for visual axis calibration is lit in the viewfinder, the target is turned off.

As described above, the EEPROM as the storage means
If the line-of-sight correction data (calibration data) is not newly stored in 100a, the operation using the line of sight is stopped.

When the photographer rotates the electronic dial 45, the signal input circuit 104 which has detected the rotation by the pulse signal as described above transmits a signal to the LCD drive circuit 105 via the CPU 100. As a result, the calibration number displayed on the monitor LCD 42 changes in synchronization with the rotation of the electronic dial 45. This is shown in FIG.

First, when the electronic dial 45 is rotated clockwise, “CAL-1” → “CAL-2” → “CAL-
3 "→" CAL-4 "→" CAL-5 ", and the photographer can store the calibration data in any of the five desired calibration numbers by the calibration operation described later. In the state shown in FIG. 18, “CAL-1, 2, 3” already contains calibration data, and “CAL-4, 5”
Indicates that it is not entered and remains at the initial value.

Next, when the camera is further rotated one click in the clockwise direction, "OFF" is displayed, the calibration operation is not performed, and the visual axis detection inhibition mode is set. Further rotation by one click returns to “CAL-1”, and the calibration number is cyclically displayed as described above.
When rotated in the counterclockwise direction, the image is displayed in the exact opposite direction of FIG.

When the photographer selects a desired calibration number while looking at the calibration number displayed on the monitor LCD 42 in this manner, the visual line detection circuit 101 confirms the corresponding calibration data number by inputting a signal. This is performed via the circuit 104 (# 303). The confirmed calibration data number is stored at a predetermined address in the EEPROM of the CPU 100.

However, if the confirmed calibration data number is not changed, the calibration data number is not stored in the EEPROM.

Subsequently, the line-of-sight detection circuit 101 checks the photographing mode via the signal input circuit 104 (# 30).
4). If it is confirmed that the photographer has switched to the shooting mode other than the eye gaze calibration mode by rotating the mode dial 44 (# 304), if the eye gaze calibration target is blinking in the viewfinder. Then, it is turned off (# 305), and the process returns to the main routine, which is a photographing operation of the camera (# 338).

Then, the calibration number "CA
Mode dial 44 with "L1-5" displayed.
Is switched to another photographing mode (shutter priority AE), the line of sight is detected using the data of the calibration number, and the photographing operation using the above-described line of sight information can be performed. LCD for monitor at this time
The state of 42 is shown in FIG. 19. In addition to the normal shooting mode display, the gaze input mode display 61 is turned on to inform the photographer that the gaze input mode is controlling the shooting operation based on the gaze information. Let me know.

Here, the mode dial is again rotated to
When the optotype is aligned with the AL position 44d, the calibration number used for the above-mentioned line-of-sight detection is displayed, and the calibration operation is started. However, if the photographer does not operate the camera within a predetermined time, or EEPR when calibration data of
No change is made to the OM calibration data.

When it is confirmed that the line-of-sight calibration mode is still set (# 304), the calibration number set by the electronic dial 45 is confirmed again (# 306). At this time, if the calibration data number is set to 0 and the gaze prohibition mode is set, the calibration data number is stored again in the EEPROM of the CPU 100 (# 3).
03). If gaze prohibition is selected in the calibration mode, the camera waits with the mode dial 44 until the mode is changed to a shooting mode other than the gaze calibration mode.

That is, if the mode dial 44 is switched while “OFF” is displayed, the photographing operation is performed without performing the line-of-sight detection. It is lit.

As described above, the EEPROM 1 of the CPU 100
The camera (optical device) controls various driving operations for photographing according to the properties of the calibration data (eye-gaze correction data) stored in 00a.

If the calibration data number is set to a value other than 0 (# 306), the CPU
100 detects the posture of the camera by the posture detecting means via the signal input circuit 104 (# 307). The signal input circuit 104 processes the output signal of the mercury switch 27 to determine whether the camera is in the horizontal position or the vertical position. If the camera is in the vertical position, for example, the release button 41 is in the top direction or the ground (surface) direction. Judge whether it is.

Since the camera is generally used in a horizontal position, a hardware configuration for calibrating the line of sight is set so that the camera can be calibrated when the camera is held in the horizontal position. Therefore, the gaze detection circuit 1
01 indicates that the camera is not in the horizontal position.
When the communication is performed, the gaze calibration is not executed (# 308). That is, detection of the eye-gaze correction data is prohibited.

The eye-gaze detecting circuit 101 is provided in the camera viewfinder as shown in FIG. 21A to warn the photographer that the eye-gaze cannot be calibrated because the camera is in the horizontal position. "CAL" is displayed on the LCD 24 in the finder which is one element of the provided warning means.
Flashes the display. At this time, a warning sound may be emitted by a sounding body (not shown) as warning means.

On the other hand, when it is detected that the posture of the camera is in the horizontal position (# 308), the gaze detection circuit 101 sets the number n of gaze detection to 0 (# 309). However, when the number of gaze detection times n is 20, the number is kept. At this time, if the "CAL" display is blinking on the LCD 24 in the viewfinder, the blinking is stopped. The line-of-sight calibration is set to be started by turning on the switch SW1. In order to prevent the photographer from starting calibration on the camera side before the photographer is ready to calibrate the gaze, the gaze detection circuit 101
Checks the state of the switch SW1, and checks the state of the switch SW1.
If the button is pressed by the release button 41 and is in the ON state, it waits until the switch SW1 is turned off (#
310).

The line of sight detection circuit 101 is a signal input circuit 104
When the switch SW1 is confirmed to be in the OFF state via # (# 310), the number of gaze detection times n is confirmed again (# 311). If the gaze detection frequency n is not 20 (# 311), the gaze detection circuit 101
A signal is transmitted to 06 to blink the visual axis calibration target (# 313). The target for eye gaze calibration is guided to the superimposed display of the calibration operation described below, and also partly uses the ranging point mark so that the photographer can perform smoothly. Ranging point mark 204 and dot mark 206
Blinks (FIG. 20A).

If the ON signal of the switch SW1, which is a trigger signal for starting the line-of-sight calibration, is not input, the camera stands by (# 314). When the photographer gazes at the target whose blinking has been started and presses the release button 41 to turn on the switch SW1 (# 314), the line-of-sight detection is executed (# 315). The operation of gaze detection is as described in the flowchart of FIG.

Dot marks 206 and 205 are engraved on the distance measuring point mark 204 on the right end and the distance measuring point mark 200 on the left end, indicating that calibration is performed at these two positions. Lighting, blinking, and non-lighting can be displayed by being illuminated by the LED 21 for superimposition. Further, since the ranging point marks 200 to 204 indicate a focus detection area, it is necessary to display an area corresponding to the area.

However, in order to perform the calibration with high accuracy, it is necessary for the photographer to pay attention to one point as much as possible. The dot marks 205 and 206 are distance measuring point marks so that one point can be easily observed. 200-20
It is provided smaller than 4. Eye-gaze detection circuit 101
Stores the eyeball rotation angles θx and θy, the pupil diameter rp, and the reliability of each data, which are the returns from the line-of-sight detection subroutine (# 316). Further, the number of gaze detection times n is counted up (# 317).

Since the line of sight of the photographer has some variation, it is necessary to use one line to obtain accurate line-of-sight calibration data.
It is effective to execute a plurality of gaze detections on a point target and use the average value. In the present embodiment, the number of gaze detection times for one target is set to 10 times. If the number n of gaze detection times is not 10 or 30 (# 318), gaze detection is continued (# 315).

By the way, in this embodiment, the line of sight is calibrated when the brightness of the viewfinder is different.
That is, the measurement is performed twice with different pupil diameters. Therefore, the number of gaze detections n when starting the second gaze calibration is 20 times. If the number of sight line detections n is 10 or 30, the sight line detection for the target 1 (the ranging point mark 204 and the dot mark 206) is terminated (# 318).

In order for the photographer to recognize that the visual line detection for the visual target 1 has been completed, the visual line detection circuit 101
The electronic sound is made to sound several times by using a sounding body (not shown) via 100. At the same time, the line-of-sight detection circuit 101
The target 1 is fully lit for a predetermined time via the drive circuit 106 (# 319) (FIG. 20B).

Subsequently, the visual line detection circuit 101 checks via the signal input circuit 104 whether the switch SW1 is in the OFF state (# 320). Switch SW
If the switch 1 is on, the control waits until the switch 1 is turned off. If the switch SW1 is off, the optotype 1 is turned off, and at the same time, the optotype 2 (the distance measuring point mark 200 and the dot mark 205) blinks. Start (# 321) (FIG. 20)
(C)).

The line-of-sight detection circuit 101 is again in the signal input circuit 1
It is confirmed via the switch 04 whether the switch SW1 is in the ON state (# 322). If the switch SW1 is in the OFF state, the operation waits until the switch SW1 is turned on.
When W1 is turned on, visual line detection is executed (# 323).
The gaze detection circuit 101 stores the eyeball rotation angles θx and θy, the pupil diameter rp, and the reliability of each data, which are the returns from the gaze detection subroutine (# 324). Further, the number of gaze detection times n is counted up (# 325). Furthermore, if the number of gaze detection times n is not 20 or 40 (# 32
6) Gaze detection is continued (# 323). If the number of gaze detection times n is 20 or 40, the gaze detection for the target 2 is terminated (# 326).

In order for the photographer to recognize that the visual line detection for the visual target 2 has been completed, the visual line detection circuit 101
The electronic sound is made to sound several times by using a sounding body (not shown) as a warning means via 100. At the same time, the visual line detection circuit 101 fully lights the visual target 2 via the LED drive circuit 106 (# 327) (FIG. 20 (D)).

If the visual axis detection for the visual targets 1 and 2 is performed once and the number of visual line detections n is 20 (# 32)
8) In the state where the brightness of the viewfinder is different, the second gaze detection for each target is performed. Eye-gaze detection circuit 10
1 confirms the state of the switch SW1 via the signal input circuit 104 (# 310). If the switch SW1 is in the ON state, the process waits until the switch SW1 is turned off. If the switch SW1 is in the OFF state, the number of sight line detections n is confirmed again (# 311).

If the number of sight line detections n is 20 (# 31)
1), the line-of-sight detection circuit 101 transmits a signal to the aperture driving circuit 111 via the CPU 100 to
Set to minimum aperture. At this time, the photographer feels that the inside of the viewfinder has become dark, and greatly widens the pupil. At the same time, the visual line detection circuit 101 turns off the visual target 2 (# 31
2). Then, the optotype 1 at the right end starts blinking in order to perform the second gaze detection (# 313). Operation # 31 below
4 to # 327 are as described above.

If the visual axis detection is performed on the visual targets 1 and 2 in the state where the brightness of the viewfinder is different (the state where the pupil diameter is different), the number of visual line detections n is 40 (# 3).
28), gaze detection for obtaining gaze calibration data ends. The line-of-sight detection circuit 101 sends a signal to the aperture driving circuit 111 to set the aperture 31 of the photographic lens 1 to the open state (# 329). Furthermore, the rotation angles θx and θy of the eyeball stored in the line-of-sight detection circuit 101 and the pupil diameter r
The line-of-sight calibration data is calculated from p (# 330). The method of calculating the line-of-sight calibration data is as follows.

The coordinates of the visual target 1 and the visual target 2 on the focus plate 7 are (x1, 0) and (x2, 0), respectively.
The average values of the rotation angles (θx, θy) of the eyeballs when gazing at each of the targets stored in No. 1 are (θx1, θy1), (θx
2, θy2), (θx3, θy3), (θx4, θy
4), let the average value of the pupil diameter be r1, r2, r3, r4.

However, (θx1, θy1), (θx3, θy
3) is the average value of the rotation angles of the eyeballs detected when the photographer gazes at the target 1, (θx2, θy2), (θx4, θ
y4) represents the average value of the rotation angle of the eyeball detected when the photographer gazes at the target 2.

Similarly, r1 and r3 are the average values of the pupil diameters detected when the photographer gazes at the target 1, and r2 and r4 are the pupil diameters detected when the photographer gazes at the target 2. It is an average value. The suffixes 1 and 2 attached to the average value of each data indicate the data when the gaze is detected when the viewfinder of the camera is bright, and the suffixes 3 and 4 indicate when the gaze is detected when the viewfinder of the camera is dark. Data.

The formula for calculating the calibration data of the horizontal line of sight (x direction) differs depending on the pupil diameter at the time of data acquisition. (1-1)
2) / 2, k0 = − [(θx3 + θx4) − (θx1 + θx2)] /
{2 * rx- (r1 + r2)} ax = (x3-x4) / m / (θx3-θx4) b0x = − (θx3 + θx4) / 2 (1-2) rx ≧ (r3 + r4) / 2> (r1 + r
2) / 2, k0 = − [(θx3 + θx4) − (θx1 + θx2)] /
[(r3 + r4)-(r1 + r2)] ax = (x3-x4) / m / ｛θx3-θx4 + k0
* (R3-r4)｝ b0x = −k0 * {(r3 + r4) / 2−rx} −
(Θ3 + θ4) / 2 is calculated.

The calibration data of the line of sight in the vertical direction (y direction) is as follows: ky = − [(θy3 + θy4) − (θy1 + θy2)] /
[(r3 + r4)-(r1 + r2)] b0y = [(θy1 + θy2) * (r3 + r4)-(θy3
+ Θy4) * (r1 + r2)] / 2 / ｛(r1 + r2)-
(R3 + r4)｝ is calculated.

After calculating the line-of-sight calibration data,
Alternatively, the timer is reset after the end of gaze detection (# 331).

The gaze detection circuit 101, which also serves as a means for determining the reliability of the calibration data, determines whether the calculated calibration data of the visual axis is appropriate (# 332). The determination is made using the reliability of the eyeball rotation angle and pupil diameter, which are the returns from the visual axis detection subroutine, and the calculated visual axis calibration data itself. That is, when the rotation angle of the eyeball and the pupil diameter detected in the gaze detection subroutine are not reliable, it is determined that the calculated calibration data of the gaze is also unreliable.

When the rotation angle of the eyeball and the pupil diameter detected in the gaze detection subroutine are reliable, it is appropriate if the calculated gaze calibration data falls within the range of general individual differences. On the other hand, if the calculated line-of-sight calibration data greatly deviates from the range of general individual differences, it is determined that the calculated line-of-sight calibration data is inappropriate. Eye-gaze detection circuit 1
01 not only determines whether or not the calculated line-of-sight calibration data is appropriate, but also determines how reliable the calculated line-of-sight calibration data is.

The degree of reliability depends on the reliability of the eyeball rotation angle and the pupil diameter detected in the line-of-sight detection subroutine. The reliability of the line-of-sight calibration data is digitized into two bits according to the degree and stored in the EEPROM of the CPU 100 as described later.

If the calculated line-of-sight calibration data is determined to be inappropriate (# 332), the LED drive circuit 106 stops energizing the superimposing LED 21 and turns off the targets 1 and 2 (#). 339). Further, the line-of-sight detection circuit 101 emits an electronic sound for a predetermined period of time using a sounding body (not shown) via the CPU 100 to warn that the line-of-sight calibration has failed. LCD at the same time
A signal is sent to the drive circuit 105 and the LCD 2 in the finder is
4 and the LCD 42 for the monitor blink the "CAL" display to warn (# 340) (FIG. 21 (A), FIG.
(A)).

After a warning sound by the sounding body and a warning display by the rCDs 24 and 42 have been performed for a predetermined time, the flow shifts to the initial step (# 301) of the calibration routine, and the state is set such that the visual line calibration can be executed again.

If the calculated line-of-sight calibration data is appropriate (# 332), the line-of-sight detection circuit 10
1 displays the end of the line-of-sight calibration via the LCD drive circuit 105 and the LED drive circuit 106 (#
333). The LED driving circuit 106 energizes the superimposing LED 21 to blink the optotypes 1 and 2 several times, and the rCD driving circuit 105 controls the LCD 24 and the LCD.
42 to send an “End-calibration N
"o" is displayed for a predetermined time (FIG. 2).
1 (B), FIG. 22 (B)).

The gaze detection circuit 101 sets the number of gaze detection times n to 1
(# 334), and the reliability of the calculated line-of-sight calibration data, the photographer's eyeglass information, and the calculated line-of-sight calibration data is stored in the EEPROM 100a corresponding to the currently set calibration data number. (# 33)
5). At this time, if the line-of-sight calibration data has already been stored at the address of the EEPROM to be stored, the calibration data is updated.

As described above, the reliability of the line-of-sight correction data is determined, and the line of sight is detected based on the result, whereby the optical device can be controlled with high accuracy.

After a series of line-of-sight calibration is completed,
The camera waits until the electronic dial 45 or the mode dial 44 is operated by the photographer. If the photographer rotates the electronic dial 45 to select another calibration number, the line-of-sight detection circuit 101 detects a change in the calibration number via the signal input circuit 104 (# 336), and performs line-of-sight calibration. The routine proceeds to the initial step (# 301) of the routine. If the photographer rotates the mode dial 44 to select another photographing mode, the visual line detection circuit 101 detects a change in the photographing mode via the signal input circuit 104 (# 33).
7) Return to the main routine (# 338).

At the time of returning to the main routine, if the calibration data set by the electronic dial 45 has not been input and the initial value has not been entered, the line-of-sight detection circuit 101 executes the corresponding calibration. The data number is reset to 0 and the gaze prohibition mode is forcibly set. Actually CPU10
The currently set calibration data number stored in the EEPROM of 0 is set to 0 (gaze prohibited mode).
Reset to.

In the present embodiment, an example has been shown in which the number of gaze detections is made 10 times while gazing at a single target, and the gaze is calibrated. It does not matter.

In the present embodiment, the taking lens 1
When the aperture 31 is stopped down, calibration is performed by setting a state where the brightness of the finder is different, that is, a state where the pupil diameter of the photographer is different. It is also possible to change the light emission luminance of the application LED 21.

[0164]

As described above, according to the present invention, the eyeball can be illuminated from a position suitable for each case whether the observer wears spectacles or the naked eye. Thus, accurate gaze detection can be performed without being disturbed by noise such as ghosts.

[Brief description of the drawings]

FIG. 1 is a schematic view of a main part of an embodiment when the present invention is applied to a single-lens reflex camera.

FIG. 2 is an external view of a main part of the single-lens reflex camera of FIG. 1;

FIG. 3 is a viewfinder view of FIG. 1;

FIG. 4 is an explanatory diagram of the mode dial of FIG. 2;

FIG. 5 is an explanatory diagram of the electronic dial of FIG. 2;

FIG. 6 is a block diagram of a main part of an electric circuit according to the embodiment of the present invention.

FIG. 7 is an explanatory view of a part of FIG. 2;

8 is a flowchart of the operation of the single-lens reflex camera of FIG. 6;

FIG. 9 is a flowchart of a ranging point automatic selection algorithm.

FIG. 10 is a flow chart of gaze detection.

FIG. 11 is a flowchart of gaze detection.

FIG. 12 is a flowchart of a calibration according to the present invention.

FIG. 13 is a flowchart of a calibration according to the present invention.

FIG. 14 is a flowchart of a calibration according to the present invention.

FIG. 15 is an explanatory diagram of a display state in the viewfinder field of FIG. 1;

FIG. 16 is an explanatory diagram of a display state in the viewfinder field of FIG. 1;

FIG. 17 is an explanatory diagram of a display state of the monitor LCD of FIG. 2;

FIG. 18 is an explanatory diagram of a display state of the monitor LCD of FIG. 2;

19 is an explanatory diagram of a display state of the monitor LCD of FIG. 2;

FIG. 20 is an explanatory diagram of a display state in the finder visual field of FIG. 1;

FIG. 21 is an explanatory diagram of a display state in the finder visual field of FIG. 1;

FIG. 22 is an explanatory diagram of a display state of the monitor LCD of FIG. 2;

FIG. 23 is an explanatory diagram of calibration data according to the embodiment of the present invention.

FIG. 24 is a schematic diagram of a main part of an eyeball image.

FIG. 25 is a schematic view of a main part of a conventional gaze detection device.

[Explanation of symbols]

Reference Signs List 1 shooting lens 2 main mirror 6 focus detection device 6f image sensor 7 focus plate 10 photometric sensor 11 eyepiece 13 infrared light emitting diode (IRED) 14 image sensor (CCD-EYE) 15 eyeball 16 cornea 17 iris 21 LED for superimpose 23 Field Mask 24 LCD in Viewfinder 25 LED for Illumination 27 Mercury Switch 31 Aperture 41 Release Button 42 LCD for Monitor 42a Fixed Display Segment 42b 7 Segment Display 43 AE Lock Button 44 Mode Dial 45 Electronic Dial 61 Eye-Gaze Input Mode Display 78 Eye-Gaze Input mark 100 CPU 101 Eye-gaze detection circuit 103 Focus detection circuit 104 Signal input circuit 105 LCD drive circuit 106 LED drive circuit 107 IRED drive circuit 110 Focus adjustment circuit 200 to 204 Distance measuring point mark (calibration target) 205 to 206 Dot mark 207 Outside viewfinder 213 Observation screen

Continuation of the front page (72) Inventor Yoshiaki Irie 3-30-2 Shimomaruko, Ota-ku, Tokyo F-term in Canon Inc. (reference) 2H002 GA63 GA64 2H011 BA23 BB04 DA01 2H051 BA04 DA08 DA24 EB01

Claims (4)

[Claims] A determining means for determining whether or not the eyeglasses are worn; illuminating the eyeball from a first position when the determining means determines that the eyeglasses are worn; Illuminating means for illuminating the eyeball from a second position different from the first position when it is determined that the eyeball is not worn, and utilizing a corneal reflection image generated on the eyeball by being illuminated by the illuminating means A gaze detection device for detecting a gaze by performing a detection. 2. The apparatus according to claim 1, wherein the illuminating means simultaneously illuminates with a plurality of light-emitting elements, and the second position is farther from the finder optical axis than the first position. The line-of-sight detection device according to the above. 3. The lighting device according to claim 1, wherein the lighting unit has a plurality of light emitting elements, and selectively illuminates a light emitting element located below an eyeball according to a direction of the device. The line-of-sight detection device according to the above. 4. The eye-gaze detecting device according to claim 3, wherein the plurality of light-emitting elements illuminate an eyeball obliquely.