JPH06125874A - Sight line detecting device - Google Patents

Abstract

PURPOSE:To satisfactorily detect a line of sight even in the case of a photogra pher putting spectacles by providing a means for discriminating whether a result of detected line of sight is normal or not, and a means for reselecting a cornea reflected image in accordance with result of the discriminating means. CONSTITUTION:A sight line detecting circuit 101 receives a signal from a CPU 100, and the CPU 100 decides whether a result of detected line of sight is useable or not. When it is not normal, in the case there are three pairs of (52a, 52b), (53a, 53b), and (54a, 54b) as pairs of a P image, the pairs of (53a, 53b), and (54a, 54b) are ghosts caused by spectacles. When the pair of (53a, 53b) is selected in the beginning and a line of sight is detected, a result of detection is not normal and when there are plural candidates of the P image pair, other P image pair is selected and it is tried again to detect the line of sight. Also, in the case there is nothing but one P image pair, even if it is incorrect P image pair, it is impossible to select again another pair, therefore, it is regarded as a failure of detection of a line of sight, and a routine is returned.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an optical device having line-of-sight detecting means, and more particularly to an observer (photographer) through a viewfinder system on an observation plane (focus plane) on which an object image is formed by the photographing system. The so-called line of sight (visual axis) in the direction of the gazing point that is being observed by the observer is detected using the reflected image of the eyeball obtained when the observer's eyeball is illuminated, and various shooting operations are performed. The present invention relates to an optical device having the line-of-sight detection means.

[0002]

2. Description of the Related Art Conventionally, various devices (for example, eye cameras) for detecting what position on an observation surface an observer observes, that is, for detecting a so-called line of sight (visual axis) have been proposed.

For example, in Japanese Unexamined Patent Publication No. 1-274736, a parallel light flux from a light source is projected onto the anterior segment of the eyeball of an observer, and a corneal reflection image by the reflected light from the cornea and an image forming position of the pupil are used. Seeking the visual axis.

FIG. 10 is a diagram for explaining the principle of the visual axis detection method.

FIG. 1A shows the image sensor 14 of FIG.
It is an eyeball image in a very normal case projected on the surface,
Reference numeral 60 in (B) indicates the image signal output on the lines (I)-(I) '.

In the figure, 50 is the so-called white part of the eyeball, 5
Reference numeral 1 represents a pupil, and 52a and 52b represent corneal reflection images of an eyeball illumination light source.

FIG. 2 is an example of an eyeball image of a photographer wearing glasses. (52a, 52b) is a cornea reflection image, (53a,
53b) and (54a, 54b) represent ghost images of the light source by the glasses.

Next, the visual axis detection method will be described with reference to FIGS. 10, 1A and 1B. The infrared light emitting diodes 13a and 13b are arranged substantially symmetrically in the Z direction with respect to the optical axis A of the light receiving lens 12, and divergently illuminate the eyes of the photographer.

The infrared light emitted from the infrared light emitting diode 13b illuminates the cornea 16 of the eyeball 15. At this time, the corneal reflection image d by a part of the infrared light reflected on the surface of the cornea 16
Is collected by the light receiving lens 12 and the image sensor 14
The image is re-imaged at the upper position d '.

Similarly, the infrared light emitted from the infrared light emitting diode 13a illuminates the cornea 16 of the eyeball. At this time, the cornea reflection image e by a part of the infrared light reflected on the surface of the cornea 16 is condensed by the light receiving lens 12 and the image sensor 1
The image is re-imaged at the position e'on the position 4.

The light beams from the ends a and b of the iris 17 form the images of the ends a and b at the positions a'and b'on the image sensor 14 via the light receiving lens 12. When the rotation angle θ of the optical axis a of the eyeball 15 with respect to the optical axis of the light receiving lens 12 (optical axis a) is small, the Z coordinate of the ends a and b of the iris 17 is set to Z.
Assuming a and Zb, the coordinate Zc of the center position c of the pupil 19 is expressed as Zc≈ (Za + Zb) / 2.

Further, since the Z coordinate of the midpoint of the corneal reflection images d and e and the Z coordinate Zo of the center of curvature O of the cornea 16 coincide with each other,
Z coordinates of the generation positions d and e of the corneal reflection image are Zd and Ze, a standard distance from the center of curvature O of the cornea 16 to the center C of the pupil 19 is L _OC, and a coefficient that considers an individual difference with respect to the distance L _OC Is Al, the rotation angle θ of the eyeball optical axis a substantially satisfies the relational expression of (Al * L _OC ) * sin θ≈Zc− (Zd + Ze) / 2 (1). Therefore, by detecting the positions of the respective feature points (corneal reflection images d and e and the edges a and b of the iris) projected on a part of the image sensor in the visual line arithmetic processing device as shown in FIG. The rotation angle θ of the optical axis a of the eyeball can be obtained. At this time, the equation (1) can be rewritten as β (Al * L _OC ) * sin θ≈ (Za ′ + Zb ′) / 2− (Zd ′ + Ze ′) / 2 (2). However, β is a magnification determined by the position of the eyeball with respect to the light receiving lens 12, and is substantially obtained as a function of the interval | Zd′−Ze ′ | of the corneal reflection image. The rotation angle θ of the eyeball 15 can be rewritten as θ≈ARCSIN {(Zc′−Zf ′) / β / (Al * L _OC )} (3). However, Zc'≈ (Za ′ + Zb ′) / 2 and Zf′≈ (Zd ′ + Ze ′) / 2. By the way, since it does not match the optical axis a of the photographer's eye, when the horizontal rotation angle θ of the optical axis a of the photographer's eye is calculated, the angle correction δ between the optical axis of the eye and the visual axis should be performed. Thus, the horizontal line of sight θH of the photographer can be obtained. Letting B1 be a coefficient considering the individual difference with respect to the correction angle δ between the optical axis a of the eyeball and the visual axis, the horizontal line of sight θH of the photographer is calculated as θH = θ ± (B1 * δ) (4) To be Here, if the angle of rotation to the right of the photographer is positive, the sign ± is selected as + when the eye of the photographer looking through the observation device is the left eye, and when the eye of the photographer is the right eye.

Further, in the figure, the eyeball of the photographer is Z-X.
It shows an example of rotation in a plane (eg horizontal plane),
The same can be detected when the eyeball of the photographer rotates in the XY plane (for example, a vertical plane). However, since the vertical component of the line of sight of the photographer coincides with the vertical component θ ′ of the optical axis of the eyeball, the vertical line of sight θV is θV = θ ′. Further, from the line-of-sight data θH, θV, the position (Zn, Y
n) is Zn≈m * θH≈m * [ARCSIN {(Zc′−Zf ′) / β / (Al * L _OC )} ± (B1 * δ)] (5) Yn≈m * θV Desired. However, m is a constant determined by the finder optical system of the camera.

Here, the values of the coefficients Al and B1 for correcting the individual differences of the eyeballs of the photographer have the photographer fixate the index arranged at a predetermined position in the viewfinder of the camera, and the position of the index. And the position of the fixation point calculated according to the equation (5).

The calculation for obtaining the line of sight and gazing point of the photographer in this embodiment is executed by the software of the microcomputer of the line-of-sight calculation processing device based on the above equations.

The coefficient for correcting the individual difference of the line of sight is obtained, and the position of the line of sight of the observer looking into the viewfinder of the camera is calculated by using the formula (5), and the line of sight information is used for the focus adjustment or exposure of the photographing lens. It is used for control.

[0017]

As described above, the line-of-sight direction / coordinates of the photographer are determined by the corneal reflection image (“Purkinje image”, hereinafter “infrared light emitting diode” (hereinafter referred to as “IRED”) for illumination. P image ”), and can be calculated from the positional relationship of the pupils in the eyeball image. This specific detection method is disclosed in Japanese Patent Application No. 3-121098 filed by the present applicant.

Since the P image is a corneal reflection image of IRED,
The brightness is high in the whole eyeball image, and the image is IRED
Since it is a reflection image of the light emitting surface of the chip, its size is also extremely small. Therefore, the P image can be extracted from the eyeball image signal by utilizing the feature. In particular,
When the luminance value of a pixel in the eyeball image signal is equal to or more than a predetermined value and the luminance difference between the pixel and a peripheral pixel is equal to or more than a predetermined value, the pixel (or a peripheral pixel area including the pixel) is converted into a P image. I am trying to make it a candidate.

Furthermore, in order to detect the distance between the camera and the eyeball, two IREDs are turned on in a set and P images are used in pairs. Therefore, when there are a plurality of P image pair candidates, an appropriate interval is set. A P-image pair having is selected.

As shown in FIG. 1, the high-intensity part is indicated by (52
If there is only one set a, 52b), it can be easily detected that the high-intensity part is a correct P image under the conditions described above.

However, when there are a plurality of high-intensity portions as shown in FIG. 2, it often happens that the correct P image is erroneously selected only by the above-mentioned conditions, which is very inconvenient for the sight line detection.

[0022]

SUMMARY OF THE INVENTION The present invention is directed to solving the above-mentioned problems.

That is, when the line-of-sight result calculated based on the initially selected P image pair is obviously not normal and there are more candidate P image pairs as shown in FIG.
By trying to calculate the line of sight using another P image pair, it is possible to select the correct P image.

Specifically, means for illuminating the eyeball of the observer, means for detecting the pupil part from the eyeball image of the observer, means for detecting a corneal reflection image from the eyeball image of the observer,
A visual axis detection device having means for selecting a corneal reflection image from a plurality of corneal reflection images obtained by the detection means, and means for detecting the line of sight of an observer from the positional relationship between the pupil and the selected corneal reflection image. That is, it is provided with a unit that determines whether or not the result of the detected line of sight is normal, and a unit that reselects the corneal reflection image according to the result of the determination unit.

[0025]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS FIG. 3 is a schematic view of the essential parts of a first embodiment when the present invention is applied to a single-lens reflex camera.

In each of the drawings, reference numeral 1 is a taking lens, which is 2 for convenience.
Although shown with one lens, it is actually composed of a larger number of lenses. Reference numeral 2 denotes a main mirror, which is obliquely contacted with or retracted from the photographing optical path according to the observation state of the subject image by the finder system and the photographing state of the subject image. Reference numeral 3 denotes a sub-mirror, which reflects the light flux transmitted through the main mirror 2 toward a focus detection device 6 described below below the camera body.

Reference numeral 4 is a shutter, and 5 is a photosensitive member, which is composed of a silver salt film, a CCD or MOS type solid-state image pickup device, or an image pickup tube such as a vidicon.

Reference numeral 6 denotes a focus detection device, which is a field lens 6a and reflection mirrors 6b and 6 arranged near the image plane.
c, a secondary imaging lens 6d, a diaphragm 6e, a line sensor 6f including a plurality of CCDs, and the like.

The focus detecting device 6 in this embodiment uses a well-known phase difference method, and as shown in FIG. 4, a plurality of regions (5 places) in the observation screen (in the viewfinder field) are used as distance measuring points. The focus detection point is configured to be capable of focus detection.

Reference numeral 7 is a focusing plate arranged on the planned image forming surface of the taking lens 1, 8 is a pentaprism for changing the finder optical path, and 9 and 10 are image forming lenses for measuring the brightness of the subject in the observation screen. It is a photometric sensor. The imaging lens 9 conjugately connects the focusing plate 7 and the photometric sensor 10 via the reflection optical path in the penta roof prism 8.

Next, an eyepiece lens 11 having a light splitter 11a is arranged behind the exit surface of the penta roof prism 8 and is used for observation of the focusing plate 7 by the photographer's eye 15. The light splitter 11a includes, for example, a dichroic mirror that transmits visible light and reflects infrared light.

Reference numeral 12 is a light-receiving lens, and 14 is an image sensor in which a photoelectric element array such as a CCD is two-dimensionally arranged so as to be conjugated with the vicinity of the pupil of the photographer's eye 15 at a predetermined position with respect to the light-receiving lens 12. ing. 13a to 13f
Are infrared light emitting diodes which are illumination light sources.

Reference numeral 21 is a high-intensity superimposing LED that can be visually recognized even in a bright subject, and the emitted light is reflected by the main mirror 2 via the projection prism 22 and is provided on the display portion of the focusing plate 7. It is bent in the vertical direction by the micro prism array 7a, passes through the penta prism 8 and the eyepiece lens 11 and reaches the photographer's eye 15.

Therefore, the fine prism array 7a is provided at a plurality of positions (distance measuring points) corresponding to the focus detection area of the focusing plate 7.
Are formed in a frame shape, and five superimposing LEDs 21 (each of which are LED-L1 and LED
-L2, LED-C, LED-R1, LED-R2).

As a result, as can be seen from the viewfinder field shown in FIG.
Numerals 1, 202, 203, and 204 illuminate in the viewfinder field to display a focus detection area (distance measuring point) (hereinafter referred to as superimpose display).

Reference numeral 23 is a field mask for forming a finder field area. Reference numeral 24 denotes an LCD in the finder for displaying photographing information outside the finder field of view.
-LED) 25.

The light transmitted through the LCD 24 is the triangular prism 2
6 is guided into the field of view of the finder, and 207 of FIG.
It is displayed outside the field of view of the viewfinder as shown by, and the photographer can know the photographing information.

Reference numeral 31 denotes an aperture provided in the taking lens 1, 32
Is a diaphragm driving device including a diaphragm driving circuit 111 described later, 3
Reference numeral 3 is a lens driving motor, 34 is a lens driving member including a driving gear, and 35 is a photocoupler which detects rotation of the pulse plate 36 interlocking with the lens driving member 34 and transmits it to the lens focus adjusting circuit 110. Focus adjustment circuit 110
On the basis of this information and the information on the lens driving amount from the camera side, the lens driving motor is driven by a predetermined amount to move the taking lens 1 to the in-focus position. 37
Is a mount contact that serves as an interface between a known camera and a lens.

FIG. 5 is an explanatory diagram of an electric circuit diagram incorporated in the camera of the present invention. The same parts as those in FIG. 3 are given the same numbers.

A central processing unit (hereinafter referred to as a CPU) 100 of a microcomputer incorporated in the camera body includes a visual axis detection circuit 101, a photometric circuit 102, and an automatic focus detection circuit 10.
3, signal input circuit 104, LCD drive circuit 105, LE
The D drive circuit 106, the IRED drive circuit 107, the shutter control circuit 108, and the motor control circuit 109 are connected. Further, a focus adjustment circuit 1 arranged in the photographing lens
10. Signals are transmitted to the diaphragm drive circuit 111 via the mount contact 37 shown in FIG.

EEPROM 10 attached to CPU 100
Reference numeral 0a has a function of storing the line-of-sight correction data as a storage unit for correcting individual differences in line-of-sight.

The line-of-sight detection circuit 101 A / D converts the output of the eyeball image from the image sensor 14 (CCD-EYE) and sends this image information to the CPU 100. CPU1
As will be described later, 00 extracts each feature point of the eyeball image required for sight line detection according to a predetermined algorithm, and further calculates the photographer's sight line from the position of each feature point.

The photometric circuit 102, after amplifying the output from the photometric sensor 10, performs logarithmic compression and A / D conversion, and sends it to the CPU 100 as luminance information of each sensor. The photometric sensor 10 is the left focus point 2 in the viewfinder field shown in FIG.
SPC-L for photometry of the left area 210 including 00 and 201
And SPC-C for measuring the central area 211 including the center distance measuring point 202, SPC-R for measuring the right area 212 including the right distance measuring points 203 and 204, and their peripheral areas 21.
SPC-A for photometry of 3 and a photodiode for photometry of four areas.

The line sensor 6f shown in FIG. 5 has five sets of line sensors CCD-L2 and CCD-L corresponding to the five distance measuring points 200 to 204 on the screen as shown in FIG.
This is a known CCD line sensor composed of 1, CCD-C, CCD-R1 and CCD-R2.

The automatic focus detection circuit 103 A / D-converts the voltage obtained from these line sensors 6f, and the CPU 10
Send to 0. SW-1 is a switch that is turned on by the first stroke of the release button 41 to start photometry, AF, line-of-sight detection, etc., SW-2 is a release switch that is turned on by the second stroke of the release button, and SW-AEL is an AE lock. Button 43
AE lock switch, SW that turns on by pressing
-DIAL1 and SW-DIAL2 are signal switches 104 provided by a dial switch provided in an electronic dial (not shown).
It is input to the up-down counter of and counts the amount of rotation clicks of the electronic dial.

Reference numeral 105 denotes a known LCD drive circuit for driving the liquid crystal display device LCD to display the aperture value, the shutter speed, the set photographing mode, etc. according to a signal from the CPU 100 on the monitor LCD 42 and the viewfinder LCD 24. Both can be displayed at the same time. L
The ED drive circuit 106 is a lighting LED (F-LED) 25.
Then, the superimposing LED 21 is turned on and controlled to blink. The IRED drive circuit 107 is an infrared light emitting diode (I
RED1 to 6) 13a to 13f are selectively turned on according to the situation.

When the shutter control circuit 108 is energized, it controls the magnet MG-1 for moving the front curtain and the magnet MG-2 for moving the rear curtain to expose the photosensitive member with a predetermined amount of light. The motor control circuit 109 is for controlling the motor M1 for winding and rewinding the film and the motor M2 for charging the main mirror 2 and the shutter 4. These shutter control circuits 108,
A series of camera release sequences operate by the motor control circuit 109.

Next, a flow chart of the operation of the camera having the visual axis detecting device is shown in FIG. 6 and will be described below based on these.

When the mode dial (not shown) is rotated to set the camera from the inoperative state to the predetermined photographing mode (this embodiment will be described based on the case where the shutter priority AE is set), the power of the camera is turned on. Yes (# 100), C
The variable used for the sight line detection of the PU 100 is reset.

Then, the camera stands by until the release button 41 is pressed and the switch SW1 is turned on (# 10).
2). The release button 41 is pressed and the switch SW1 is turned off.
When the signal input circuit 104 detects that N has been input, CP
U100 confirms with the line-of-sight detection circuit 101 (# 10
3).

At this time, if the line-of-sight prohibition mode is set, a specific range-finding point is selected by the range-finding point automatic selection subroutine (# 116) without executing line-of-sight detection, that is, without using line-of-sight information. At this focus detection point, the automatic focus detection circuit 103 performs focus detection operation (# 107).

In this way, there are a photographing mode in which the distance measuring point is selected without using the line-of-sight information (a line-of-sight prohibited automatic focus photographing mode) and a photographing mode in which the distance measuring point is selected using the line-of-sight information (a line-of-sight automatic focus photographing mode). Both of them are provided so that the photographer can arbitrarily select whether to set the line-of-sight prohibition mode.

Although several methods can be considered as the algorithm for automatically selecting the distance measuring points, the near point priority algorithm in which the central distance measuring point is weighted is effective and is not directly related to the present invention. Therefore, the description is omitted.

When the visual axis detection mode is set, visual axis detection is executed (# 104). LED at this time
The drive circuit 106 turns on the illumination LED (F-LED) 25, and the LCD drive circuit 105 sets the LCD in the viewfinder.
The line-of-sight input mark 78 of 24 is turned on, and the photographer can confirm that the camera is performing line-of-sight detection outside the viewfinder screen 207.

The line of sight detected by the line-of-sight detection circuit 101 is converted into the gazing point coordinates on the focus plate 7. C
PU100 selects a distance measuring point close to the gazing point coordinates,
A signal is transmitted to the LED drive circuit 106 to cause the distance measuring point mark to blink using the superimposing LED 21 (# 105).

When the photographer sees that the distance measuring point selected by the photographer's line of sight is displayed, he recognizes that the distance measuring point is not correct, and releases the release button 41 to release the switch SW.
When 1 is turned off (# 106), the camera switches SW
It waits until 1 is turned on (# 102).

As described above, the fact that the distance measuring point is selected according to the line-of-sight information is indicated by blinking the distance measuring point mark in the finder field so that the photographer can arbitrarily select the distance measuring point. You can check whether or not.

When the photographer sees that the distance measuring point selected by the line of sight is displayed, the switch SW1 is continuously turned on.
If continued (# 106), the automatic focus detection circuit 103
Performs focus detection of one or more focus detection points using the detected line-of-sight information (# 107).

It is determined whether or not the distance measuring point selected here cannot be measured (# 108). If the distance measuring point is impossible, the CPU 100 sends a signal to the LCD drive circuit 105 to set the L in the finder.
The focus mark on the CD24 blinks, and distance measurement is NG (impossible)
That is, the photographer is warned (# 118) and continues until SW1 is released (# 119).

Distance measurement is possible, and the focus adjustment state of the distance measurement point selected by the predetermined algorithm is not in focus (#
109), the CPU 100 sends a signal to the lens focus adjustment circuit 110 to drive the taking lens 1 by a predetermined amount (# 11).
7). After driving the lens, the automatic focus detection circuit 103 performs focus detection again (# 107), and determines whether or not the taking lens 1 is in focus (# 109).

If the taking lens 1 is in focus at a predetermined distance measuring point, the CPU 100 causes the LCD drive circuit 105 to operate.
Is sent to turn on the focus mark on the LCD 24 in the finder, and also the signal is sent to the LED drive circuit 106 to display the focus on the focusing point 201 in focus (# 110).

At this time, the blinking display of the distance measuring point selected by the line of sight is turned off, but the distance measuring point displayed in focus is often coincident with the distance measuring point selected by the line of sight. The focus range-finding point is set to a lighting state so that the photographer can recognize that the focus has been achieved. When the photographer sees the focusing point displayed in the viewfinder and recognizes that the focusing point is not correct, he releases the release button 41 and turns off the switch SW1 (# 111). The camera waits until the switch SW1 is turned on (# 10
2).

If the photographer looks at the focus detection point and continues to turn on the switch SW1 (# 1
11), the CPU 100 sends a signal to the photometry circuit 102 to perform photometry (# 112). At this time, the exposure value is calculated by weighting the photometric areas 210 to 213 including the in-focus distance measuring points.

Further, it is judged whether or not the release button 41 is pushed and the switch SW2 is turned on (#
113), if the switch SW2 is OFF, the state of the switch SW1 is checked again (# 111). or,
When the switch SW2 is turned on, the CPU 100 sends signals to the shutter control circuit 108, the motor control circuit 109, and the diaphragm drive circuit 111, respectively.

First, M2 is energized to raise the main mirror 2 and the diaphragm 31 is narrowed down. Then, MG1 is energized to open the front curtain of the shutter 4. The aperture value of the aperture 31 and the shutter speed of the shutter 4 are determined from the exposure value detected by the photometric circuit 102 and the sensitivity of the film 5.
M after a predetermined shutter time (1/250 second, for example)
Energize G2 and close the rear curtain of the shutter 4. When the exposure of the film 5 is completed, the M2 is energized again, the mirror is down and the shutter is charged, and the M1 is energized to advance the film frame, and the series of shutter release sequence operations is completed (# 114). . After that, the camera waits until the switch SW1 is turned on again (# 102).

7 and 8 are flow charts of the visual axis detection of the present invention.

As described above, the line-of-sight detection circuit 101 is a CPU
When a signal is received from 100, line-of-sight detection is executed (# 1
04).

First, the CPU 100 causes the infrared light emitting diodes (IRED) 13a to 13f for illuminating the photographer's eyes.
Select an appropriate combination of IREDs from among the above and light them up (# 201). The selection of IRED is made depending on whether the camera is in the horizontal position or the vertical position by an unillustrated posture switch, or whether the photographer wears glasses.

Next, the image sensor 14 accumulates charges for a predetermined accumulation time (# 202). When the storage is completed, the IRED is turned off together with it (# 203).

The CPU 100 reads the eyeball image of the photographer from the image sensor 14 whose accumulation has been completed, and at the same time, sequentially performs the process of extracting the features of the P image and the pupil portion (# 20).
4). The specific method is described in Japanese Patent Application No. 3-12 by the applicant.
No. 19098, the detailed description is omitted here.

After the reading of the entire eyeball image is completed and the feature extraction of a plurality of P images and pupils is completed, a set of P image positions is detected based on these pieces of information (# 205).
As described above, since the P image is the corneal reflection image of the IRED for eyeball illumination, it appears as a bright spot with a strong light intensity in the image signal, and therefore a pair of P images is detected with that characteristic,
The positions (xd ', yd') and (xe ', ye') can be obtained. In the case of an eyeball image as shown in FIG. 1, since there is only one set of bright spots that look like P images, it is easy to detect it, but as shown in FIG. 2, ghosts caused by the glasses worn by the photographer If so, it often fails to detect the correct P-image pair. P
In order to recognize that the image is an image, in addition to the bright spot condition, it is desirable to add the interval between the P images forming a pair or the size of the bright spot on the sensor to the condition. This is because the ghost with glasses generally has large bright spots.

Anyway, one P image pair (# 205)
If it can be determined by, the center of the pupil (xe ′, ye ′) and the diameter of the pupil (νc) are detected (# 206).

If the P image position and the pupil can be detected from the photographer's eyeball image, the photographer's line-of-sight direction or the coordinates on the finder can be calculated from equation (5) (# 20).
7).

In FIG. 8, the CPU 100 next determines whether or not the result of the detected line of sight can be used (# 20).
8). Specifically, it is determined whether the detected rotation angle of the eyeball is abnormally large in both the horizontal and vertical directions. This is because the photographer cannot normally make a rotation angle that is too large as long as he looks through the viewfinder, and if the rotation angle is calculated to be larger than a predetermined angle (for example, about 20 °), the P image may be erroneously detected. I think there is. Note that this determination criterion may be not only the rotation angle but also the determination criterion whether the gazing point is inside or outside the visual field coordinates.

If the detection result is OK, it is regarded as "succeeding in line-of-sight detection"(# 209) and the line-of-sight detection routine is returned (# 210).

If the detection result is not normal, it is considered that the P image pair may have been erroneously selected, and it is checked whether there is another P image pair (# 211). In the case of an eyeball image as shown in FIG. 2, a pair of P images is (52a, 52b),
There are three sets of (53a, 53b) and (54a, 54b). Of course, the correct P image pair is (52a, 52b)
And the pair of (53a, 53b) and (54a, 54b) is a ghost with glasses. If the P image pair of (52a, 52b) is selected from the beginning, a correct detection result is given, but if the pair of (53a, 53b) is selected first and the visual axis detection is performed, the detection result becomes abnormal. Therefore, if there are a plurality of P image pair candidates as in the example of FIG. 2, another P image pair is selected (# 212), and line-of-sight detection is tried again (T1).

If there is only one P image pair in the first place as shown in FIG. 1, even if it is an incorrect P image pair, it is impossible to reselect another pair. Is regarded as "gaze detection failure"(# 213), and the gaze detection routine is returned (# 214).

In the embodiment described above, if the result of the line-of-sight calculated based on the P image pair selected first is normal, the process ends there, and if not, another P image pair is reselected. When the result is normal, the line-of-sight detection process ends.

However, when there are a plurality of P image pair candidates,
In particular, when the ghost due to the glasses exists in the immediate vicinity of the true P image, if the ghost is used as the P image and the line-of-sight calculation is performed, a modest result may be presented, and the processing may end.

Therefore, in the second embodiment of the present invention, a processing method is proposed in which the line-of-sight calculation is executed for all the P image pair candidates and the most probable result is adopted from the results. is doing.

FIG. 9 shows a flowchart of the second embodiment.

When the line-of-sight calculation is completed in step # 207 of FIG. 7, the process proceeds to step # 220, and it is determined whether the line-of-sight calculation is completed for all P image pair candidates. In the example of the eyeball image shown in FIG. 1, since there is only one P image pair, the calculation is completed once, and the result is regarded as the final result (# 222), and the gaze calculation routine is returned (# 223).

In the example of the eyeball image as shown in FIG. 2, since there are three P image pairs (actually, the high-brightness area having a large spread is regarded as a ghost and is first rejected, and the P image is originally generated. Should not be a pair candidate,
Here, for the sake of explanation, it is assumed that there are three P image pair candidates in the example of the eyeball image of FIG. 2). Even if the result of the line of sight in the P image pair selected first is good, Another P image pair is reset (# 221), and the line-of-sight calculation process is performed again.

When the calculation for the three P image pairs is completed, the most probable line-of-sight result is adopted from those results (# 222). Specifically, a method of adopting a result having the smallest angle among the detected eyeball rotation angles in each P image pair is practical. Then, if the final result of the line of sight is adopted, the line of sight detection routine is returned (# 223).

[0085]

As described above, according to the present invention, when the line-of-sight detection result calculated on the basis of the selected P-image pair is obviously not normal, if another P-image candidate is present. By calculating the line-of-sight again based on the P image pair, a good line-of-sight detection operation can be performed even for a photographer wearing glasses.

[Brief description of drawings]

FIG. 1 is a diagram showing an eyeball image of an ordinary photographer.

FIG. 2 is a diagram showing an eyeball image of a photographer wearing glasses.

FIG. 3 is a schematic view of the essential portions of Embodiment 1 in which the present invention is applied to a single-lens reflex camera.

FIG. 4 is an explanatory view in the viewfinder field of FIG.

FIG. 5 is an electric circuit diagram of an optical device according to the present invention.

6 is a flowchart of the operation of FIG.

FIG. 7 is a flowchart of eye gaze detection according to the present invention.

FIG. 8 is a flowchart of the visual line detection according to the present invention (first
Example)

FIG. 9 is a flowchart of the gaze detection according to the present invention (second
Example)

FIG. 10 is an explanatory diagram of the principle of a conventional eye gaze detection method.

[Explanation of symbols]

 1 Photography Lens 2 Main Mirror 6 Focus Detection Device 6f Image Sensor 7 Focus Plate 10 Photometric Sensor 11 Eyepiece Lens 13 Infrared Light Emitting Diode (IRED) 14 Image Sensor (CCD-EYE) 15 Eyeball 16 Cornea 17 Iris 21 Superimposing LED 24 LCD in the viewfinder 25 LED for illumination 31 Aperture 50 White part of the eye 51 Eye pupil part 100 CPU 101 Line-of-sight detection circuit 103 Focus detection circuit 104 Signal input circuit 105 LCD drive circuit 106 LED drive circuit 107 IRED drive circuit 11 Focus adjustment circuit 200-204 Distance measuring point mark

Claims (2)

[Claims] 1. A means for illuminating an eyeball of an observer, a means for detecting a pupil part from an eyeball image of the observer, a means for detecting a corneal reflection image from the eyeball image of the observer, and the detection means. A line-of-sight detection device having means for selecting a corneal reflection image from a plurality of corneal reflection images, and means for detecting the line of sight of an observer from the positional relationship between the pupil and the selected corneal reflection image, An eye gaze detection device, comprising: a unit that determines whether or not the result is normal, and a unit that reselects a corneal reflection image according to the result of the determination unit. 2. A means for illuminating an eyeball of an observer, a means for detecting a pupil part from an eyeball image of the observer, a means for detecting a corneal reflection image from the eyeball image of the observer, and the means obtained by the means. A line-of-sight detection device having means for selecting a corneal reflection image from a plurality of corneal reflection images, and means for detecting the line of sight of an observer from the positional relationship between the pupil and the selected corneal reflection image, the detection method comprising: And a means for determining the most probable corneal reflection image according to the result of the sight line that has been taken.