JP2744406B2 - Eye gaze detection device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a gaze detecting device for detecting a gaze of a user or a subject.

[0002]

2. Description of the Related Art A camera for inputting information with a line of sight has already been proposed in Japanese Patent Application Laid-Open No. 61-61135, but a method for optically detecting a line of sight is described as "Accurate tw".
o-dimensional eye tracker
using first and fourth Pu
rkinje images ”Journal o
f the Optical Society of
America, vol. 63, No. 8, pa
Ge 921 (1973), a method of detecting using the first and fourth Purkinje images, JP-A-61-172.
No. 552 discloses a method of detecting using the first Purkinje image and the center of the pupil. However, it is the optical axis of the eyeball that can be directly detected by these methods, and the optical axis of the eyeball is different from the line of sight.
It is generally known that the angle is deviated by about {7}.

For this reason, accurate eye gaze detection cannot be performed only by detecting the optical axis of the eyeball, and it is necessary to obtain a deviation between the eyesight and the optical axis of the eyeball and perform correction in order to detect the eye gaze.

[0004]

However, when detecting the line of sight, it is not possible to detect the line of sight correctly without first obtaining the deviation between the line of sight and the optical axis of the eyeball, making the line of sight detection complicated.

[0005]

In view of such problems,
The invention described in claim 1 of the present application is an eyeball optical axis detection unit that detects an eyeball optical axis, and a storage unit that stores a predetermined fixed correction value that corrects a shift between the eyeball optical axis and a line of sight,
By having the line-of-sight detection means for detecting the line of sight of the detection target eye using the fixed correction value, the line of sight can be easily detected.

[0006] Further, the invention described in claim 3 of the present application is:
Detecting means for detecting a correction value for correcting the deviation between the line of sight and the optical axis of the eyeball, and control means for controlling line of sight detection based on the correction value detected by the detecting means; When detection is not performed, by controlling gaze detection based on a preset correction value,
The line of sight can be detected even when the correction value is not detected by the detection means.

[0007]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings. FIG. 1 shows a first embodiment in which the present invention is applied to a single-lens reflex camera. The present invention can be applied to a single-lens reflex camera as well as a camera having a photographic optical path and a finder optical path separately.

In FIG. 1, reference numeral 1 denotes an objective lens, which is shown as a single lens for convenience. However, it is well known that the objective lens is actually composed of a large number of lenses. Reference numeral 2 denotes a main mirror which is inclined or retreated to a photographing optical path depending on an observation state and a photographing state. Reference numeral 3 denotes a sub-mirror, which reflects a light beam transmitted through the main mirror 2 downward from a camera body (not shown). A shutter 4a is used for exposing a light receiving surface of a photosensitive member described later for a predetermined time. Reference numeral 4b denotes a stop arranged in the objective lens 1, and 4c denotes a drive mechanism for moving the objective lens 1 in the optical axis direction for focusing.

Reference numeral 5 denotes a photosensitive member, which is a silver halide film or C
This is an image pickup tube such as a solid-state image pickup device such as a CD or a MOS type, or a Vitaicon. However, if the electronic imaging device has an electronic shutter function, the shutter can be omitted.

Reference numeral 6a denotes a focus detection device, for example, as shown in FIG.
Field lens 20, perforated field mask 2
1, a secondary imaging lens 22 in which two positive lenses are arranged side by side, and a light receiving device in which a plurality of pairs of photoelectric element arrays are arranged. In FIG. 1, the field lens is provided at a predetermined imaging plane position of the objective lens 1 near the sub-mirror 3.
2A is described in detail in Japanese Patent Application No. 62-3154.
No. 90, the multi-hole visual field mask 2
Each of the slits 21a, 21b, and 21c determines the distance measurement field of view. The secondary imaging lens 22 is, for example, a slit 21
a part of the image of the object field defined in FIG.
Re-image on a and 23b. Further, the portion defined by the slit 21b or the slit 21c is re-imaged substantially on the pair 23c and 23d or 23e and 21f of the photoelectric element row.
The light receiving information of each pair of the photoelectric element array is read out as an electric signal, and a correlation operation is performed to calculate a value representing a focus adjustment state of the objective lens with respect to a subject in the distance measurement field of view determined by each slit. . Incidentally, as the focus detection device, FIG.
Can be adopted, or Japanese Patent Application No. 61-1
U.S. Pat. No. 6,824,608, a photoelectric element array is electrically divided using a pair of photoelectric elements longer than usual, and a correlation operation is performed using signals corresponding to the corresponding divided areas. May be applied.

As described above, the focus detection device 6a can detect the focus at a plurality of positions in the field of view. Reference numeral 6b denotes an exposure value detection unit, which includes an imaging lens and a light receiver capable of performing divided photometry. The imaging lens associates the focusing plate 7 and the light receiver, which are arranged on the predetermined imaging surface of the objective lens 1, via the optical path in the penta roof prism 8. The light receiving surface of the light receiver is divided, for example, as shown in FIG. 3, and it is assumed that photometry can be performed for each of the divided areas. The output of the light receiver is input to the microcomputer mc, and the weight can be changed so as to have a photometric sensitivity distribution centered on a plurality of center points.

Next, an eyepiece 9 is disposed behind the exit surface of the penta- roof prism 8 for changing the finder optical path.
It is used for observation of the focus plate 7 by the observer's eye 15. A Fresnel lens may be provided near or integrally with the focus plate. Reference numeral 10 denotes a light splitter for a line-of-sight detection system, which uses, for example, a dichroic mirror that reflects infrared light, and is provided in the eyepiece 9 here. 11 is a condenser lens, 12 is a light splitter like a half mirror, 13 is an LED
And preferably emits infrared light (and near infrared light). The light beam emitted from the infrared illumination light source 13 is emitted along the finder optical path as, for example, parallel light with the power of the rear surface (viewer side surface) of the condenser lens 11 and the prescription lens 9. Reference numeral 14 denotes a photoelectric converter, which will be described in detail later. When the observer properly looks into the eyepiece 9, the rear surface of the eyepiece 9 and the anterior part of the observer's eye with respect to the condenser lens 11, specifically, the vicinity of the pupil. Place them together. That is, the vicinity of the appointment of the finder optical systems 8 and 9 and the photoelectric converter 14
Is conjugately arranged, and the imaging magnification is preferably 1 or less.

With the above configuration, the image forming light beam that has passed through the objective lens 1 is partially transmitted, and is split by the main mirror 2 into a finder light beam and a focus detection light beam. The focus detection light beam is reflected by the sub mirror 3 after passing through the main mirror 2,
The light enters the focus detection device 6. The focus detection device 6 has, for example, three focus detection points 19L, 19C, and 19R in the horizontal direction on the photographing screen of the focus plate 7 shown in FIG. 2B. During shooting, the main mirror 2 is flipped up and the sub-mirror 3
Is laminated and folded on the main mirror, and the film 5 is exposed for a predetermined time by opening and closing the shutter blade 4.

On the other hand, the finder light beam enters the penta- roof prism 8 via the focus plate 7. However, a Fresnel lens or the like integrated with or separate from the focus plate may be provided in the vicinity of 8. The luminous flux is a diopter adjusting eyepiece 9
Accordingly, the subject image on the focus plate 7 is incident on the observer's eye 15 while being enlarged and projected.

The human eye has a corneal surface 16a, a posterior corneal surface 1a.
6b, the lens front surface 18a, and the lens back surface 18b can be regarded as a cemented lens having a cemented surface or interface.
7 is near the front of the crystalline lens. FIG. 4 shows the standard shape of the human eye and the refractive index of each part. FIG. 5 shows an example using this as a model eye.

Generally, there is a certain deviation between the direction of the optical axis X of the eyeball and the direction Y of the gazing point (line of sight). Normally, the gazing point direction Y is on a line connecting the macula B and the anterior ocular node A. When the movement of the eyeball is detected photoelectrically, it is easy to detect the optical axis X of the eyeball by utilizing the axial symmetry of the eyeball optical system, but it is high unless the deviation from the direction of the gazing point is corrected. This is inconvenient when precision is required. The correction method will be described later.

The optical path of the line-of-sight detection system is as follows. The illumination light emitted from the infrared illumination source 13 passes through the half mirror 12,
The mirror 10 is collimated to some extent by the lens 11,
And is incident on the finder optical path. The light splitter 10 is a dichroic mirror that transmits the finder light in the visible region coming from the subject and reflects the illumination light in the infrared region. Is also desirable. However, if an infrared light source having sufficiently high luminance is used, it is possible to design in consideration of a decrease in illumination efficiency and to substitute an ND half mirror.

The infrared illumination light introduced into the finder optical path passes through the rear surface of the eyepiece 9 and illuminates the observer's eyeball. One method is that the illumination light is substantially collimated when entering the eyeball so that the illumination condition is maintained even if the position of the observer's eye changes. This can be realized by adjusting the power arrangement of each unit so that the power of the lens 11 and the power of the rear surface of the eyepiece 9 as a whole are realized. Since the change in the refractive index at each interface of the human eye is as shown in FIG. 4, the illumination light is reflected in the order of the anterior corneal surface, the anterior and posterior lens surfaces, and the posterior corneal surface in accordance with the magnitude of the refractive index change. The position of the reflection image at each interface when the parallel light beam enters is understood as a result of paraxial tracking. These images are Purkinje
inje) images, which are numbered sequentially from the anterior corneal surface, and are referred to as a Purkinje first image, a second image, and the like. 3 except for the third image
The Purkinje images are concentrated on the third surface, that is, immediately after the anterior surface of the lens, and from the consideration of the refractive index change, the first image,
The fourth image is a strong reflection image in the order of the second image. Since the illuminating light forming these images is in the infrared wavelength range, it is not perceived by the eyes and does not hinder finder image observation. For this purpose, the wavelength of the illumination light is desirably longer than 700 nm, and if it is 750 nm or longer, the human eye does not perceive regardless of individual differences.

The light reflected by the observer's eye follows the reverse path, is reflected by the half mirror 12 via the mirror 10 and the lens 11, and is received by the photoelectric converter 14. It is desirable that a visible cut and infrared transmission filter be inserted in the optical path until the reflected light is separated from the finder optical path and received by the photoelectric converter. This is because the corneal reflected light due to the viewfinder image visible light is cut, and only the reflection of infrared illumination light, which is significant as an optical signal, is photoelectrically converted. The photocathode is located at such a position as to form an image near the front surface of the crystalline lens of the observer's eye, that is, near the pupil with the full power of the lens 11 and the rear surface of the eyepiece 9. By this, the first of Purkinje,
The second and fourth images are received in a state of being formed, and the amount of reflected light is not necessarily weak. The third image does not contribute much to the photoelectric conversion signal because the third image is defocused and the light is diffused.

The principle of operation of detecting the optical axis of the eyeball of the eye gaze detecting apparatus according to the present embodiment will be described below. In the apparatus shown in FIG. 1, the infrared illumination light source 13 is used as a point light source, and an illumination point is set so as to emit light from a position at the center of the screen, that is, a point optically equivalent to the position 19c in FIG. The position of the light source 13 is adjusted. In this case, if the optical axis of the observation eyeball passes through the center of the screen, the illumination light source is on an extension of the optical axis of the eyeball.
As already shown in FIG. 3, each Purkinje image is aligned as a point image on the optical axis of the eyeball. FIG. 6A shows the vicinity of the eyeball pupil viewed from the front. In FIG. 6, 41 is an iris, 4
Reference numeral 2 denotes a pupil, and 43 denotes an overlapping Purkinje image. The brightly illuminated iris is observed in a ring, and the dark circular pupil 42
A bright spot where the Purkinje images of the respective surfaces overlap at the center of is observed. On the other hand, if the eyeball is rotating and the optical axis of the eyeball is directed to one of the left and right sides, the illumination light enters the eyeball optical axis obliquely, so each Purkinje image moves to a position eccentric from the center of the pupil. And the direction and amount of movement are different for each reflecting surface, so that a plurality of Purkinje images 43, 4
4 mag is seen from the front. FIG. 6B corresponds to this state. If the optical axis of the observer's eye looks further away from the center of the screen, the tendency becomes stronger as shown in FIG. 6C, and if the observer's eye looks in the opposite direction, the moving direction of the Purkinje image also changes. Invert. These movements are summarized in a graph in FIG. The amounts of movement of the first and fourth Purkinje images that become strong reflection images near the pupil with respect to the rotation angle of the observer's eye are shown. If the movement of these Purkinje images is captured photoelectrically, the direction of the optical axis of the eyeball can be detected.

In the above-mentioned gaze detection method, it is necessary to deal with the parallel movement of the eyeball. Generally, the viewfinder system of a camera is designed so that the entire screen can be seen if the pupil of the observer is within a certain allowable range with respect to the opening position of the eyepiece lens.
In fact, it is known that if the allowable range is narrow, the positional relationship between the camera and the pupil must be accurately maintained, and the camera becomes extremely difficult to use. However, the position of the pupil, that is, the position of the Purkinje image can vary within this allowable range when viewed with reference to the eye gaze detecting device, and it is necessary to compensate for this. The method is not limited, but the following method can be considered as one that can be easily realized from an optical point of view.

The position of the center of the pupil is constantly detected, and the relative displacement of the Purkinje image with respect to the center of the pupil is converted into a visual line detection amount. This method is the most direct and easy to use, but requires a wide view of the photoelectric conversion element because the green of the pupil (that is, the boundary with the iris) must be reliably grasped.

The relative displacement of two or more Purkinje images is measured. In this case, a combination of the first image and the fourth image is easily detected as an object. This is because the image formation position is close and can be measured on the same image plane, and the reflected image is relatively strong.

Regardless of which method is used, the amount of eyeball rotation required to change the position of the observer on the focusing screen is at most about ± 10 ° to 15 °, and the displacement of the Purkinje image due to this is at most. Since the relative translation amount between the eyeball and the camera is several times as large as ± 1 mm, the movement of the line of sight may not be able to be tracked by a simple differential sensor. On the other hand, by using a photoelectric element array formed by integrating several photoelectric elements, the light amount distribution near the pupil of the observer's eye is measured and numerically analyzed so that it is not affected by the position of the eyeball or the pupil diameter. An excellent gaze detection device is configured.

In the application shown in FIG. 1, only a horizontal line-of-sight movement needs to be detected, so a simple configuration using a one-dimensional array of photoelectric elements is shown below. FIG. 8 is for explaining the method. As a result of ignoring the detection capability in the vertical direction, a photoelectric device having a vertically long shape as shown in FIG. Is almost insensitive to vertical translation or rotation of. However, a similar effect can be obtained by bonding a cylindrical lens before the row of photoelectric elements.

In FIG. 8, when a first Purkinje image 62 and a fourth Purkinje image 63 illuminating in the pupil 61 are received by a one-dimensional photoelectric element array 64 (photoelectric converter 14), FIG.
The photoelectric output as shown in (b) is obtained. The high output values on both sides represent the iris. Signals 65 and 66 corresponding to the first and fourth Purkinje images are obtained in the dark pupil.

The pupil center is obtained from the position information of the edge portions 67 and 68. In the simplest case, the pixel numbers that produce an output close to half the average of the iris part at the edge part are i ₁ and i ₂
Is given by i ₀ = (i ₁ + i ₂ ) / 2. Since the position of the Purkinje first image is obtained from the maximum peak locally appearing in the pupil dark part, the relative position relationship between this position and the previous pupil center indicates that:
The rotation state of the eyeball, that is, the direction of the line of sight, can be known from the relationship in the graph of FIG. In this case, the interpretation of FIG. 7 can be considered assuming that the center of the pupil is the origin of the amount of movement of the Purkinje image. Considering that the origin is fixed to the camera, almost only translation of the eyeball is required. The Purkinje fourth image is obtained as the second peak of the pupil dark part, and the calculation may be performed using this position and the position of the previous first image. In this case, it is not always necessary to know the position of the pupil center. However, since the intensity of the Purkinje first image and the fourth image is different by 10 times or more, a photoelectric element array having a relatively high dynamic range is required.

However, the same effect can be obtained by detecting the center position from the edge of the iris (the part covered by the cornea) instead of the center of the pupil. Using the iris to determine the center is highly accurate because the diameter of the iris does not change with the brightness of the outside world unlike the pupil, but it is necessary to be able to detect a wide range because the diameter becomes large .

Although it is insensitive in the direction perpendicular to the arrangement direction of the elements as is clear from FIG. 8, if the photoelectric device is made of a vertically long photoelectric element in the direction perpendicular to the arrangement direction, the iris may be vertically arranged depending on the position of the pupil. There is a limit to how long it will be because it will be picked up. Therefore, when a plurality of photoelectric element arrays composed of elements having a relatively small vertical length are arranged in the vertical direction and the line of sight is detected only by the arrangement that has obtained the most appropriate output, it is insensitive in the vertical direction, and The detection device can always obtain a Purkinje image signal. Further, in the above-described detection only in the one-dimensional direction, a better signal can be obtained if the illumination light source is not a point light source but a slit shape. In this case, a linear light source may be constituted by an LED, or an infrared transmitting and visible cutoff filter and a white light source may be arranged behind the slit.

The method described above is applied to the photoelectric converter 14 shown in FIG.
Is executed by the microcomputer mc to which the input has been input, and the microcomputer mc calculates the focus detection value at the distance measuring position corresponding to the line of sight of the observer from the output of the focus detection device 6a, and according to the calculated value, the drive mechanism 4c Can be driven to focus the objective lens 1.

As described above, a camera controlled by a line of sight according to the present invention, in which the distance measuring point for automatic focus detection is switched according to the obtained line of sight direction. Since the position of the line of sight is determined continuously, it goes without saying that the control target is not limited to three points as shown in FIG.

The output of the exposure detection unit 6b is signal-processed by the microcomputer mc to determine an exposure condition that places emphasis on a position corresponding to the line of sight of the observer, and the shutter 4a and the aperture 4b are synchronized with the release operation. One or both can be set.

When controlling a camera, even when multiple points can be measured by both automatic focus detection and automatic exposure control, only one of them can be used or both can be used simultaneously according to the intention of the observer. Shall be. In addition to focus detection and exposure control, mode display such as shutter priority, aperture priority, program shooting, etc. is changed and displayed in the viewfinder field of view, for example, according to the mode display confirmed at the time of pressing the first step of release operation You can also shoot.

A method for correcting a deviation between the optical axis direction of the eyeball and the gazing point direction will be described below.

A simple method of correcting the deviation is to manually input a correction value or other information. However, in this method, it is one method to separately measure the deviation and input a correction amount corresponding thereto, but generally, the average correction amount of the human eye is stored in a microcomputer in advance. deep. IP of FIG. 1 is an input device for these,
If the correction amount is known in advance, enter that value; otherwise, look in the viewfinder with your right eye or
Enter the distinction of peeping with the left eye. This is because the position of the macula described above is symmetrical between the left and right eyes, and the direction of the deviation is + or-.
Because it becomes. In the majority of human eyes, the deviation between the gaze direction and the direction of the optical axis of the eyeball is about 5 ° to 7 °. Detection of about ± 1 ° to 2 ° is possible.

Next, a method considering individual differences will be described.
When looking through the eyepiece 9, the distance measurement field marks 19C, 19R, and 19L of the focus plate 7 shown in FIG. 2B can be seen. For example, the distance measurement field mark 19C at the center of the observation field is used.
Prior to the measurement, the observer (photographer of the camera) gazes at the distance measurement field mark 19C, and inputs a measurement start signal from the input device IP in that state.

The eye-gaze detecting system operates as described above to measure the optical axis of the eyeball of the observer's eye, and to determine the direction of the optical axis of the eyeball, for example, by the displacement of the Purkinje first image with respect to the center of the pupil or the Purkinje first image. It is quantified as the amount of relative displacement with the fourth image. At this time, since there is a physiological characteristic that the human line of sight tends to fluctuate considerably, signal processing software such as adopting the direction of the optical axis of the eyeball which occurs most frequently within a certain period of time may be used.

The measurement result obtained by the visual axis detection system is stored in a storage element in the microcomputer mc.

The recording element is preferably a nonvolatile EEPROM or the like, but is not limited to this. For example, a RAM backed up by a battery may be used. By providing such an operation state, the visual axis direction can be obtained in a situation where it is determined that the observer is gazing at the center of the screen. At the time of framing for photographing, a gazing point on the screen is obtained by calculating a relative difference between the measured direction of the optical axis of the eyeball and the direction of the optical axis of the eyeball at the time of gazing at the center of the screen. Expressed mathematically, for example, when the position of the first Purkinje image with respect to the pupil center point or the iris center point is x, the gazing point direction X is expressed as X = k (x−x ₀ ) (1) It is. Here, x ₀ is x when the observer is gazing at the center of the screen, and k is a proportional constant, which is determined by a finder system constant as a main factor.

Further, in order to increase the detection accuracy, the following embodiment is preferably employed.

That is, a slight difference generally occurs between the gaze direction detected by the gaze detection system and the actual gaze direction of the observer. Therefore, it is effective to confirm the detection result and adjust if there is a deviation, and it is better to detect again if the deviation is large.

FIG. 9A shows a focus plate, but the observation field of view also looks like this. Reference numeral 71 denotes a display mark indicating a detection result, which is displayed by using, for example, a liquid crystal display or an EL display laminated on a focus plate, or an optical display that illuminates the diffraction grating from the side. x ₀ is preset to an appropriate value. FIG. 9B shows a part of a liquid crystal display. 73a is a liquid crystal layer, which is formed of a uniform transparent electrode layer 7.
3b and a transparent electrode layer 73c arranged in a discontinuous line shape are sandwiched therebetween, and further sandwiched by a polarizing sheet 73d. Power can be sequentially supplied to the electrodes of the lower transparent electrode layer 73c to perform display.

The observer can look into the viewfinder eyepiece 9 and observe the display mark 71. At this time, when the observer gazes at a desired subject (not shown) in the observation field, the subject and the display mark 71 overlap. If so, the detection was accurate. However, if the display mark 71 deviates from the observer's subjective gazing point 72, for example, the position of the subject or the center distance measurement mark, etc., there will be an error in detection, so it is better to make adjustment. .

When the correction amount is measured, the distance measurement mark is used in the above-described example. However, a display mark on a display may be used, for example, displayed at the center of the screen. Blinking helps to connect the gaze.

The observer recognizes the position of his / her gaze point,
Until the position detected by the camera as the fixation point matches,
Expression (1) by means of input device dial, switch, etc.
Changing the constants x ₀ of. Observer may be fixed to where x ₀ if Mitomere the detection display position of the own subjective sight and camera match. Input means of the x ₀ is operated by the resistance partial pressure of the constant voltage power supply as, for example, FIG. 10 (a), AD
It may be converted and associated with x ₀ , or digitally, the contents of the register 81 storing x ₀ may be raised and lowered by two opposite switches as shown in FIG. . In the case of the above-mentioned method, a display is required, but there is no need to guarantee that the observer fixes the reference point at the time of measurement.

In the camera of the present invention, in order to detect a gazing point with high accuracy, the content of the invention includes correcting an individual difference between a deviation between the direction of the optical axis of the eyeball and the gazing point direction. When the photographer changes, the amount of the above-mentioned shift is slightly different. Therefore, the above-mentioned gazing point display is effective as a foolproof measure against the shift. If the gaze point displayed by the camera overlapping the shooting screen matches the subjective gaze point of the photographer, it can be used as it is, and when the user changes and the two do not match. What is necessary is just to redo the above-mentioned correction value setting. If the gazing point display appears during the gaze detection operation, the necessity of setting the correction value can be instantaneously determined and will not be forgotten.

As described above, when the position of the gazing point is not so strict, the deviation between the direction of the optical axis of the eyeball and the gazing point direction is set to a universal constant that does not depend on the individual difference. May be fixed. In this case as well, the input point of gaze can be displayed to confirm the position.

On the basis of the information on the gazing point position of the observer's eye detected by the above method, for example, the three points 19 in FIG.
Automatic focus adjustment can be performed at one point of L, 19C, and 19R, and automatic exposure correction can be performed as described later. Since the position of the gazing point in the above method can be detected continuously or at an extremely fine pitch, the moving object is not limited to three points as shown in FIG. 1 as a matter of course.

Although the line of sight detection has been described only in the one-dimensional direction, in order to detect not only one direction but also the movement of the visual field in two orthogonal directions, a photoelectric element array in which pixels are arranged two-dimensionally using a square. May be used. If a one-dimensional array including the Purkinje first image is selected for each of the vertical and horizontal directions, the line-of-sight positions in two orthogonal directions can be obtained by a method based on the center of the pupil. That is, as shown in FIG.
A light image near the pupil is formed on a two-dimensionally arranged photoelectric element array, and signals in a vertical and horizontal arrangement of 91 and 92 in the figure may be used. As the photoelectric element row, a known CCD image pickup element,
A MOS type imaging device can be used, and the selection of an array to be calculated vertically and horizontally using the position of the Purkinje first image as an intersection can be easily realized by a microcomputer.

Also in the case of this embodiment, the method of correcting the deviation between the direction of the optical axis of the eyeball and the direction of the gazing point is basically the same. That is, most simply, the average value of the anatomical data of the human eye is used, and a shift correction amount is built in in advance, and correction is applied to the detected direction of the optical axis of the eyeball. Assuming that the gazing point direction is (X, Y), X = k (x−x ₀ ) (2a) Y = k (y−y ₀ ) (2b) where (x, y) is the pupil center or The position (x ₀ , y ₀ ) of the Purkinje first image with respect to the center of the iris is (x, y) when the observer is gazing at the center of the screen.

In order to detect the gazing point more accurately, the correction amount (x ₀ , y ₀ ) is detected for each specific photographer. Methods include, for example, detecting the direction of the optical axis of the eyeball while gazing at the center of the screen, or adjusting the correction amount so that the gazing point detection position display matches the photographer's subjective gazing point. Can be used.

In the above description, it has been assumed that the posture of the camera is always fixed. In order to guarantee the operation of the gaze detection device under more general conditions, it is desirable to detect the relative rotation amount between the eyeball of the observer and the camera.
The most standard situation with respect to the rotational degree of freedom is a state in which the horizontal axis 101 of the observer's eye and the horizontal axis 102 of the camera are parallel as shown in FIG. It often happens that both are inconsistent, such as 13. Most typically, θ = ± 90 °.
12 and 13, reference numeral 103 denotes a single-lens reflex camera using a penta roof prism, and 104 denotes an observer's eyeball for observing a field of view from a finder eyepiece behind the penta roof prism. As a result of the relative rotation between the eyeball and the camera in FIG. 13, the fixation point correction amount (x ₀ , y ₀ ) undergoes the following change.

[0053]

[Outside 1] Correction value according to rotation amount θ by the above formula

[0054]

[Outside 2] May be calculated, and the observer's eye fixation point may be obtained from the optical axis optical axis measurement value by Expression (2).

A general method for measuring θ is preferably a photoelectric method. For example, the position of a part of the eye, such as the outer corner of the eye, is imaged and measured with respect to the camera reference coordinates to obtain the horizontal axis of the observer's eye. 101 can be relatively determined. However, since the horizontal axis of the observer's eye is fixed and only the posture of the camera changes and most of the time the photographing frame is selected,
The above operation of measuring θ can be substantially replaced by detection of the attitude of the camera with respect to the earth's horizon. This includes, for example, a slider 1 coupled to a weight 112 as shown in FIG.
Taking advantage of the fact that 13 faces vertically downward, the variable resistor 111
A detector that detects the posture based on the angle between the reference and the slider 113 is used. In the figure, reference numeral 114 denotes a rotation center of the slider and an output terminal of a divided voltage.

On the other hand, FIG.
The fifth mercury switch 115 may be used. Contact 117
The location of the mercury 116 sealed in the ring 115 is determined by examining where there is electrical continuity between adjacent contacts such as a and 117b, so that the vertical downward direction is detected. If the attitude detectors shown in FIGS. 14 and 15 are incorporated in the camera body, the rotation of the camera is determined. Therefore, the measured value of the optical axis of the eyeball is corrected by using the equation (3) according to the amount of rotation. Gazing point can be detected.

Needless to say, the present invention is not limited in its use to a single-lens reflex camera. FIG. 16 shows an example in which the present invention is applied to an inverted Galilean type finder system. The finder optical system basically includes a concave lens 121 and a convex lens 12.
2 is an afocal system having an angular magnification of 1 or less. In the embodiment shown in FIG. 12A, the block-shaped optical member 123 is disposed between the positive lens and the negative lens, and the finder optical system and the line-of-sight detection optical system are connected by a dichroic mirror or a half mirror 124. The lens 125 collimates the light coming from the infrared illumination light source 127 and forms an anterior ocular segment reflected light on a light receiving surface of the photoelectric element array 128. 126 is a half mirror. The method of gaze detection is the same as the embodiment of FIG. FIG.
(B) is an example in which an infrared illumination system and a detection optical system are separately arranged.

The present invention is suitably used in general for cameras having a finder such as a video camera and a still video camera in addition to a silver halide photographic camera. In particular, the present invention is extremely effective for a video camera that often captures a moving object.

The application of the camera having the visual axis detection system according to the present invention is not limited to the control of automatic focus adjustment. Generally, it can be used as input means for controlling the operation method of the camera. FIG. 17 illustrates an example of a photometric sensitivity distribution in a screen of a photometric device for exposure control of a camera. In FIG. 7A, five local photometric points S _{1 to} S ₅ are arranged in the lower part of the screen. By detecting the line-of-sight direction, it is possible to configure a camera that selects one of these five photometric points and controls the exposure based on the photometric output. FIG.
(B) it is are arranged extensive metering area P ₁ to P ₅ by the outside of the local photometric point. For example, when S ₂ is specified in the line-of-sight direction, taking into account photometric information on both sides with S ₂ as the center,

[0060]

[Outside 3] By calculating the amount V, it is possible to provide a photometric sensitivity characteristic having a spread centering on the gazing point.

[0061]

As described above, claim 1 of the present application is as described above.
The invention described in the above, the eyeball optical axis detection means for detecting the eyeball optical axis, storage means for storing a predetermined fixed correction value for correcting the deviation between the eyeball optical axis and the line of sight, the fixed correction value A line of sight detecting means for detecting the line of sight of the eye to be detected, wherein the shift between the optical axis of the eyeball and the line of sight can be corrected by the fixed correction value, so that a simpler line of sight detection Operation becomes possible.

Further, the invention described in claim 3 of the present application is
Detecting means for detecting a correction value for correcting the deviation between the line of sight and the optical axis of the eyeball, and control means for controlling line of sight detection based on the correction value detected by the detecting means; When detection is not performed, by controlling gaze detection based on a preset correction value,
Even when the detection unit does not detect the correction value, the line of sight can be detected, so that the failure to detect the correction value does not cause the line of sight to be undetectable.

The invention described in claim 2 of the present application is
Detecting means for detecting a correction value for correcting the deviation between the line of sight and the optical axis of the eyeball, and control means for controlling line of sight detection based on the correction value detected by the detecting means; When detection is not performed, by controlling gaze detection based on a preset correction value,
Even when the detection unit does not detect the correction value, the line of sight can be detected, so that the failure to detect the correction value does not cause the line of sight to be undetectable.

[Brief description of the drawings]

FIG. 1 is an optical sectional view showing an embodiment of the present invention.

FIG. 2 is a diagram illustrating a partial configuration of a focus detection device.

FIG. 3 is a diagram illustrating a configuration of an exposure detection unit.

FIG. 4 is an explanatory diagram of a human eye.

FIG. 5 is a schematic sectional view of a human eye.

FIG. 6 is a diagram showing a reflection image of an eye.

FIG. 7 is a diagram showing the movement of a Purkinje image.

FIG. 8 is a diagram illustrating detection of a reflected image.

FIG. 9 is a plan view of a focus plate.

FIG. 10 is a diagram showing an adjuster.

FIG. 11 is a diagram illustrating two-dimensional detection of a reflected image.

FIG. 12 is a diagram illustrating a change in the attitude of a camera.

FIG. 13 is a diagram illustrating a change in the attitude of the camera.

FIG. 14 is a diagram showing a posture detector.

FIG. 15 is a diagram showing a posture detector.

FIG. 16 is an optical sectional view for explaining another embodiment.

FIG. 17 is a view showing a visual field.

FIG. 18 is a diagram illustrating a conventional technique.

[Explanation of symbols]

 Reference Signs List 2 Main mirror 3 Sub-mirror 6a Focus detection device 6b Exposure control photometric device 7 Focus plate 8 Penta-Dach prism 9 Eyepiece lens 10,12 Light splitter 11 Condenser lens 13 Illumination light source 14 Photoelectric converter

 ──────────────────────────────────────────────────続 き Continuing on the front page (72) Inventor Yasuo Suda 770 Shimo-noge, Takatsu-ku, Kawasaki-shi, Kanagawa Prefecture Inside the Tamagawa Office (72) Inventor Hidehiko Fukahori 770 Shimo-noge, Takatsu-ku, Kawasaki-shi, Kanagawa Canon Inc. Tamagawa Office

Claims (4)

(57) [Claims] 1. An eyeball optical axis detecting means for detecting an eyeball optical axis; a memory means for storing a predetermined fixed correction value for correcting a deviation between the eyeball optical axis and a line of sight; An eye-gaze detecting device for detecting the eye-gaze of the eye to be detected using the eye-gaze detecting means. 2. The gaze detection apparatus according to claim 1, wherein the fixed correction value is a correction value for correcting an average shift amount between the optical axis of the human eye and the line of sight of the human eye. 3. A detecting means for detecting a correction value for correcting a deviation between the line of sight and the optical axis of the eyeball, and a control means for controlling the line of sight detection based on the correction value detected by the detecting means; Is a gaze detection device that controls gaze detection based on a preset correction value when detection by the detection means is not performed. 4. The gaze detection device according to claim 3, wherein the preset correction value is a correction value for correcting an average shift amount between the optical axis of the human eye and the line of sight of the human eye.