JP2002301030A - Equipment to detect sight line - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable each person to use equipment by minimizing the influence of fluctuation of a sight line among the each person. SOLUTION: This equipment includes a visual line detecting means for detecting an observer's sight line; an area selecting means for dividing the screen of the equipment into a plurality of areas for control of the various movements of the equipment and for selecting an area from which information should be obtained according to the output of the sight line detecting means; a fixation point distribution computing means for obtaining information about a plurality of positions of visual lines when the observer fixes his sight at the same point; and a varying means for varying either the size of the area (70-74 or 70, 72 and 74) to be selected by the area selecting means or the number of areas and the size of each area.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an improvement in a device having a line-of-sight detection function for detecting a line of sight of an observer.

[0002]

2. Description of the Related Art Conventionally, the direction of the line of sight of a photographer (observer) is detected, and the photographer can determine which region (position) in the finder field of view.
The gaze direction of the photographer is detected by a gaze detection unit provided in a part of the camera, and various photographing functions such as automatic focus adjustment and automatic exposure are controlled based on a signal from the gaze detection unit. Various cameras have been proposed.

[0003] For example, the present applicant has disclosed in
In Japanese Patent Publication No. 1 (1993), there is provided a line-of-sight detecting means for detecting a gaze direction of a photographer, and automatic exposure control means having a plurality of photometric sensitivity distributions, and a focus detecting means based on an output signal from the line-of-sight detecting means. A camera that controls the driving of an automatic exposure control unit has been proposed.

[0004]

When using not only the camera described above but also a device having a line-of-sight detection function, the observer looks at the display of the area to be selected on the display section of the device. However, this act of gazing itself is a factor that causes a large error in performing the gaze detection.

In other words, since the ability to gaze at one point or one direction for a predetermined time differs from individual to individual, even if the user gazes at the display of the area to be selected on the display unit of the device having the gaze detection function. It has happened that one can select the target area, but another cannot.

That is, the variation in the ability to gaze at one point or one direction for a predetermined time has been a major obstacle for anyone to use the gaze detection function.

As one countermeasure against the above problem, Japanese Patent Laid-Open No. 4-101126 discloses a method in which the screen is divided into a plurality of regions, a plurality of gaze detections are performed, and a gaze position exists for each of the regions. A proposal is made to determine the point of regard and area based on frequency and parking time. That is, the purpose is to average and stabilize the variation in the line-of-sight position by collecting a plurality of line-of-sight positions.

However, if the line-of-sight detection is always performed continuously a plurality of times, it takes a long time to finally determine the line-of-sight position, so that the responsiveness of the line-of-sight detection device is deteriorated.

Further, the present applicant has disclosed Japanese Patent Application Laid-Open No. 6-308373.
In the publication, in a camera for selecting an area for obtaining information for controlling various operations of the camera based on the output of the line-of-sight detecting means from a shooting screen, the selected area is determined by the reliability of the output of the line-of-sight detecting means. A camera with a visual line detection function characterized by switching is proposed. For example,
When the reliability of the line of sight is low, the number of selectable areas is switched so that five selectable areas are originally three.

[0010] The reliability of the eye-gaze detecting means described herein means the evaluation of the contrast of the eyeball image or the size of the pupil, or the calculated Purkinje image and the position of the center of the pupil. However, the effect of these reliability judgments is actually effective when the eyeball image itself is temporarily defective for some reason, and it is caused by the difference in the ability to gaze, which is originally a physiological problem of human beings, as described at the beginning. It does not solve the variation in the line of sight.

(Object of the Invention) An object of the present invention is to minimize the influence of variations in the line of sight caused by individual differences in gazing ability, and to select an intended area for each person who uses the device. It is an object of the present invention to provide a device with a visual line detection function that can be performed.

[0012]

In order to achieve the above object, the present invention provides a gaze detecting means for detecting a gaze of an observer and a plurality of screens of the device for controlling various operations of the device. In a device with a line-of-sight detection function having an area selection means for selecting an area from which information is to be obtained based on the output of the line-of-sight detection means, a plurality of pieces of line-of-sight position information when the observer gazes at the same place Gazing point distribution calculating means for obtaining, based on the output of the gazing point distribution calculating means,
A device with a line-of-sight detection function having a size of an area to be selected by the area selecting means, or a variable means for changing the number of areas and the size is provided.

[0013]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the present invention will be described in detail based on illustrated embodiments.

(First Embodiment) First, a first embodiment of the present invention will be described with reference to FIGS.

FIG. 1 is a schematic diagram showing a main part of an embodiment in which the present invention is applied to a single-lens reflex camera.
FIG. 3B is a diagram showing an upper surface and a rear surface of the single-lens reflex camera of FIG. 1, and FIG. 3 is a diagram showing an inside of a finder of the single-lens reflex camera of FIG.

In FIG. 1, reference numeral 1 denotes a photographing lens, which is shown by two lenses for convenience, but is actually composed of a larger number of lenses. Reference numeral 2 denotes a main mirror which is inclined or retreated to a photographing optical path depending on the state of observation of the subject image by the finder system and the state of photography of the subject image. Reference numeral 3 denotes a sub-mirror, which reflects a light beam transmitted through the main mirror 2 toward a later-described focus detection device 6 below the camera body.

Reference numeral 4 denotes a shutter. Reference numeral 5 denotes a photosensitive member, which is formed of a silver halide film, a solid-state image pickup device such as a CCD or MOS type, or an image pickup tube such as a vidicon. Reference numeral 6 denotes a focus detection device, which includes a field lens 6a, reflection mirrors 6b and 6c, a secondary imaging lens 6d, a diaphragm 6e, a line sensor 6f including a plurality of CCDs, and the like, which are arranged in the vicinity of the image plane. . The focus detection device 6 in the present example performs focus detection by a well-known phase difference method.
As shown in FIG. 3, a plurality of areas (five points indicated by focus detection area marks 70 to 74) in the field of view are set as focus detection areas so that focus detection can be performed in each of the focus detection areas. Have been.

Reference numeral 7 denotes a focusing plate arranged on a predetermined image forming plane of the photographing lens 1, and 8 denotes a pentaprism for changing a finder optical path. Reference numerals 9 and 10 denote an imaging lens and a photometric sensor for measuring the luminance of the subject in the observation screen, respectively. The imaging lens 9 conjugates the focusing plate 7 and the photometric sensor 10 via a reflection optical path in the pentaprism 8. Have a relationship.

Reference numeral 11 denotes an eyepiece 11 provided behind the pentaprism 8 and having a light splitter 11a. The eyepiece 11 is used for observing the focus plate 7 with an eyeball 15 of a photographer. The light splitter 11a is composed of, for example, a dichroic mirror that transmits visible light and reflects infrared light.

The main mirror 2, focus plate 7, pentaprism 8, and eyepiece 11 constitute a finder optical system.

Reference numeral 12 denotes an imaging lens. Reference numeral 14 denotes an image sensor (CCD-EY) in which a photoelectric conversion element array such as a CCD is two-dimensionally arranged with 80 pixels vertically and 100 pixels horizontally.
E), and is arranged so as to be conjugate with the vicinity of the pupil of the eyeball 15 of the photographer at a predetermined position with respect to the imaging lens 12, and the imaging lens 12 and the image sensor 14 (CC
D-EYE) constitutes a light receiving means for detecting the line of sight. Reference numerals 13a to 13h denote illumination means each including a light emitting element for illuminating the vicinity of the pupil of the eyeball 15 of the photographer. These light emitting devices include an infrared light emitting diode (hereinafter, referred to as an infrared light emitting diode).
2B, and is disposed around the eyepiece lens 11 as shown in FIG. 2B, and emits a light at the time of a single line-of-sight detection. IRED.

As described above, the eye-gaze detecting device is constituted by the light receiving means, the illuminating means, and the above-described dichroic mirror 11a.

Reference numeral 21 denotes a high-intensity superimposing LED that can be visually recognized even in a bright subject. Light emitted from the superimposing LED is reflected by the main mirror 2 through a light projecting prism 22 and is displayed on the focusing screen 7. Is bent in the vertical direction by the micro prism array 7a provided to the photographer, and reaches the photographer's eye 15 through the pentaprism 8 and the eyepiece 11.

That is, as can be seen from the finder visual field shown in FIG. 3, each of the focus detection area marks 70 to 74 shines in the finder visual field, so that the focus detection area can be displayed. Is called superimposed display).

In FIG. 3, dot marks 70 'and 74' are engraved inside the left and right focus detection area marks 70 and 74, respectively, for correcting a line-of-sight detection error due to an individual difference of an eyeball. It shows a target when collecting line-of-sight correction data (referred to as calibration).

Further, 60 to 64 drawn by broken lines in FIG.
The five areas up to are eye-gaze effective areas set in order to associate the photographer's gaze position calculated by gaze detection with the five focus detection areas. For example, the calculated gaze position is within the area 61. If there is, focus detection is performed by a CCD-L1 sensor (described later) corresponding to the focus detection area mark 71 in the area 61. The line-of-sight effective areas 60 to 64 substantially match the divided shape of the photometric sensor 10 in FIG.
It is a split sensor. Although the details will be described later, the five visual line effective areas may be changed to three areas according to the degree of the photographer's gazing point distribution obtained at the time of calibration. One area is selected from the three areas, and focus detection is performed.

At the bottom of the finder screen in FIG.
Is a shutter speed display, 52 is an aperture value display segment,
50 is a line-of-sight input mark indicating that it is in a line-of-sight input state;
Reference numeral 53 denotes a focusing mark indicating the focusing state of the taking lens 1. Reference numeral 24 denotes an LCD in the finder for displaying photographing information outside the finder field of view (hereinafter also referred to as F-LCD).
The illumination is performed by the illumination LED 25.

The light transmitted through the F-LCD 24 is guided by the triangular prism 26 to the outside of the viewfinder as shown in FIG. 3 so that the photographer can know various photographing information.

Returning to FIG. 1, reference numeral 31 denotes an aperture provided in the taking lens 1, reference numeral 32 denotes an aperture driving device including an aperture driving circuit 111, reference numeral 33 denotes a lens driving motor, and reference numeral 34 denotes a lens driving member including a driving gear and the like. is there. Reference numeral 35 denotes a photocoupler which detects rotation of the pulse plate 36 interlocked with the lens driving member 34 and transmits the rotation to the lens focus adjustment circuit 110. The focus adjustment circuit 110 uses this information to drive the lens from the camera. The lens driving motor 33 is driven by a predetermined amount based on the information on the amount, and the photographing lens 1 is moved to a focusing position. Reference numeral 37 denotes a mount contact which serves as an interface between a known camera and a lens.

Reference numeral 27 denotes a posture detection switch such as a mercury switch, which detects whether the camera is held in a horizontal position or a vertical position.

In FIGS. 2A and 2B, reference numeral 41 denotes a release button. Reference numeral 42 denotes a monitor LCD as an external monitor display device, which comprises a fixed segment display section for displaying a predetermined pattern and a 7-segment display section for displaying a variable numerical value. Reference numeral 44 denotes a mode dial for selecting a shooting mode and the like. By matching the indicator 43 engraved on the camera body with the display, the shooting mode is set based on the displayed content. For example,
Locking position to deactivate the camera, position in automatic shooting mode controlled by a preset shooting program, manual shooting mode in which the photographer can certify shooting contents, program AE, shutter priority AE, aperture priority AE, depth of field Each shooting mode of priority AE and manual exposure can be set. In addition, "CA"
The "L" position is also in the mode dial 44, and the "CA"
By operating the electronic dial 45 described later in the "L" position, it is possible to turn on / off the line-of-sight input, and execute and select the calibration.

Reference numeral 45 denotes an electronic dial, which is rotated to generate a click pulse, so that the mode dial 4
This is for selecting a set value that can be further selected from among the modes further selected from among the modes selected in Step 4. For example, when a shooting mode with shutter priority is selected with the mode dial 44, the currently set shutter speed is displayed on the LCD 24 in the viewfinder and the LCD 42 for monitoring. When the photographer rotates the electronic dial 45 while watching this display, the shutter speed is sequentially changed from the currently set shutter speed according to the rotation direction.

The other operation members are not directly related to the present invention, and the description is omitted.

FIG. 4 is a block diagram showing an electrical configuration incorporated in the single-lens reflex camera having the above-described configuration. The same components as those in FIG. 1 are denoted by the same reference numerals.

A central processing unit (hereinafter referred to as a CPU) 100 of a microcomputer built in the camera body includes a line-of-sight detection circuit 101, a photometry circuit 102, an automatic focus detection circuit 103, a signal input circuit 104, an LCD drive circuit 1
05, LED drive circuit 106, IRED drive circuit 10
7, a shutter control circuit 108 and a motor control circuit 109 are connected. Further, signals are transmitted to the focus adjustment circuit 110 and the aperture driving circuit 111 disposed in the taking lens 1 via the mount contact 37 shown in FIG.

EEPROM 10 attached to CPU 100
Reference numeral 0a has a storage function of a line of sight correction data for correcting an individual difference in line of sight as storage means.

The eye-gaze detecting circuit 101 converts an eyeball image signal from the image sensor 14 (CCD-EYE) into an A / D signal.
The image information is converted and transmitted to the CPU 100. CPU
100 extracts each feature point of the eyeball image required for gaze detection according to a predetermined algorithm, and further calculates the gaze of the photographer from the position of each feature point. The photometric circuit 102
After amplifying the signal from the photometric sensor 10, logarithmic compression, A / D
The data is converted and transmitted to the CPU 100 as luminance information of each sensor. The photometry sensor 10 measures the light in six areas (corresponding to the areas 60 to 65 in FIG. 3) in the finder screen.
PC-L2, SPC-L1, SPC-C, SPC-R
It is composed of six photodiodes consisting of 1, SPC-R2 and SPC-M, and is capable of so-called split photometry.

The line sensor 6f comprises five sets of line sensors CCD-L2, CCD-L1, CCD-C, C corresponding to the five focus detection areas in the screen shown in FIG.
A known CCD composed of a CD-R1 and a CCD-R2
It is a line sensor. The automatic focus detection circuit 103 includes:
A / D converting the voltage obtained from the line sensor 6f,
Send to CPU100.

SW1 is a switch which is turned on by the first stroke of the release button 41 to start photometry, AF, line-of-sight detection operation, etc., and SW2 is a release switch which is turned on by the second stroke of the release button 41. SW-DIALI
And SW-DIAL2 are the electronic dial 45 already described.
The dial switch is provided in the inside, and is input to an up / down counter of the signal input circuit 104, and counts a rotation click amount of the electronic dial 45. SW-HV
1, SW-HV2 is a posture detection switch corresponding to the posture detection switch 27, and the posture state of the camera is detected based on this signal.

The state signal input circuit 104 of these switches
And transmitted to the CPU 100 via the data bus.

The LCD drive circuit 105 has a well-known configuration for driving and driving an LCD which is a liquid crystal display element.
At the time of shutter release, the display of the set photographing mode and the like are displayed on the monitor LCD 42 and the LCD (F-LCD) 24 in the viewfinder.
Can be displayed at the same time. The LED drive circuit 106 controls lighting and blinking of the illumination LED (F-LED) 25 and the superimposition LED 21.
The IRED drive circuit 107 controls the illumination power by selectively turning on the IREDs 13a to 13h or changing the output current value (or the number of pulses) to the IREDs 13a to 13h in accordance with an instruction from the CPU 100.

The shutter control circuit 108 controls the magnet MG-1 for running the front curtain and the magnet MG-2 for running the rear curtain when energized, and exposes the photosensitive member to a predetermined amount of light. The motor control circuit 109 includes a motor M1 for winding and rewinding the film and a main mirror 2
And for controlling the motor M2 for charging the shutter 4.

The shutter control circuit 108 and the motor control circuit 109 perform a series of camera release sequence operations.

Next, the operation of the camera having the above configuration having the visual axis detection device will be described with reference to the flowchart of FIG.

When the mode dial 44 is turned to set the camera from a non-operation state to a predetermined shooting mode (this embodiment will be described based on a case where the shutter priority AE is set), the power of the camera is turned off. ON (Step #
100), variables used for line-of-sight detection other than line-of-sight calibration data stored in the EEPROM 100a of the CPU 100 are reset (step # 10).
1). Then, the camera waits until the release button 41 is pressed and the switch SW1 is turned on (step #).
102).

When the release button 41 is depressed and the switch S
When the signal input circuit 104 detects that W1 has been turned on, the photometric timer for 6 seconds operates, during which the camera continuously operates the photometric sensor 10 and the photometric circuit 102, and the CPU 100 captures the photometric value of the scene light of the camera. And the calculation are repeated, and the F-LCD 24 in the finder and the LC for external monitor
In D42, the latest photometric calculation value, that is, the display 51 of the shutter speed and the aperture value display 52 of the photographing lens are always displayed.

At the same time when the switch SW1 is turned on and the photometric timer is activated, the CPU 100 detects through the signal input circuit 104 whether the setting of the mode dial 44 in FIG. 2 is set to ON or OFF. Step # 103). Here, if the line-of-sight input is OFF, that is, the line-of-sight prohibition mode is set, or if the line-of-sight detection described later is unsuccessful, a specific focus detection region is selected without using line-of-sight information by the focus detection region automatic selection subroutine. (Step # 116). Then, in this focus detection area, the automatic focus detection circuit 103 performs a focus detection operation (Step # 107).

Several methods can be considered as an algorithm for automatic selection of a focus detection area, but a near-point priority algorithm in which weighting is applied to a central focus detection area, which is known for a multipoint AF camera, is effective.

When the operation mode of the camera is set to the line-of-sight detection mode in which the line-of-sight detection operation is performed in step # 103, the line-of-sight detection circuit 101 determines which calibration data is used in line-of-sight detection. To confirm. Further, when the gaze calibration data corresponding to the calibration data number is set to a predetermined value and it is recognized that the data is input by the photographer, the gaze detection circuit 101 performs the calibration. The line of sight is detected in accordance with the application data, and the line of sight is converted into the coordinates of the line of sight on the focus plate 7 (step # 104).

The details of the line-of-sight detection operation in step # 104 will be described later.
The gaze detection detected in step 4 is performed to determine whether it is successful (step # 105). The determination conditions here include the Purkinje image, which is a corneal reflection image, the reliability of the pupil center position, the rotation angle of the eyeball, and the like. As a result, if unsuccessful, the process proceeds to a focus detection area automatic selection subroutine of step # 116. If the gaze detection is successful, the CPU 100 selects a focus detection area corresponding to five or three gaze effective areas to which the gaze position coordinates belong (step # 106).
Then, the automatic focus detection circuit 103 executes focus detection in the focus detection area selected in step # 106 or step # 116 (step # 107).

Next, before driving the lens, the CCD
It is determined from the signal of the line sensor whether or not the focus detection of the focus detection area selected by the above flow is possible (step # 108).
U100 sends a signal to the lens focus adjustment circuit 110 to drive the photographing lens 1 by a predetermined amount (step # 109).

Further, it is determined whether or not the photographing lens 1 is in focus in a predetermined focus detection area after driving the photographing lens (step # 110).
The CPU 100 sends a signal to the LCD drive circuit 105 to turn on the focus mark 53 of the LCD 24 in the finder, and also sends a signal to the LED drive circuit 106 to output a superimposing L corresponding to the focused focus detection area.
The ED 21 is turned on, and the focus detection area is illuminated to display an in-focus state (step # 111).

If it is determined in step # 108 that the focus detection is not possible, or if it is determined in step # 110 that the focusing has failed after the lens is driven, then the CPU 100 drives the LCD drive circuit 105 To make the focus mark 53 on the LCD 24 in the viewfinder blink, indicating that the subject could not be focused (step # 117), and again switch SW1
Turns off once and waits for it to turn on again (step # 1).
18).

Here, when the flow returns to step # 111, after the in-focus display of the photographic lens 1 is made, the CPU 100 keeps the state of the photometry timer turned on in step # 102 as O.
It is detected whether or not N is performed (step # 112).
The photographer sees that the focused focus detection area is displayed in the viewfinder, and the focus detection area is incorrect.
Alternatively, when it is determined that the photographing is to be stopped, the hand is released from the release button 41, and when the switch SW1 is turned off, step # is performed.
The process returns to step 102, where the switch SW1 is turned on.

On the other hand, if the photographer looks at the focus detection area displayed in focus and continues to turn on the switch SW1, the CPU 100 causes the photometric circuit 102 to execute the measurement of the field luminance, and Is determined (step # 113). If the switch SW2 is turned on by further pressing the release button 41 (step # 11)
4), the CPU 100 transmits a signal to each of the shutter control circuit 108, the motor control circuit 109, and the aperture drive circuit 111 to perform a known shutter release operation (step # 115).

Specifically, first, the motor control circuit 109
Then, the motor M2 is energized to raise the main mirror 2 and the aperture 31 is stopped down. Then, the magnet MG-1 is energized to open the front curtain of the shutter 4. The aperture value of the aperture 31 and the shutter speed of the shutter 4 are determined by the photometric circuit 1
02 is determined from the exposure value detected and the sensitivity of the film 5. After the lapse of a predetermined shutter time (1/125 seconds),
The magnet MG-2 is energized, and the rear curtain of the shutter 4 is closed. When the exposure of the film 5 is completed, the motor M2 is energized again to perform mirror down and shutter charging, and is also energized to the film feeding motor M1 to feed the film frame, thereby performing a series of shutter release sequence operations. Ends.

After that, the camera turns on the switch SW1 again.
It waits until N is reached (step # 102).

FIG. 6 is a flow chart showing the gaze detection operation algorithm.

In FIG. 6, as described above, the line-of-sight detection circuit 101 executes line-of-sight detection upon receiving a signal from the CPU 100 (step # 104). Eye-gaze detection circuit 101
Determines whether the gaze is detected in the photographing mode or the gaze is detected in the gaze calibration mode (step # 200).

Actually, when the mode dial 44 in FIG. 2 is set to the calibration mode for the line-of-sight detection operation, a calibration (CA
L) Perform the operation (Step # 300).

The mode dial 44 has a line-of-sight detection mode setting. At this set position, calibration data numbers 1, 2, and 3 for registering and executing calibration data for three persons and line-of-sight detection are set. A total of four positions that are not executed can be set by operating the electronic dial 45 in FIG.

Therefore, when the camera is not set to the calibration mode, the line-of-sight detection circuit 101 recognizes which calibration data number the camera is currently set to.

Subsequently, in the case of gaze detection in the photographing mode, the gaze detection circuit 101 first detects the attitude of the camera via the signal input circuit 104 (step # 201). The signal input circuit 104 processes the output signal of the attitude detection switch 27 (SW-ANG) in FIG. 1 to determine whether the camera is in the horizontal position or the vertical position. If the camera is in the vertical position, for example, the release button 41 is pressed. It is determined whether it is in the top direction or the ground (plane) direction.

Next, the IREDs 13a to 13h are selected based on the camera attitude information detected earlier and the photographer's eyeglass information included in the calibration data (step #).
202). That is, if the camera is held in the horizontal position and the photographer does not wear glasses, the IREDs 13a and 13b shown in FIG. 2B are selected. Also, if the camera is in the horizontal position and the photographer wears glasses, the IREDs 13a, 1a, 1a, 1a, 1a, 1b are arranged so that the influence of the reflected light of the photographer's glasses decreases.
Ic of 13c and 13d wider than the interval of 3b
RED is selected.

If the camera is held, for example, with the release button 41 in the top direction or in the vertical position in the ground (plane) direction, a combination of IREDs that illuminates the photographer's eyeball from below, that is, shooting If the user does not wear glasses, 13a and 13f are selected, and if the photographer wears glasses, a combination of 13c and 13e is selected.

Next, the image sensor 14 (hereinafter referred to as CCD)
Preliminary accumulation is performed prior to the actual accumulation of (−EYE) (step # 203). The pre-accumulation of the CCD-EYE 14 means that an image signal is always taken in after a fixed time, for example, 1 ms, is determined immediately before the main accumulation, and the actual accumulation of the eyeball image is made according to the level of the signal level. By controlling the time, a stable eyeball image signal can be obtained. That is, the brightness (external light amount) near the eyeball of the observer is measured based on the image signal output of the preliminary accumulation (step # 204).

Next, the accumulation time of the CCD-EYE 14 and the illumination power of the IRED are set based on the external light amount in step # 204 or information on whether or not the glasses are worn (step # 205). When the accumulation time of the CCD-EYE 14 and the illumination power of the IRED are set, the CPU
Reference numeral 100 denotes an IRED 13 via an IRED drive circuit 107.
Is turned on at a predetermined power, and the gaze detection circuit 1 is turned on.
01 starts accumulation of the CCD-EYE 14 (step # 206). In addition, the CCD-EYE14 set earlier is used.
The accumulation here is terminated according to the accumulation time, and the IRED 13 is also turned off.

Next, the image signals stored in the CCD-EYE 14 are sequentially read out, and the eye-gaze detecting circuit 101 performs A / D conversion.
After the conversion, it is stored in the CPU 100 (step # 207).

Here, the principle of gaze detection will be briefly described with reference to FIGS. 7 and 8. FIG.

FIG. 7 is an image of an eyeball image signal of the CCD-EYE 14. The IREDs 13a and 13b emit light at the cornea 16 of the eyeball 15 in FIG. 8, and the Purkinje 19a shown in FIGS. , 19b.
Reference numeral 17 denotes an iris, and reference numeral 18 denotes a pupil.

Returning to FIG. 6, known line-of-sight detection processing is performed on these image signals (step # 2).
08). That is, the CPU 100 detects the positions of the Purkinje images 19a and 19b, which are virtual images of the pair of IREDs 13a and 13b used for illuminating the eyeball.

As described above, Purkinje images 19a and 19b
Appears as a bright spot having a high light intensity. Therefore, a predetermined threshold value for the light intensity is provided, and a light intensity exceeding the threshold value can be detected as a Purkinje image. The center position of the pupil is calculated by detecting a plurality of boundary points between the pupil 18 and the iris 17 and performing a least square approximation of a circle based on each boundary point. The rotation angle θ of the eyeball is determined from the Purkinje image position and the pupil center position, and the distance between the eyepiece 11 of the camera and the photographer's eyeball 15 is calculated from the interval between the two Purkinje images 19.
Thus, the imaging magnification β of the eyeball image projected on the image No. 4 can be obtained.

From the above, the position coordinates on the focus plate 7 in the line of sight of the photographer are determined using the rotation angle θ of the eyeball, the imaging magnification β, and the individual difference correction information obtained by the calibration. Can be.

When the line-of-sight position is determined in the next step # 209, the operation returns to step # 105 of FIG. 5 for determining whether or not to select an area based on the line-of-sight. Is determined, and a focus detection area corresponding to the area is determined.

Next, the calibration operation algorithm will be described with reference to the flowchart of FIG.

As described above, the calibration is
The photographer fixes the right and left dot marks 74 'and 70' of the focus detection area in the viewfinder visual field, respectively, for a certain period of time, and collects eye-gaze correction data from eyeball image data obtained therefrom. In the present embodiment, the calibration operation itself serves as means for determining the number of times of execution of the eye gaze detection in the eye gaze detection algorithm of FIG.

When the mode dial 44 is set to the "CAL" position, the calibration operation starts (step # 300).

First, the attitude of the camera is detected via the output signal of the attitude detection switch 27 (SW-ANG) and the signal input circuit 104 (step # 301). This is the same detection processing as step # 201 in FIG. Next, the right end 74 'of the dot mark in the focus detection area in the finder field of view is blinked to display a target to be fixed to the photographer (step # 302). At the same time, the calibration data stored in the EEPROM 100a is checked based on the currently set calibration number.
The "CAL" display on the monitor LCD 42 in FIG. 2 is lit as it is, and if not registered, the "CAL" display blinks.

Subsequently, the camera performs an operation of selecting an IRED for illuminating the photographer's eyeball at the time of performing calibration (step # 303). The operation of selecting the IRED in this case is slightly different from the operation described with reference to FIG. 6 and is similar to using the posture information of the camera. However, calibration is performed in the past, and the calibration data is already stored in the camera. If so, the stored IRED
, That is, either the pair for wearing glasses or the pair for non-wearing (the naked eye) is selected from the beginning according to the stored information. On the other hand, when the calibration is performed for the first time, there is no information for selecting the pair of IREDs for wearing or not wearing glasses, so the first eye image illumination of the calibration is performed using the IRED for wearing no glasses. Select a group to emit light, and
According to the determination of the presence or absence of the ghost due to the reflection of the spectacles in the eyeball image signal of D, if the generation of the spectacles ghost is detected, the illumination after the second time is changed to the set of the IRED for the spectacles.

At step # 303, I
The set of RED is determined, and the camera switches the photographer's switch SW.
1 and waits for the ON signal of the switch SW1. When the ON signal of the switch SW1 is detected, the same eyeball image capturing operation as in steps # 203 to # 205 described in the flowchart of FIG. 6 is performed.

That is, the accumulation time of the CCD-EYE 14 and the illumination power of the IRED 13 are set (step # 30).
4) Actually accumulating CCD-EYE14 and IRED13
(Step # 305), and the CCD-EY
The image signals accumulated at E14 are sequentially read out and stored in the CPU 100 after AD conversion (step # 30).
6). Next, the CPU 100 calculates the rotation angle θ of the photographer's eyeball in accordance with a predetermined calculation formula of the AD-converted image signal on the memory (step # 307).

During the eyeball image capturing operation, the blinking of the right end 74 'of the dot mark in the focus detection area in the finder field of view changes to a lit display, and the photographer is informed that the eyeball image capturing operation is being performed. I know.

Subsequently, the calculated rotation angle θ is subjected to a determination as to whether or not the value is appropriate (step # 30).
8). Since the deviation between the optical axis of the eyeball and the visual axis is hardly displaced by several tens of degrees biologically, the threshold value for the determination is set to ±
It is set to 10 degrees. In this step # 308, only the detected rotation angles OK and NG of the detected eyeball are determined, and if the result is OK or NG, the next step # 3
09, if the total number of eyeball rotation angle detections is less than ten, the process returns to step # 304, where the eyeball image capturing operation is performed again, and when the total number of eyeball rotation angle detections reaches ten, this time Determines whether calibration (CAL) has succeeded or failed depending on how many times OK has occurred among the 10 times (step # 310).

In this case, the calibration at the right end 74 'is successful when the rotation angle is detected six times or more, and the calibration operation is started at the left end 70'. If the calibration at the left end 70 'is similarly successful, the distribution of the gazing point (more precisely, the gazing rotation angle) at each of the right end index 74' and the left end index 70 'is obtained.
FIGS. 10A and 10B show an example thereof.

FIGS. 10A and 10B show the photographer a
And two photographers b, who are to be calibrated by the camera, perform calibration by looking at the right end index 74 ′ of each person. Are plotted at the origin of the θx, θy coordinate system for convenience.
Since the average value of every ten data is matched, the distribution has the origin at the average center.

As is clear from FIGS. 10 (a) and 10 (b), even if the photographer b looks at one point (in this case, the index 74 ') with respect to the photographer a. It can be seen that the gazing points vary widely. Therefore, in order to quantify these gazing point distributions, a standard deviation is calculated for each of θx and θy (step # 311). The standard deviation σ is calculated by the following equation (the equation is for θx).

[0087]

(Equation 1)

Therefore, from the above equation, the standard deviations in the X direction and the y direction when the indicator 47 'is watched are represented by σRx,
σRy and standard deviations in the X direction and the y direction when the indicator 40 ′ is watched can be calculated as σLx and σLy, respectively. In the camera of the present description, the eye-gaze effective areas 60 to 64 to be selected for eye-gaze detection are arranged only in the horizontal direction. Therefore, here, the standard deviation “σx = (σRx + σLx) / 2” indicating the gaze point distribution only in the X-direction. And the description is continued below.

Next, the calculated standard deviation σx of the gazing point distribution in the X direction is compared with a threshold value (step # 312). Here, the threshold value is 0.7 degrees. Actually, it should be possible to select the regions 60 to 64 to be selected in the line-of-sight detection if the distance between the centers of the adjacent regions / 2 (here, the line-of-sight angle is approximately 2 degrees) can be separated. However, in addition to variations in gaze, differences in the way the photographer looks through the viewfinder, errors in calibration, errors in gaze detection, etc. affect the gaze detection results. Is set.

Assuming that “σx = σRx” for convenience of explanation,
Since the photographer “a” has a standard deviation σx = 0.39 and falls below the threshold value 0.7, it is determined that the variation in the gazing point distribution is small, and the gaze effective area 60 described with reference to FIG. The line-of-sight selection mode 1 for selecting one area from five areas 64 to 64 is determined (step # 313). On the other hand, since the photographer b has the standard deviation σx = 0.85 and thus exceeds the threshold value 0.7, it is determined that the variation in the gazing point distribution is a large person, and as shown in FIG. The line-of-sight selection mode 2 for selecting one region from the three line-of-sight effective regions 80, 81, 82 is determined (step # 314).

FIGS. 11 (a) and 11 (b) correspond to FIG.
This explains that the photographers a and b described with reference to FIG. 10 (a) and FIG. 10 (b) actually perform gaze detection, and as a result, one gaze effective area is selected.

In FIG. 11A, since the photographer a is determined to be a person with little variation in the gazing point distribution by the gazing point distribution calculation (# 311) at the time of calibration, the focus plate detected by the gaze detection is determined. The coordinate 90 on 7 is 5
The region to be selected in the line-of-sight detection is determined depending on which of the two line-of-sight effective regions 60 to 64 belongs to.
In (a), the line-of-sight effective area 61 is selected, and finally a focus detection operation is performed in the focus detection area 71.

On the other hand, in FIG. 11B, since the photographer b is determined to be a person having a large variation in the gazing point distribution, five regions 60 to 64 are selected with high probability as shown in FIG. 11A. It is presumed to be difficult to do. Therefore, one area is selected from the five focus detection areas, the function of performing focus detection is abandoned, and only three areas in which the intermediate area is intentionally thinned out of the originally five focus detection areas can be selected by gaze detection. By doing so, it is possible to avoid a selection error caused by a variation in the line of sight, and to increase the probability of coincidence with the intention to select. In other words, in FIG. 11B, the region selected by the line-of-sight detection is determined depending on which of the three line-of-sight effective regions 80 to 82 the coordinate 91 on the focus plate 7 detected by the line-of-sight detection belongs to. In FIG. 11B, the visual line effective area 81 is selected.

Returning to FIG. 9, when the number of times of execution of the line-of-sight detection is determined in steps # 313 and # 314, the "CAL" display on the monitor LCD 42 changes to a light-on display indicating successful calibration, and the calibration is performed. The data is stored in the CPU 100. If the calibration is successful, the CPU 100 determines the gaze correction data for correcting the gaze detection error due to the individual difference of the photographer's eyeball obtained by the calibration, and the gaze selection mode determined in step # 313 or # 314. Either 1 or mode 2 is stored in the EEPROM 100a of the CPU 100 in association with each other. If the calibration data has already been registered, the newly collected calibration data is integrated with the past data stored in the memory (step #).
316).

If the number of successful rotation angle detections is less than 6 in step # 310, CAL failure occurs, the "CAL" display on the monitor LCD 42 changes to a blinking display, and the photographer fails calibration. Is notified (step # 315).

In the above description, the standard deviation of the gazing point distribution coordinates of the photographer was calculated and determined to determine the line of sight selection mode for determining how many regions should determine one region from the line of sight information. In addition to the deviation, it may be obtained by statistical processing such as a variance value or a value of a reliability section.

In the above description, the line-of-sight selection mode obtained from the standard deviation of the coordinates of the gazing point distribution is stored in the EEPROM 100a in association with the calibration number, but the value of the standard deviation itself is stored. The CPU 100 may read this standard deviation value at the time of executing the line-of-sight detection when actually photographing the camera, and select the line-of-sight selection mode 1 or 2 on the spot.

Further, it is more preferable to store the history of both the line-of-sight selection mode and the standard deviation, and to update the latest line-of-sight selection mode from the new standard deviation value and the past standard deviation value obtained when the calibrations are stacked. It is practical.

(Second Embodiment) FIGS. 12A to 12C are viewfinder views of a video camera with a visual line detection function according to a second embodiment of the present invention.

In the present video camera, a calibration operation for correcting the gaze position of the photographer is necessary. As described in the first embodiment, the calibration operation using FIG. It is possible to measure the gaze point distribution of the photographer during the calibration operation.

FIGS. 12A to 12C show the viewfinder display when the photographer of the present video camera looks into the viewfinder of the video camera and presses the mode setting button to enter the eye-gaze selection setting mode. I have.

Here, FIG. 12A shows a view displayed on the viewfinder of the video camera in the case of a person who is determined to have a small variation in the gazing point distribution by the gazing point distribution calculation at the time of calibration. Is a subject image picked up by a CCD through a photographic lens of a video camera. On both sides of the subject image, six types of menus from which functions can be selected by gaze detection are displayed in such a manner that a part of the subject image is displayed.

Reference numeral 83 denotes a wide display area for moving the photographic lens of the video camera to the wide angle side, 84 denotes a tele display area for moving the photographic lens to the telephoto side, and 85 denotes whether or not to perform camera shake correction on the image to be shot. 86 is a date display area for setting whether or not to display the year, month, hour, and minute in a video to be shot, and 87 is a fade-in screen that gradually appears from a white screen at the start of video shooting. A fade-in display area for setting whether or not to record a moving image in a format, and a fade-out area for setting whether or not to record a moving image in a fade-out format in which a screen gradually disappears white from a shooting screen at the end of moving image shooting. This indicates a display area.

When it is detected that the line-of-sight position continuously detected in these display areas remains for a predetermined time, the function becomes effective. Further, in the alternative setting, a change form of a formula that changes to a state before the gaze position is valid when the gaze position is detected in the area for a certain period of time is adopted.
The setting is completed, and the setting is completed by pressing the mode setting button again.

On the other hand, FIG. 12B shows a case where the gaze point is displayed on the viewfinder of the video camera when a person whose gaze point distribution is determined to be large in variation by the gaze point collection means during calibration. The menus that can be selected are shown. The wide display area 90, the tele display area 91, the camera shake prevention display area 92, which is a part of the various menus described above, and the next page display area 93 for switching the display to the menu on the next page. A total of four menus are displayed with areas larger than the size of each of the six menus in FIG. For this reason, even a person with a large variation in the gazing point distribution can reliably select the function setting by gaze detection.

In FIG. 12B, 6 in FIG.
Since only three functions can be selected for each,
When the next page display area is selected with the line of sight, FIG.
, The remaining three functions 94 to 96 are displayed and selectable. Reference numeral 97 denotes a previous page display area for returning to the menu screen of the previous page.

According to each of the above embodiments, there is provided means (step # 311 in FIG. 9) for performing a gaze point distribution calculation for obtaining a plurality of pieces of gaze position information when the photographer gazes at the same place. Based on the output of the means for performing the gazing point distribution calculation,
The size of the region to be selected in step # 106 of FIG. 5 (see FIGS. 11A and 11B), or the number and size of the regions (FIGS. 12A, 12B, and 12C) )).

Therefore, the influence of the variation in the gaze position caused by the difference in the ability to gaze, which is a physiological problem of humans, is minimized, and the gaze that can be operated accurately and comfortably for each person who uses the device. The detection function can be executed.

(Modification) In the above embodiment, the case where the present invention is applied to a single-lens reflex camera is described. However, the present invention is also applicable to a camera such as a lens shutter camera and a video camera.
Furthermore, the present invention can also be applied as a device in which a head-mounted line-of-sight detection unit and a display are combined, another device, and a constituent unit.

[0110]

As described above, according to the present invention,
It is possible to provide a device with a line-of-sight detection function capable of selecting an intended region for each person who uses the device, while minimizing the influence of line-of-sight variations caused by individual differences in gaze ability. .

[Brief description of the drawings]

FIG. 1 is a schematic diagram of a single-lens reflex camera according to a first embodiment of the present invention.

FIG. 2 is a diagram showing an upper surface and a rear appearance of the single-lens reflex camera of FIG. 1;

FIG. 3 is a view of a viewfinder of the single-lens reflex camera of FIG. 1;

FIG. 4 is a block diagram showing an electrical configuration of the single-lens reflex camera of FIG. 1;

FIG. 5 is a flowchart showing a series of operations of the single-lens reflex camera of FIG. 1;

FIG. 6 is a flowchart showing a line-of-sight detection operation in step # 104 of FIG. 5;

FIG. 7 is a diagram for explaining the general principle of gaze detection.

FIG. 8 is a diagram for explaining a general principle of line-of-sight detection.

FIG. 9 is a flowchart showing a calibration operation in step # 300 of FIG. 6;

FIG. 10 is a gazing point distribution diagram according to the first embodiment of the present invention.

FIG. 11 is an explanatory diagram of a gaze selection area according to the first embodiment.

FIG. 12 is an explanatory diagram of a line-of-sight selection area according to the second embodiment.

[Explanation of symbols]

 Reference Signs List 6 Focus detection device 6f Image sensor 10 Photometry sensor 13 Infrared light emitting diode (IRED) 14 Image sensor (CCD-EYE) 70-74 Focus detection area mark 100 CPU 101 Line of sight detection circuit 103 Focus detection circuit 106 LED drive circuit 107 IRED drive Circuit 110 Focus adjustment circuit

Claims (3)

[Claims] 1. A gaze detecting means for detecting a gaze of an observer, and a plurality of divided areas in a screen of the device for controlling various operations of the device to obtain information. A gaze detection function device having a gaze detection function for selecting based on an output, a gaze point calculating means for obtaining a plurality of pieces of gaze position information when an observer gazes at the same place; An apparatus with a visual line detection function, comprising: a variable means for changing the size of the area to be selected by the area selecting means or the number of the areas and the size based on the output of the means. 2. The gaze detection function according to claim 1, wherein data of a plurality of gaze positions of the observer obtained by the gaze point distribution calculating means are digitized by statistical processing and stored. machine. 3. The apparatus according to claim 1, further comprising: a calibration unit configured to calculate a correction value for correcting the gaze position calculated by the gaze detection unit. The apparatus with a gaze detection function according to claim 1, wherein the distribution calculation unit operates.