JPH09262209A - Photographing apparatus provided with means for detecting line of sight and means for detecting contact with eye - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a photographing apparatus capable of surely detecting contact with eye. SOLUTION: An electric current value of a light emitting diode as a lighting means for detecting a line of sight and a contact detecting means at the same time is increased on contact detection than that on detection of a line of slight. Therefore, the level of an output of corneal reflex image xe', xd' is increased compared with that of detection of a line of sight thereby the threshold level Vs is raised too. Therefore, even if disturbance light enters at the state not to be in contact with the eye, it does not exceed the threshold level Vs and a microcomputer for a line of sight slightly is risen carelessly and the reliability of contact optometry by an examiner distant from the eye is improved.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image pickup apparatus having a visual axis detecting means and an eyepiece detecting means.

[0002]

2. Description of the Related Art First, the principle of line-of-sight detection will be described with reference to FIGS.

FIG. 8 is a top view for explaining the principle of visual axis detection, and FIG. 9 is a side view for explaining the principle of visual axis detection.

8 and 9, 906a and 906b are light sources such as a light emitting diode (IRED) that emits infrared light insensitive to an observer, and each light source is an imaging lens 91.
1 is substantially symmetrical in the x direction (horizontal direction) with respect to the optical axis 1 (FIG. 8), and slightly downward in the y direction (vertical direction) (FIG. 9)
It is placed and divergently illuminates the eyes of the observer. Part of the illumination light reflected by the eyeball forms an image on the image sensor 912 by the imaging lens 911. FIG. 7A is a schematic diagram of an eyeball image projected on the image sensor 912, and FIG. 7B is an output intensity diagram of the image sensor 912.

A method of detecting the line of sight will be described below with reference to the drawings.

Considering the horizontal plane first, the infrared light emitted from the light source 906b in FIG. 8 illuminates the cornea 910 of the eyeball 908 of the observer. At this time, the corneal reflection image d (virtual image) formed by the infrared light reflected on the surface of the cornea 910 is condensed by the imaging lens 911, and the image sensor 912.
An image is formed at the upper position d '. Similarly, the infrared light emitted from the light source 906a illuminates the cornea 910 of the eyeball. At this time, the corneal reflection image e (virtual image) formed by the infrared light reflected on the surface of the cornea 910 is condensed by the imaging lens 911 and is imaged at the position e ′ on the image sensor 912. Further, the light flux from the ends a and b of the iris 904 is formed by the imaging lens 91.
The images of the end portions a and b are formed at the positions a ′ and b ′ on the image sensor 912 via the image sensor 1. When the rotation angle θ of the optical axis of the eyeball 908 with respect to the optical axis of the imaging lens 911 is small, and the x coordinates of the ends a and b of the iris 904 are xa and xb,
A large number of points xa and xb can be obtained on the image sensor (mark x in FIG. 7A). Therefore, first, the pupil center xc is calculated by the method of least squares of a circle. On the other hand, when the x coordinate of the center of curvature o of the cornea 910 is xo, the rotation angle θx of the eyeball 908 with respect to the optical axis is oc * sin θx = xc−xo (1). Further, when xo is obtained by considering a predetermined correction value δx at the midpoint k of the corneal reflection images d and e, xk = (xd + xe) / 2 xo = (xd + xe) / 2 + δx (2) where δx is This is a numerical value geometrically determined from the installation method, eyeball distance, etc., and the calculation method is omitted. Therefore, by substituting the equation (2) into the equation (1) to obtain θx, θx = arcsin [[xc− {xd + xe) / 2 + δx}] / oc] (3) Further, each image is projected on the image sensor 912. When the coordinates of the feature points of are rewritten in equation (4) with ´ (dash), θx = arcsin [[xc´ − {(xd´ ＋ xe´) / 2 + δx´} \ / oc / β \ (4) Becomes Here, β is a magnification determined by the distance sze of the eyeball with respect to the imaging lens 911, and is actually obtained as a function of the interval | xd′−xe ′ | of corneal reflection images.

Considering the vertical plane, the structure is as shown in FIG. Here, the corneal reflection image generated by the two IREDs 906a and 906b is generated at the same position, and is referred to as i. The method of calculating the rotation angle θy of the eyeball is almost the same as that for the horizontal plane, but only the equation (2) is different, and the y coordinate of the corneal curvature center o is y.
Let o be: yo = yi + δy (5) Here, δy is a numerical value obtained geometrically from the device arrangement method, eyeball distance, etc., and the calculation method is omitted. Therefore, the rotation angle θy in the vertical direction is θy = arcsin [[yc ′ − (yi ′ + δy ′)] / oc / β] (6).

Further, when the constant m determined by the finder optical system is used as the line-of-sight position coordinates (xn, yn) on the viewfinder screen of the video camera, xn = m * arcsin [[xc'- {(xd '+ xe') / 2 + δx '} \ / oc / β \ (7) yn = m * arcsin [[yc'-{(yi '+ δy') \ / oc / β \ (8).

As is apparent from FIG. 7, the detection of the pupil edge uses the fall (xb ') and rise (xa') of the image sensor output waveform. In addition, the coordinates of the corneal reflection image utilize sharp rising portions (xe ′) and (xd ′).

Next, an example of a video camera having a visual axis detection function and an eyepiece detection function will be described.

FIG. 10A is a schematic diagram showing an example of a video camera having a visual axis detection function and an eyepiece detection function.

The video camera shown in the figure has a zoom lens and a display device having a lens image pickup system 1001 for picking up an image of a subject and an LCD (liquid crystal display) for observing the subject picked up by the lens image pickup system 1001. 1002
And a first eyepiece lens 1003 arranged in front of the display element 1002, a second eyepiece lens 1010 arranged immediately in front of the photographer's eye, and a line-of-sight detecting means for detecting the line of sight of the photographer's eye 1005. 1006, an AF frame showing an outline of the focus area, a display circuit 1007 for displaying information necessary for the photographer such as a tape counter and a photographing mode on the display element 1002, and a system control means 1008 for controlling each part of the camera. , A memory 1009 for storing various data, an operation switch detection means 1004 for executing a function of a video camera, an infrared light emitting diode drive circuit 1012 for driving an infrared light emitting diode described later, and an eye. Reflected light detection circuit 10 for detecting the reflected light of the camera and detecting that the photographer's eye is approaching
11 and 11, and is generally configured.

The line-of-sight detecting means 1006 emits infrared light to the eye 1005 of the photographer.
0, 1065, a dichroic mirror 1061 that reflects visible light and transmits infrared light, and a condenser lens 10 that collects infrared light that has passed through the dichroic mirror 1061.
62, an image sensor (photoelectric conversion element) 1063 for converting the infrared light condensed by the condenser lens 1062 into an electric signal, and an image of the photographer's eye 1005 on the image sensor 1063. Display element 1002
A gazing point detection circuit 1064 for obtaining the gazing point is provided.

Since the dichroic mirror 1061 reflects visible light, the photographer can see the eyepieces 1003, 101.
0, the screen of the display element 1002 can be observed. Also, the dichroic mirror 1061
Since infrared light is transmitted, the infrared light emitting diode 1060,
The reflected image of the eye 1005 illuminated by 1065 passes through the second eyepiece lens 1010, and the condenser lens 1062.
The image is focused on the image sensor 1063 to form an image.

FIG. 10B is a view of the finder seen from the observer side. The infrared light emitting diodes 1060, 10
65 is a different arrangement of 1060 for naked eyes and 1065 for glasses.
Are used properly. The two for the naked eye 1060 are at the same height from the lower portion slightly away from the optical axis, and one is left and right with a narrow width symmetrical to the optical axis. In addition, one on each side is arranged with a wide width symmetrically with respect to the optical axis. There are three reasons. One is to obtain a better illumination condition depending on the eyeball distance, and the position is such that the detection area of the eye is illuminated as evenly as possible. Secondly, the infrared light emitting diode 1060 for the naked eye is at a higher position than the infrared light emitting diode 1065 for spectacles, because it is necessary to make the height so that the corneal reflection image is not vignetted by the eyelids. Thirdly, the infrared light emitting diode 106 for glasses is used in order to allow a ghost, which is formed by the infrared light reflected by the glasses, to appear in the peripheral portion that has little influence on the detection.
Reference numeral 5 is at a position apart left, right, lower than that for the naked eye. It should be noted that the eyeball and the eyeglass are discriminated from each other by the interval | xd'-
It is performed by calculating the distance between the eyeball and the second eyepiece lens 1010 from xe ′ |.

Next, the image sensor 1 for detecting the normal line of sight
The image capture of 063 will be described. FIG. 12 is a timing chart for capturing an image from an image sensor. The upper part shows a read pulse, and the lower part shows an infrared light emitting diode 1060.
The current is 1065.

Regarding the read pulse, first, all the pixels are read out at time t1, and the reset operation is performed to discharge the charge accumulated last time. Next, take the time t2,
Charge is accumulated in the image sensor 1063 by the imaged light. Then, the accumulated charge is read out for a time period of t3. Further, at time t4, the system controller 1008 sets the read charge level of each pixel.
Then, the gazing point is obtained by the above-mentioned algorithm.

On the other hand, the infrared light emitting diodes 1060 and 106
The current of 5 flows for the time of t5 at the same timing as the time of t2 during which the electric charge is accumulated, and illuminates the eye of the photographer 1005.

The gazing point detection circuit 1064 is based on the image of the photographer's eye 1005 on the image sensor 1063, and the above-described principle, Japanese Patent Laid-Open No. 1-241511, and Japanese Patent Laid-Open No. 2-151511.
According to the algorithm disclosed in Japanese Patent No. 32312 or the like, the gazing point on the screen of the display element 1002 of the photographer is obtained.

With the above configuration, a so-called line-of-sight switch input function, etc., in which the line-of-sight autofocus for focusing on the line-of-sight position and the index indicating functions such as zoom and fade are displayed on the finder screen and the line of sight is selected Can be realized.

Next, the eyepiece detecting means will be described. FIG. 11 is a flowchart when eyepiece detection is performed. Here, first, the infrared light emitting diode is turned on, and the image sensor 1063 is stored (S1102). Next, the charges accumulated in the image sensor 1063 are read (S110).
3). The above operation is the same as the image capturing (FIG. 12) of the image sensor 1063 at the time of detecting the normal line of sight described above. The infrared light emitting diode to be emitted at this time may be either the naked eye 1060 or the eyeglasses 1065, and the current flowing through the infrared light emitting diode is the same as that at the time of sight line detection.

Further, the reflected light detection circuit 1011 detects by hardware whether or not the reflected object is in the vicinity of the second eyepiece lens 1010 (S1104), and if there is the reflected object, the process proceeds to the next step and does not exist. In this case, the process returns to the step before S1102. S1
The reflector detection method 104 determines whether the highest level of the image sensor 1063 is equal to or higher than a predetermined threshold level in order to simplify the circuit configuration. FIG. 13A is a corneal reflection image output diagram, which is the output of the image sensor 1063 when the photographer's eyes approach, and the reflection images xe ′, xd ′ on the cornea by the two infrared light emitting diodes.
Has risen. Here, Vs is a predetermined threshold level for the above-described reflection object determination, and when it exceeds this level, it is determined that the eyepiece has been contacted.

The operation from S1102 to S1104 consumes little power because only these two circuits are powered while the reflected light detection circuit 1011 and the infrared light emitting diode drive circuit 1012 are operating. is there.

If there is a reflected object in S1104, only the line-of-sight microcomputer in the system control means 1008 is turned on.
N (S1105), the infrared light emitting diode is turned on again, and the image sensor 1063 is accumulated (S1105).
1106), and the charges accumulated in the image sensor 1063 are read (S1107). Next, in the line-of-sight microcomputer, is there a corneal reflection image (S1108) or a pupil (S1)?
109), and if neither of them is S1,
Return to the front of 102 and turn on the main power when both are present
Then, the entire camera system is started up (S1110).
Here, it is determined whether there is a corneal reflection image (S1108)
Since it is a condition that two pairs are used and the rising edge is steep, the reliability is higher than that of the determination as to whether or not the reflective object is possible (S1104).

With the above configuration, the system can be started up by eyepiece detection with the minimum standby power.

[0026]

However, in the above-mentioned conventional example, when eyepiece detection is performed when the eye of the photographer is far from the eyepiece lens, the reflected light level decreases, and the reflected light detection circuit 1011 cannot detect the eyepiece. And the system is on
It often didn't. In particular, this will be a major obstacle when using a high-eye point finder that allows observation at a distance from the eye.

In addition, strong external light enters from the eyepiece lens,
If the external light level becomes higher than the corneal reflection image, the reflected light detection circuit 1011 can easily determine that there is a reflecting object even if the eyepiece is not in the eyepiece, and the loss of power consumption increases. FIG. 13B is an output diagram of the image sensor 1063 when such disturbance light enters, and the output level exceeds the threshold level Vs due to the disturbance light directly or due to strong light reflected by the object. There is.

Here, the same thing occurs in the case of detecting the visual axis, but the visual axis function cannot be used, and the photographing function of the camera is not significantly affected.

The present invention has been made under such circumstances, and an object of the present invention is to provide an image pickup apparatus capable of more reliably performing eyepiece detection.

[0030]

In order to achieve the above-mentioned object, according to the present invention, an imaging device is configured as described in the following (1) to (7).

(1) An image pickup apparatus comprising a line-of-sight detecting means and an eyepiece detecting means, wherein the illuminance of the illuminating means for the line-of-sight detecting means is larger than that of the eye. apparatus.

(2) An image pickup apparatus comprising a line-of-sight detecting means and an eyepiece detecting means, wherein the current flowing through the eyepiece detecting means illuminating means is larger than the current flowing through the line-of-sight detecting means illuminating means.

(3) An image pickup apparatus comprising a line-of-sight detection means and an eyepiece detection means, wherein the light emission time of the eyepiece detection means illumination means is shorter than the light emission time of the eyeline detection means illumination means.

(4) An image pickup apparatus comprising a line-of-sight detection means and an eyepiece detection means, wherein the accumulation time of the eyepiece detection image sensor is shorter than the accumulation time of the image sensor for the line-of-sight detection means.

(5) An image pickup apparatus comprising a line-of-sight detecting means and an eyepiece detecting means, wherein the light-emission interval time of the eye-gaze detecting means illuminating means is made longer than the light-emission interval time of the eye-gaze detecting means illuminating means. The imaging device according to any one of (1) to (4).

(6) The above-mentioned (1), wherein the illumination means for the line-of-sight detection means and the illumination means for the eyepiece detection means are combined.
The imaging device according to any one of (1) to (5).

(7) The image pickup apparatus as described in (4) above, wherein an image sensor that is also used as a line-of-sight detection image sensor and an eyepiece detection image sensor is used.

[0038]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described in detail below with reference to examples of "video camera". The present invention is not limited to a video camera, and can be similarly implemented in a still video camera, for example. Further, in the embodiment, the infrared light emitting diode and the image sensor which are used for both the sight line detection and the eyepiece detection are used, but the infrared light emitting diode and the image sensor are not limited to this, and the infrared light emitting diode and the image sensor are respectively used for the eyeline detection and the eyepiece detection. It can be implemented using a sensor.

[0039]

【Example】

(Embodiment 1) Embodiment 1 in which the present invention is implemented by a video camera.
Will be described. In addition, the gaze detection principle (FIGS. 7 to 9),
The block diagram of the video camera (FIG. 10A), the arrangement of the infrared light emitting diodes (FIG. 10B), and the flowchart for detecting the eyepiece (FIG. 11) are the same as those in the conventional example, and thus the description thereof is omitted.

FIG. 1A is a corneal reflection image output diagram in the eyepiece state according to this embodiment, FIG. 1B is a disturbance light output diagram, and FIG. 4 is an image sensor incorporation timing chart. In the present embodiment, the current value of the infrared light emitting diode is made larger at the time of eyepiece detection than at the time of sight line detection (FIG. 12).
Therefore, as shown in FIG. 1A, the corneal reflection image output x
The levels of e ′ and xd ′ are also higher than when the line of sight is detected, and the threshold level Vs is accordingly increased.
The timing chart of the image sensor fetching is the same as that of the conventional example except that the current value of the infrared light emitting diode is high as shown in FIG.

With the above configuration, as shown in FIG. 1 (b), the threshold level Vs is not exceeded even if ambient light enters without eye contact, and the line-of-sight microcomputer is inadvertently started up. Is less. Further, the reliability of eyepiece detection of a photographer having a long eyepiece distance is also improved.

(Second Embodiment) A second embodiment in which the present invention is implemented by a video camera will be described. It should be noted that the line-of-sight detection principle (FIGS. 7 to 9) and the video camera configuration block diagram (FIG. 10)
(A)), Infrared light emitting diode layout (Fig. 10)
(B)) and the flowchart (FIG. 11) at the time of detecting the eyepiece are the same as those in the conventional example, and therefore the description thereof will be omitted.

FIG. 2A is a corneal reflection image output diagram in the eyepiece state according to this embodiment, FIG. 2B is a disturbance light output diagram, and FIG. 5 is an image sensor incorporation timing chart. In the present embodiment, as shown in FIG. 5, the accumulation time t2 and the light emission time t5 of the infrared light emitting diode are the time when the line of sight is detected (see FIG.
It is shorter than 2). Therefore, FIG. 2 (a)
As shown in, the overall level of the image sensor output becomes low. However, the levels of the corneal reflection image outputs xe 'and xd' often reach the saturation level from the beginning, and due to the steep rise, the level of decrease is smaller than that of the other parts.

Therefore, by setting the threshold level Vs to the optimum level, as shown in FIG.
Even when ambient light enters without eye contact, the threshold level Vs is not exceeded, and the line-of-sight microcomputer is less likely to be inadvertently started up.

(Third Embodiment) A third embodiment in which the present invention is implemented by a video camera will be described. It should be noted that the line-of-sight detection principle (FIGS. 7 to 9) and the video camera configuration block diagram (FIG. 10)
(A)), Infrared light emitting diode layout (Fig. 10)
(B)) and the flowchart (FIG. 11) at the time of detecting the eyepiece are the same as those in the conventional example, and therefore the description thereof will be omitted.

FIG. 3A is a corneal reflection image output diagram in the eyepiece state according to the present embodiment, FIG. 3B is a disturbance light output diagram, and FIG. 6 is an image sensor incorporation timing chart. In the present embodiment, as shown in FIG. 5, the accumulation time t2 and the light emission time t5 of the infrared light emitting diode are the time when the line of sight is detected (see FIG.
It is shorter than 2) and the current of the infrared light emitting diode is high. In addition, the image reading interval t4 is long. As a result, the levels of the corneal reflection image outputs xe 'and xd' in FIG. 3 (a) are made equal to or higher than those at the time of detecting the line of sight (FIG. 12),
The disturbance level in (b) can be lowered.

Therefore, by setting the threshold level Vs to an optimum level, as shown in FIG.
Even if ambient light enters without eye contact, the threshold level Vs is not exceeded, and the line-of-sight microcomputer rarely starts up, and the threshold level Vs and the corneal reflection image in FIG. The level margin (difference) becomes large, and reliable eyepiece detection becomes possible. Further, the eyepiece detection reliability of a photographer with a long eyepiece distance is also improved.

Further, as the image reading interval t4 becomes longer, the light emitting interval of the infrared light emitting diode also becomes longer. Therefore, even if the current of the infrared light emitting diode becomes high, the photographing per unit time is performed. Since the power of the infrared light applied to the eyes of the person does not increase, the medical influence on the eyes of the photographer can be reduced.

Since the image reading interval t4 is long, the number of detections per unit time at the time of eyepiece detection is smaller than that at the time of detecting the line of sight, but the influence on the operation is less than that at the time of detecting the line of sight. ,No problem.

[0050]

As described above, according to the present invention,
It is possible to provide an imaging device capable of surely performing eyepiece detection. Further, according to the invention of claim 5, it is possible to further provide an imaging device which has less medical influence on human eyes.

[Brief description of drawings]

FIG. 1 is an explanatory diagram of Example 1.

FIG. 2 is an explanatory diagram of a second embodiment.

FIG. 3 is an explanatory view of a third embodiment.

FIG. 4 is a timing chart of the image sensor according to the first embodiment.

FIG. 5 is a timing chart of the image sensor according to the second embodiment.

FIG. 6 is a timing chart of the image sensor according to the third embodiment.

FIG. 7 is an explanatory diagram of the principle of eye gaze detection.

FIG. 8 is a top view for explaining the principle of line-of-sight detection.

FIG. 9 is a side view for explaining the principle of eye gaze detection.

FIG. 10 is a block diagram of a conventional example.

FIG. 11 is a flowchart for eyepiece detection in a conventional example.

FIG. 12 is a timing chart of an image sensor in a conventional example.

FIG. 13 is an explanatory diagram of a conventional example.

[Explanation of symbols]

 xe ', xd' Corneal reflection image output Vs threshold level

Claims (7)

[Claims] 1. An image pickup apparatus comprising a line-of-sight detection unit and an eyepiece detection unit, wherein the illuminance of the eyepiece detection unit illumination unit to the eye is increased with respect to the illuminance of the line-of-sight detection illumination unit to the eye. A characteristic imaging device. 2. An image pickup apparatus comprising a line-of-sight detecting means and an eyepiece detecting means, wherein the current flowing through the illuminating means for the line-of-sight detecting means is made larger than the current flowing through the illuminating means for the line-of-sight detecting means. Image pickup device. 3. An image pickup apparatus comprising a line-of-sight detection means and an eyepiece detection means, wherein the light emission time of the eyepiece detection means illumination means is made shorter than the light emission time of the sightline detection means illumination means. Image pickup device. 4. An image pickup apparatus comprising a line-of-sight detection means and an eyepiece detection means, wherein the accumulation time of the eyepiece detection image sensor is shorter than the accumulation time of the image sensor for the sightline detection means. Imaging device. 5. An imaging apparatus comprising a line-of-sight detecting means and an eyepiece detecting means, wherein the light-emission interval time of the eye-gaze detecting means illuminating means is set longer than the light-emission interval time of the eye-gaze detecting means illuminating means. The imaging device according to any one of claims 1 to 4, which is characterized. 6. The image pickup apparatus according to claim 1, wherein the illumination means for the line-of-sight detection means and the illumination means for the eyepiece detection means are combined. 7. The image pickup apparatus according to claim 4, wherein an image sensor that is used as both the line-of-sight detection image sensor and the eyepiece detection image sensor is used.