JPH1014883A - Visual line detecting device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To accurately detect the edge of a pupil using only the required minimum number of IREDs by installing a control means which moves a visual line detecting unit according to the position of a visual line detecting device, and controlling the visual line detecting unit using the control means in such a way that the unit is free to rotate relative to a finder. SOLUTION: The outer ring 101 of a rolling ball bearing is secured to the front of the finder of a camera, and the ball bearing 102 is held by a holder 104. A holder 105 for holding far infrared emitting diodes (IRED)106 is installed on an inner ring 103, and the IREDs 106 for the naked eye and for glasses are mounted on the front of the IRED holder 105. A brush 107a made of a good conductor is disposed on the back of the IRED holder 105, and the terminal of each IRED 106 is connected to the brush 107a. When the position of the camera is determined, the inner ring 103 of the rolling ball bearing is brought into contact with a brake member 111 and held in place, the IREDs 106 emit light, and an image sensor reads an image of the eyeball.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a gaze detecting device.

[0002]

2. Description of the Related Art Hitherto, various eye-gaze detecting devices have been proposed which detect a gaze by illuminating a user's eyeball looking into a finder field of view mounted on a video camera, a silver halide camera or the like.

First, the principle of gaze detection will be described.
FIG. 23 is a principle diagram (top view) of the visual line detection method, and FIG.
4 is a principle diagram (side view) of the visual line detection method. 23 and 24, reference numerals 906a and 906b denote light sources such as light-emitting diodes that emit infrared light insensitive to the user. Each light source is in the x direction (horizontal direction) with respect to the optical axis of the imaging lens 911. ) And slightly below (FIG. 24) in the y-direction (vertical direction) to diverge and illuminate the observer's eyeball. Part of the illumination light reflected by the eyeball forms an image on the image sensor 912 by the imaging lens 911.

FIG. 22A is a schematic view of an eyeball image formed on the image sensor 912, and FIG. 22B is an output intensity diagram of the image sensor 912. Here, reference numeral 901 denotes a pupil image, reference numeral 902 denotes a Purkinje image (an image formed by light reflected on the corneal surface of the eyeball when illuminated by the IRED), reference numeral 903 denotes the center of the eyeball image, and reference numeral 904 denotes an iris image.

First, the light source 906 is placed on a horizontal plane in FIG.
b is reflected by the cornea 9 of the user's eyeball 908.
Illuminate 10. At this time, a corneal reflection image d (virtual image) formed by infrared light reflected on the surface of the cornea 910 is condensed by the imaging lens 911 and forms an image at a position d ′ on the image sensor 912.

[0006] Similarly, infrared light emitted from the light source 906a illuminates the cornea 910 of the eyeball. At this time, a corneal reflection image e formed by infrared light reflected on the surface of the cornea 910
The (virtual image) is condensed by the imaging lens 911 and forms an image at a position e ′ on the image sensor 912. Iris 90
In the case where the rotation angle θ of the optical axis of the eyeball 908 with respect to the optical axis of the imaging lens 911 is small, the x-coordinates of the single parts a and b of the iris 904 are xa and xb. x
Many points a and xb can be obtained on the image sensor (x in FIG. 22). Therefore, first, the pupil center xc is calculated by the least square method of the circle. On the other hand, the center of curvature o of the cornea 910
Is xo, the rotation angle θx of the eyeball 908 with respect to the optical axis is oc * sin θx = xc−xo (1). Here, o 'is the center of the eyeball. In addition, xo is determined at the midpoint k between the corneal reflection images d and e in consideration of a predetermined correction value δx.
Xk = (xd + xe) / 2 xo = (xd + xe) / 2 + δx (2) Here, δx is a numerical value obtained geometrically from the installation method of the apparatus and the eyeball distance, and the calculation method is omitted. . When (1) is substituted into (2) to obtain θx, θx = arcsin [[xc − ｛(xd + xe) / 2 + δx}] / oc] (3)

Further, similarly to the coordinates of each feature point projected on the image sensor, θx = arcsin [[xc '-{(xd' + xe ') / 2 + δx'}] / oc / β] (4) Becomes Here, β is a magnification determined by the distance sze of the eyeball to the imaging lens 911 and is actually obtained as a function of the interval | xd′−xe ′ | of the corneal reflection image. In addition,
a'b 'is the edge of the pupil formed on the image sensor 912.

When viewed from a vertical plane, the configuration is as shown in FIG. Here, two infrared light emitting diodes (hereinafter, IRED)
And ) The corneal reflection images generated by 906a and 906b are generated at the same position, and this is set as i. Eyeball rotation angle θy
Is almost the same position as in the horizontal plane, but (2)
Only the equation is different, assuming that the y coordinate of o at the center of the corneal curvature is yo, yo = yi + δy (5) where δy is a numerical value obtained geometrically from the arrangement method of the apparatus, the eyeball distance, and the like, and the calculation method is omitted. I do. Therefore, the vertical rotation angle θy is as follows: θy = arcsin [[yc ′ − (yi ′ + δy ′)] / oc / β] (6)

Further, when the position coordinates (xn, yn) on the finder screen of the video image pickup apparatus use a constant m determined by the finder optical system, the position coordinates (xn, yn) on the horizontal plane and the vertical plane are obtained. Xn = m * arcsin [[xc '-{(xd' + xe ') / 2 + δx'}] / oc / β] (7) yn = m * arcsin [[yc '-｛( yi ′ + δy ′)] / oc / β] (8), and the position of the line of sight is determined.

In Japanese Patent Application Laid-Open No. 6-88935, an infrared light emitting diode (hereinafter referred to as IRED) as an illuminating means for detecting a line of sight around a finder is arranged as shown in FIG. In FIG. 25, 12a and 12b are IREDs for the naked eye, and 12c and 12d are for glasses. Four IREDs 12a, 12b, 12c and 12d
Works when the camera is held in the horizontal position.

By the way, when the human eyes blink, the upper eyelids fall down to cover the eyes. Therefore, it is necessary to illuminate the user from obliquely below the eyeball of the user. So 12e for when the camera is held in the vertical position,
A 12f IRED (for the naked eye) is provided.

In order to properly use these IREDs, the attitude of the camera is detected using a mercury switch or the like.
The optimal IRED is selected.

[0013]

FIG. 26A shows an example of an eyeball image projected on an image sensor when a photographer holds the camera sideways. FIG. 26B is an example of an eyeball image projected on the image sensor when the photographer holds the camera vertically.

At this time, the output of the image sensor is read out and output in the direction of the arrow. For this reason, the detection of the pupil edge by the image sensor necessary for detecting the line of sight is performed as shown in FIG.
In (a), the eyelids and eyelashes are not in the way and the edge of the pupil is detected. However, conventionally, when the camera in FIG. 26 (b) is in the vertical position, the eyelids and the eyelashes are in the way. It was difficult to detect the pupil edge accurately.

In addition, in order to be able to detect the line of sight even when the camera is in a horizontal position or a vertical position, a plurality of IREDs must be provided to appropriately illuminate the user's eyeball.

It is an object of the present invention to make it easy to detect a line of sight by accurately detecting a pupil edge regardless of whether the camera is in a horizontal position or a vertical position using only a minimum necessary IRED. And

[0017]

According to the present invention, there is provided a gaze detection unit having illumination means for illuminating a user's eyeball and imaging means for imaging the user's eyeball illuminated by the illumination means. A gaze detection device that detects the gaze of a user looking through the finder by providing the gaze detection unit includes a control unit that moves the gaze detection unit in accordance with the attitude of the gaze detection device. Further, the control means controls the line-of-sight detection unit to be rotatable with respect to the viewfinder.

A gaze detecting unit having an illuminating means for illuminating the user's eye and an imaging means for imaging the user's eye illuminated by the illuminating means is provided so that the gaze of the user looking through the finder can be detected. A gaze detection device for detecting, comprising: a control means for moving the illumination means in accordance with the attitude of the gaze detection device. Further, the image processing apparatus further includes means for displacing the position of the imaging means in accordance with the position of the lighting means. The illumination means is rotatably attached to the finder, and is configured to be rotated by the control means in accordance with the attitude of the visual line detection unit. Further, the control means controls the image pickup means to rotate following the rotation of the illumination means.

A gaze detection unit having illumination means for illuminating the user's eyeball and imaging means for imaging the user's eyeball illuminated by the illumination means is provided, so that the user's gaze through the finder can be detected. A gaze detection device for detecting, comprising: a control unit that moves the imaging unit in accordance with a posture of the gaze detection device. Further, the control means controls the image pickup means to be rotated such that the scanning direction of the image pickup means is always in the horizontal direction of the eyeball image in accordance with the attitude of the visual line detection device. Further, the control means includes means for moving the illumination means in accordance with the attitude of the visual line detection device.

[0020] The illuminating means according to claim 1, 2, or 3 is controlled so as to always be located below the finder. Further, the illuminating means is located below the finder by the weight of the illuminating means. In addition, after the position of the lighting means is determined, a fixing means for fixing the position of the lighting means is provided.

A gaze detection unit having means for illuminating the user's eye and imaging means for taking an image of the user's eye illuminated by the illumination means is provided to detect the gaze of the user looking through the finder. In a gaze detection device, a control unit that moves the imaging unit to a plurality of predetermined positions, and a unit that calculates reliability data of the gaze position of the user detected by the imaging unit at the plurality of predetermined positions, A storage unit for storing the reliability data together with the position information of the imaging unit, and fixing the imaging unit to a position showing the highest reliability value among the reliability data stored together with the position information by the storage unit And fixing means. Also,
The control means controls the illumination means to rotate with respect to the finder.

A gaze detecting unit having means for illuminating the user's eye and imaging means for imaging the user's eye illuminated by the illuminating means detects the user's gaze through the finder. A gaze detecting device, wherein the control means moves the imaging means to a plurality of predetermined positions, and the means calculates reliability data of the gaze position of the user detected by the imaging means at the plurality of predetermined positions. And fixing means for fixing the imaging means when the reliability data exceeds a predetermined value. The control means controls the illumination means to rotate with respect to the finder.

[0023]

Embodiments of the present invention will be described below.

(First Embodiment) The present embodiment is an embodiment in which the sight means of the sight line detection unit for illuminating the user's eyeball is rotated around the finder to accurately detect the sight line. is there.

FIG. 12 is a schematic view of the visual line detection unit. The finder light beam (visible light) passes through a pentaprism 82
The light passes through the dichroic mirror 819 through 0 and reaches the user's eye via the eyepiece.

The reflected light from the eyeball is transmitted to an eyepiece 818,
The light is condensed on the image sensor 817 via a dichroic mirror 819 that transmits visible light and reflects infrared light, a stop 826, and a light receiving lens 827. On the image sensor 817, an eyeball image as shown in FIG. 22 is formed.

FIGS. 1 to 3 are detailed views of a rolling ball bearing portion for supporting a characteristic eyeball illuminating means of the present invention. FIG. 1 is a perspective view and FIG. 2 is a sectional view. FIG. 4 is a schematic configuration diagram of a finder section of a camera embodying the present invention. 10
Reference numeral 1 denotes an outer ring of a rolling ball bearing fixed to the entire viewfinder of the camera, 102 denotes a ball bearing, 103 denotes an inner ring of a rolling ball bearing, and 104 denotes a retainer that holds the ball bearing.

The inner ring 103 of the rolling ball bearing rotates inside the outer ring with a very small force. Reference numeral 111 denotes a wedge-shaped brake member, which is formed of an elastic body such as rubber. Reference numeral 113 denotes an electric contact, and when the brake is pressed against the inner ring of the rolling ball bearing, the electric contact comes into contact and conducts.

A signal is sent from the electric contact to a system control circuit to be described later, and the system control circuit issues a command to display means in the finder according to the signal.

Reference numeral 105 denotes a holder for holding the IRED, which is installed on the inner ring 103 of the rolling ball bearing. 10
Reference numeral 6 denotes an IRED. Two IREDs for the naked eye and two for the glasses are mounted on the front surface of the IRED holder 105.

On the back of the IRED holder 105, brushes 107a made of a good conductor are provided by the number of IRED terminals, and each terminal of the IRED 106 is connected to the brush 107a. Reference numeral 110 denotes an IRED cover formed of a visible light cut filter that transmits infrared light, and covers the entire line of sight detection unit.

FIG. 5 shows an IRED flexible substrate (hereinafter referred to as I).
Called RED flexible. FIG. The IRED flex 108 is connected to an IRED drive circuit (not shown). At the position corresponding to the brush 107a of the IRED flex 108, an exposed copper foil portion 109 on an arc is provided, and the brush is in electrical contact with the exposed copper foil portion on the arc.

The brush 107a is an inner ring 1 of a rolling ball bearing.
When the 03 rotates, the IRED is turned on because it is in contact with the exposed portion of the copper foil 109a on the arc.

The IRED holder 105 shown in FIGS. 1 and 2 has a function as a weight, and FIG. 6 shows the position of the IRED when the camera is in the vertical position.
No matter where the camera is held, as in
The weight of the holder 105 allows the holder to be located below the finder.

After the position of the camera is determined and the IRED holder 105 is positioned below the finder, the IRED holder 105 needs to be fixed in order to prevent the IRED holder 105 from shaking when the camera is moved.

After the position of the IRED holder 105 is determined at the lowermost position, when a rod 114 attached to the brake member 111 is pushed into the back by a system control circuit to be described later, the inner ring 10 of the ball bearing is rolled on the brake member 111.
3 are fixed by contact.

When the position of the IRED holder 105 is determined, the IRED 106 starts emitting light, and the image sensor 817 reads an eyeball image accordingly.

At the same time, the electric contacts 14 come into contact, and a system control circuit described later displays "fixed mode" on the display means in the finder as shown in FIG.

When the rod end 114 is pushed in, the brake member 111 is released and the inner race of the rolling ball bearing becomes rotatable. At the same time, the electric contact 114 is separated, and a system control circuit (not shown) displays "free mode" on a display means in the finder.

The position of the IRED holder 105 can be determined by a motor such as a stepping motor. With such a motor, the IRED holder 10
When the camera 5 is driven, a detection member for detecting the vertical and horizontal postures of the camera is arranged at a position (not shown).

FIG. 3 is a view showing the configuration of a line-of-sight detection unit in which the holder position of the IRED is rotated by a motor.
When the posture of the camera is confirmed by a detection member that detects the posture of the camera (not shown), a drive command for the motor is issued by a system control circuit (not shown), and the rotating member is rotated via the gear 116, and the position is adjusted to an appropriate position. The IRED holder 105 is guided. At this time, the IR
The user may determine the position of the ED holder 105 and drive the ED holder 105 to that position with a motor.

Further, a configuration diagram of a line-of-sight detection unit using an electromagnet as means for fixing the IRED holder 105 is shown. 8 is a perspective view of a flexible member and a rotating member of the IRED, FIG. 9 is a sectional view of the same, and FIG. 10 is an external view.

In FIG. 7, reference numeral 621 denotes a rail which is a semicircular fixing member fixed to the front of the viewfinder of the camera, and is made of a material such as iron. Reference numeral 623 denotes a movable member to which a roller 622 is attached,
It moves along. 624 is an electromagnet built in the movable member. The electromagnet 624 is turned on / off by a system control circuit (not shown). The IRED holder 105 is provided on the movable member 623. The movable member 623 is movable along the rail 621 to all positions except the upper part of the finder window.

In the above configuration, the movable member 623
And the IRED holder 105 attached thereto functions as a weight, so that the IRED holder 5 is always located below the finder along the rails regardless of whether the camera is held in the horizontal position or the vertical position.

After the attitude of the camera is determined, when the button 25 attached to the camera is pressed, a current flows through the electromagnet 624 and is attracted to the rail 621, so that the position of the IRED holder 105 is fixed there.

At the same time, to indicate that the IRED holder 105 is in the fixed mode, "fixed mode", "free mode", etc. are displayed in the viewfinder as shown in FIG. Alternatively, an LED may be turned on in the viewfinder to make the user recognize that the mode is the fixed mode. At this time, when the button 625 is pressed again, the current of the electromagnet is cut, the IRED holder 105 is released from the fixed mode, and moves on the rail.

The system in which the IRED rotates according to the attitude of the camera enables accurate gaze detection using the minimum required IRED.

(Second Embodiment) FIGS. 13 and 14 show a CC for picking up an image of a user's eyeball in a visual line detection unit.
FIG. 13 is a configuration diagram of an embodiment in which imaging means such as D rotates in accordance with the posture of the camera. FIG. 13 shows a state of a line-of-sight detection unit when the camera is in a horizontal position, and FIG. The state of the detection unit is shown. 13 and 14, reference numerals 211 to 218 denote IREDs.
If the user is naked with the camera held sideways,
212 and 213 are 21 if the user is wearing glasses.
1, 214 light up. When the user holds the camera vertically with the left side down, 212 and 216 are turned on when the user is naked, and 211 and 215 are turned on when the user is wearing glasses.

Similarly, when the user stands vertically with the right side down, if the user is naked, 213 and 217 are lit, and if the user is wearing glasses, 214 and 218 are lit.

Reference numeral 202 denotes a dichroic mirror that transmits visible light and reflects infrared light, 203 denotes an image forming lens, and 204 denotes an image sensor such as a CCD. 20
Reference numeral 5 denotes a plate in the shape of a spur gear, which holds an image sensor substrate (not shown). Reference numeral 206 denotes a wire to be transmitted to a main board (not shown), which is sufficiently loosened so as not to be cut even when the image sensor board rotates.

When the camera is in the initial state in which the camera is in the horizontal position, the plate 205 as shown in FIG.
4 is installed so that the long side is perpendicular to the optical axis of the camera, but when the camera is held in the vertical position,
According to a flowchart described later, the image is rotated 90 degrees from the initial state, and the state shown in FIG. 14 is obtained.

Reference numeral 207 denotes a stepping motor having a gear at the tip, and the gear meshes with the plate 205. Reference numeral 208 denotes a posture detection member for detecting the posture of the camera, such as a mercury switch. The output of the detection member is input to a system control unit (not shown), and controls the stepping motor 207 according to the output of the detection member.

Next, the operation of the visual line detection unit of the present embodiment will be described with reference to FIG.

The line-of-sight input switch of the camera (not shown) is turned on.
Then, the attitude of the camera is detected from the output of the attitude detection member 208 such as a mercury switch (S10).
2). After determining whether the orientation of the camera is horizontal or vertical, the orientation of the image sensor 204 is further determined (S103, S103).
104).

If the orientation of the camera and the orientation of the image sensor 204 are different, the stepping motor 207
(S105, S106). Therefore, when the user looks into the finder in this state, an eyeball image is formed on the image sensor 204 so that the edge of the pupil can be easily detected as shown in FIG.
7).

Here, a line-of-sight detection method for detecting the line of sight of the user based on the eyeball image formed on the image sensor 204 is the same as the conventional one, and a description thereof is omitted.

As described above, according to the present embodiment, according to the present embodiment, the image sensor rotates so as to easily detect the edge of the pupil in accordance with the posture of the camera.
As described above, the eyeball image is read, so that it is possible to accurately detect the line of sight.

Further, the IRED as in the first embodiment is used.
It is of course possible to rotate the image sensor in conjunction with the rotation of the holder (the IRED holder depicted in FIGS. 13 and 14 rotates in this case). In this case, the IRED can be minimized, and the image sensor does not need to be controlled by the attitude of the camera, which leads to a reduction in the size of the device.

The flow of the operation in such an embodiment is shown in the flowchart of FIG. 1 against FIG.
6 is that the image sensor is rotated in accordance with the position of the IRED holder in FIG. 16 whereas the operation of FIG. 15 is that the image sensor is rotated according to the attitude of the camera in the flowchart. Only the portion is the same as that of FIG. 15, and the description of the flowchart of FIG. 16 is omitted.

In the flowchart of FIG. 16, the image sensor corresponds to only two positions, a horizontal position and a vertical position. However, it is of course possible to rotate the image sensor corresponding to any position of the IRED holder.

For example, with such a system, accurate gaze detection can be performed even if the camera is held at an angle in order to obtain an effect on photographing.

FIG. 7 is an electrical block diagram of the camera according to the present embodiment.

In FIG. 7, a system control circuit 508 constituted by a CPU instructs a lens system 501 to adjust the focal point, aperture, etc. of the lens.

The camera signal processing 502 exposes the subject image onto an image pickup device such as a CCD, converts the image into an electric signal, and further converts the image into a standardized video signal such as an NTSC signal. , A unit for recording on a recording medium such as a magnetic tape or an optical disk.

Reference numeral 509 denotes a visual axis detection unit which is a feature of the present invention.

Reference numeral 505 denotes a dichroic mirror that transmits visible light and reflects infrared light.

First, when the photographer is looking into the LCD 503, the camera posture detecting sensor 510 detects the posture of the camera. After detecting the posture of the camera, the system control circuit 508 rotates the CCD 504 to an appropriate position, and moves the holder (not shown) supporting the IRED 506 so that it is located below the line-of-sight detection unit, that is, below the viewfinder. After rotating, fix it.

After the above operation is completed, the IRED 506
Is issued, the eyeball image is read by the CCD 504, and the line of sight is detected.

(Third Embodiment) In the present embodiment,
Illumination means such as RED and C for capturing the eyeball image of the user
This is a system in which the entire line-of-sight detection unit that detects the line of sight including imaging means such as a CD rotates according to the attitude of the camera.

FIG. 17 shows the arrangement of the line-of-sight detection unit when the camera is held in the horizontal position, and FIG. 18 shows the arrangement of the line-of-sight detection unit when the camera is held in the vertical position. .

In FIG. 17, reference numeral 305 denotes an outer ring of a rolling ball bearing installed on a camera chassis (not shown);
Is an inner ring of a rotatable rolling ball bearing, which rotates the inside of the outer ring with a very light force.
Is installed. The IRED holder 300 has an I for the naked eye
REDs 302 and 303 and IREDs 301 and 3 for glasses
04 is provided. Reference numeral 307 denotes a dichroic mirror that transmits visible light and reflects infrared light, 308 denotes an imaging lens, and 309 denotes an image sensor such as a CCD, which detects the line of sight including the dichroic mirror 307, the imaging lens 308, and the image sensor 309. The whole unit rotates together. The line-of-sight detection unit and a main board (not shown) are connected by, for example, an electric brush.

In the eye-gaze detecting unit according to the present embodiment, if the IRED holder 300 is designed to be the heaviest,
Since the IRED holder 300 is located below the finder by its own weight, the IRED can always illuminate the user's eyeball image from below. Also, IRED holder 3
00 may be driven by a motor to a position where the eyeball can be easily illuminated.

The illuminated eyeball image of the photographer is captured by the image sensor 309 through the dichroic mirror 307 and the imaging lens 308.

According to the present embodiment, even if the photographer holds the camera at an arbitrary inclination, the relative position between the user's eyeball and the line-of-sight detection unit is maintained, and the IRED as a means for illuminating the user's eyeball. However, it is possible to always illuminate the eyeball from below with the minimum necessary number. In the present embodiment, the illuminating means and the imaging means of the eye-gaze detecting unit are integrally rotated around the finder. However, the IRED holder and the image sensor may be individually driven and rotated by downsizing the apparatus.

(Fourth Embodiment) FIG. 19 is a schematic configuration diagram of a visual line detection unit according to this embodiment.

In FIG. 19, reference numeral 751 denotes a rolling ball bearing, the fixed side of which is fixed to a camera chassis (not shown). A gear is cut on the outer periphery on the movable side, and further, IREDs 731, 732, 733, 73 for glasses and for the naked eye.
4 is provided. Reference numeral 752 denotes a stepping motor having a gear at the tip. When the stepping motor 752 is driven, the IRED holder 730 moves to a predetermined position described later.

FIG. 20 is a flowchart of the operation of the visual line detection unit of the present embodiment. The operation of the visual line detection unit according to the present embodiment will be described with reference to FIG.

The eye-gaze input switch of the camera (not shown) is turned on.
When this is done (S201), the image sensor 4 immediately starts capturing an eyeball image (S202).

At this time, when the user is looking through the finder 701, the pupil center of the eyeball image and the contrast of the eyeball image are detected (S203), and the reliability of the calculated Purkinje image and the position of the pupil center is determined. The determination is made, and the reliability determination data is stored together with the angle information of the image sensor 704 (S204).

Next, it is determined whether or not the image sensor 704 has made one rotation (S205).

If not, the stepping motor 707 is driven to rotate the image sensor 704 by a predetermined angle (S206).

When the rotation is completed, the image sensor 704 is returned again.
Starts capturing, and the angle information of the image sensor 704 and the reliability determination data are stored in the memory. The above operation is continued until the image sensor 704 completes one rotation.

When the image sensor 704 completes one rotation, the angle of the image sensor 704 showing the best value of the reliability judgment data is detected (S207), and the image sensor 704 is rotated to that angle (S208). . At the same time, the posture of the camera is detected by a detection circuit for detecting the posture of the camera, such as a mercury switch (not shown), and the IRED is driven below the finder 701 by the stepping motor 752 in accordance with the detection circuit (S21).
0).

As described above, in the present embodiment, the reliability judgment data is stored for each predetermined angle by using the image sensor 704 rotated by the stepping motor, and the best value of the reliability judgment data is shown. Since the line of sight is detected at the position and the eyeball image is illuminated from below the finder 701 by the IRED, the line of sight can be detected accurately even if the photographing operation is performed in any posture of the camera.

In this embodiment, the best angle of the reliability data is detected and the stepping motor is driven to rotate the image pickup device. However, in order to shorten the line-of-sight detection time, the reliability determination data is set to a predetermined value. If the value is equal to or more than the value, the rotation may be stopped at that point to detect the line of sight.

FIG. 21 shows a flowchart of such an embodiment. First, when a line-of-sight input switch of a camera (not shown) is turned on (S301), the image sensor 7 is immediately
04 starts capturing the eyeball image (S302). At this time, if the user is looking through the viewfinder 701, the center of the pupil of the eyeball image and the contrast of the eyeball image are detected (S303).

The reliability of the calculated Purkinje image and the center position of the pupil is determined (S304).
It is determined whether the reliability determination data is equal to or more than a predetermined value (S
305) When it is determined that the value is equal to or more than the predetermined value, the rotation of the image sensor is stopped (S307), and the IRED holder 73 is stopped.
0 is positioned below the viewfinder, and the flow shifts to the visual line detection operation (S308).

If the reliability determination data does not reach the predetermined value in S305, the image sensor is rotated by a predetermined angle (S306), and this operation is repeated until the reliability determination data reaches the predetermined value.

According to the above-described embodiment, the speed of gaze detection is increased.

[0090]

As described above, according to the first and second aspects of the present invention, even if the user holds the apparatus in an arbitrary posture, the entire line-of-sight detection unit can be moved, so that the user's eyeball can be moved. Since the relative position of the line-of-sight detection unit is maintained, the edge of the pupil is easily detected, and since the eyeball moves to a position where it is easy to illuminate, the line-of-sight can be easily detected by using the minimum necessary illumination means. .

According to the third aspect of the present invention, the illuminating means for illuminating the user's eyeball is movable in accordance with the attitude of the apparatus, so that the apparatus can be held in any attitude using only the minimum necessary illuminating means. The gaze can be detected even if it is performed.

According to the fourth to sixth aspects, the imaging means is movable in accordance with the position of the illuminating means for illuminating the user's eyeball, so that the same effect as in the first aspect is obtained. This leads to downsizing of the device.

According to the seventh to ninth aspects, the imaging means for imaging the user's eyeball is made movable in accordance with the attitude of the apparatus, so that the edge of the pupil can be held regardless of the attitude of the apparatus. And the detection of the line of sight becomes accurate.

According to the tenth aspect, since the illuminating means for illuminating the user's eyeball is always located below the finder, the user's eyelids and eyelashes can illuminate the eyeball without obstruction. .

According to the eleventh aspect, by using the weight itself of the holder in which the illuminating means for illuminating the user's eyeball is installed, the line of sight can be detected without any equipment for detecting the attitude of the apparatus. It is also possible to do.

According to the twelfth aspect, after the attitude of the device is determined and the illuminating means is positioned at the lowermost portion of the finder, the position is fixed, so that accurate gaze detection can be performed even if the device is shaken. Becomes possible.

According to the thirteenth and fourteenth aspects, the reliability data of the eyeball image is stored for each position using an image sensor movable to a plurality of predetermined positions. Since the line of sight is detected at the position, it is possible to detect the line of sight accurately.

According to the fifteenth and sixteenth aspects, while the image sensor is moving, when the reliability data of a predetermined value or more is obtained, the movement of the image sensor is stopped and the line of sight is set at that position. Is detected, quick and accurate gaze detection can be performed.

[Brief description of the drawings]

FIG. 1 is a perspective view of a rolling ball bearing according to a first embodiment.

FIG. 2 is a sectional view of a rolling ball bearing according to the first embodiment.

FIG. 3 is a perspective view of a rolling ball bearing by a motor according to the first embodiment;

FIG. 4 is a schematic configuration diagram of a finder section of the camera according to the first embodiment.

FIG. 5 is a view showing a flexible substrate of the IRED according to the first embodiment;

FIG. 6 is a diagram showing the position of the IRED when the camera according to the first embodiment is held vertically.

FIG. 7 is a block diagram of an apparatus according to a second embodiment of the present invention.

FIG. 8 is a perspective view of the flexible substrate and the rotating member of the IRED in the embodiment of the fixing means using the electromagnet of the present invention.

FIG. 9 is a cross-sectional view of the flexible substrate and the rotating member of the IRED in the embodiment of the fixing means using the electromagnet of the present invention.

FIG. 10 is an external view of a flexible substrate and a rotating member of the IRED in the embodiment of the fixing means using the electromagnet of the present invention.

FIG. 11 is a diagram showing a display example in the viewfinder when the IRED holder is fixed and released.

FIG. 12 is a schematic diagram of a visual line detection unit.

FIG. 13 is a schematic diagram of a line-of-sight detection unit when the camera according to the second embodiment is in a horizontal position.

FIG. 14 is a schematic diagram of a gaze detection unit when the camera according to the second embodiment is set in a vertical position.

FIG. 15 is a flowchart according to the second embodiment;

FIG. 16 is a flowchart when an IRED holder movable by its own weight is used in the second embodiment.

FIG. 17 is a schematic diagram of a line-of-sight detection unit when the camera according to the third embodiment is placed in a horizontal position.

FIG. 18 is a schematic diagram of a gaze detection unit when the camera according to the third embodiment is set in a vertical position.

FIG. 19 is a schematic diagram illustrating a configuration of a finder section of a camera according to a fourth embodiment.

FIG. 20 is a flowchart for determining the optimum reliability data by making the image sensor make one rotation in the fourth embodiment.

FIG. 21 is a flowchart for determining a position at which reliability data equal to or more than a predetermined value is obtained in the fourth embodiment.

22A is a diagram illustrating an eyeball image projected on an image sensor, and FIG. 22B is a diagram illustrating a signal level of a reflected image output from the image sensor.

FIG. 23 is a diagram illustrating the principle of the visual line detection method (top view).

FIG. 24 is a diagram illustrating the principle of a gaze detection method (side view).

FIG. 25 is a conventional gaze detection camera.

26A is an eyeball image on the image sensor when the conventional eye-gaze detection camera is held in a horizontal position, and FIG. 26B is an eyeball image on the image sensor when the conventional eye-gaze detection camera is held in a vertical position. Figure showing each

[Explanation of symbols]

 105 IRED holder 106 IRED 111 Brake member 202 Dichroic mirror 207 Stepping motor 309 Image sensor 508 System control circuit 509 Eye gaze detection unit 510 Camera eye gaze detection sensor 911 Eyepiece

Claims (16)

[Claims] 1. A sight line detecting unit having an illuminating means for illuminating an eyeball of a user and an imaging means for imaging an eyeball of the user illuminated by the illuminating means. A line-of-sight detection device for detecting a line of sight, comprising: control means for moving the line-of-sight detection unit according to the attitude of the line-of-sight detection device. 2. An imaging apparatus according to claim 1, wherein said control means controls said line-of-sight detection unit to be rotatable with respect to said finder. 3. A sight line detecting unit having an illuminating means for illuminating a user's eyeball and an imaging means for imaging an eyeball of the user illuminated by the illuminating means. A line-of-sight detection device for detecting a line-of-sight, comprising: a control unit that moves the illumination unit according to the attitude of the line-of-sight detection device. 4. The eye-gaze detecting device according to claim 3, further comprising: means for displacing the position of the imaging means in accordance with the position of the lighting means. 5. The illuminating device according to claim 3, wherein the illuminating device is rotatably attached to the finder, and is configured to be rotated by the control device in accordance with a posture of the sight line detecting unit. An eye gaze detecting device; 6. The eye-gaze detecting device according to claim 5, wherein the control unit controls the image pickup unit to rotate following the rotation of the illumination unit. 7. A sight line detection unit having an illuminating means for illuminating a user's eyeball and an imaging means for imaging an eyeball of the user illuminated by the illuminating means, thereby enabling a user to look into a finder. A line-of-sight detection device for detecting a line-of-sight, comprising: a control unit that moves the imaging unit in accordance with the attitude of the line-of-sight detection device. 8. The control device according to claim 7, wherein the control unit is configured to rotate the imaging unit such that a scanning direction of the imaging unit is always a horizontal direction of an eyeball image in accordance with a posture of the visual line detection device. An eye-gaze detecting device characterized by controlling. 9. The gaze detecting device according to claim 7, wherein said control means includes means for moving said illumination means in accordance with a posture of said gaze detecting device. 10. The visual line detection device according to claim 1, wherein the control means controls the illumination means so as to be always positioned below a finder. 11. The visual axis detection device according to claim 10, wherein the illuminating means is located below the finder by the weight of the illuminating means. 12. The gaze detection device according to claim 10, further comprising a fixing unit that fixes the position of the illumination unit after the position of the illumination unit is determined. 13. A user's line of sight looking through a finder by including a line of sight detection unit having means for illuminating the user's eyeball and imaging means for imaging the user's eyeball illuminated by the illumination means. A control unit that moves the imaging unit to a plurality of predetermined positions; and a unit that calculates reliability data of the user's gaze position detected by the imaging unit at the plurality of predetermined positions. Storage means for storing the reliability data together with the position information of the imaging means; and the imaging means at a position showing the highest reliability value among the reliability data stored together with the position information by the storage means. A gaze detecting device, comprising: fixing means for fixing the line of sight. 14. An eye gaze detecting apparatus according to claim 13, wherein said control means controls said finder to rotate said illuminating means. 15. A user's line of sight looking through a finder by including a line-of-sight detection unit having means for illuminating the user's eyeball and imaging means for imaging the user's eyeball illuminated by the illumination means. A control unit that moves the imaging unit to a plurality of predetermined positions; and calculates reliability data of the user's gaze position detected by the imaging unit at the plurality of predetermined positions. An eye-gaze detecting device, comprising: means for fixing the imaging means when the reliability data exceeds a predetermined value. 16. The eye-gaze detecting device according to claim 15, wherein the control means controls the lighting means to rotate with respect to the finder.