JP3483366B2 - Camera having line-of-sight detection means - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a camera having line-of-sight detection means, and more particularly to the relationship between focus detection operation and line-of-sight detection operation during continuous shooting.

[0002]

2. Description of the Related Art Conventionally, various devices (for example, eye cameras) that detect what position on a viewing surface a photographer is observing, that is, a so-called line of sight (visual axis) have been provided. For example, in JP-A-1-274736,
Projects a parallel light flux from the light source to the anterior segment of the photographer's eye,
The gazing point is obtained by using the corneal reflection image by the reflected light from the cornea and the imaging position of the pupil. Further, this publication discloses an example in which a gazing point detection device is provided in a single-lens reflex camera and automatic focus adjustment of a photographing lens is performed using gazing point information of a photographer.

FIG. 8 is a schematic view of a line-of-sight detection optical system provided in a single-lens reflex camera. In the figure, 1 is a taking lens, 2 is a main mirror, 7 is a focusing plate, and 8 is a pentaprism. Reference numeral 11 is an eyepiece lens, and 15 is a photographer's eyeball. The photographer brings his eyes close to the eyepiece lens 11 to observe a subject in the finder. Reference numerals 13a and 13b denote light sources such as light emitting diodes that emit infrared light that is insensitive to the photographer. A part of the illumination light reflected by the eyeball of the photographer is focused on the image sensor 14 by the light receiving lens 12.

FIG. 9A is a schematic view of an eyeball image projected on the image sensor 14. In the figure, 52a and 52b are corneal reflection images of the infrared light emitting diodes 13a and 13b.
Is the white part of the eyeball, 51 is the pupil, and 53 is the part of the skin around the eye. 9B shows the intensity of the output signal from a certain line (cross section (E)-(E ')) of the image sensor 14 of FIG. The characteristic of the eyeball image is that the brightness of the corneal reflection of the light emitting diode is the highest, but the area is not so large. (52a ', 52b') Furthermore, since the pupil portion has a very low reflectance, the luminance level is the lowest and occupies a certain area. (51 ') White eye part (50')
Is between the corneal reflection and the brightness level of the pupil part, and the skin part has high or low brightness depending on the external light and lighting conditions.

FIG. 10 is an explanatory view of the principle of line-of-sight detection. In the figure, 15 is the photographer's eyeball, 16 is the cornea, and 17 is the iris.

A method of detecting the line of sight will be described below with reference to the drawings.

The infrared light emitted from the light source 13b illuminates the cornea 16 of the eyeball 15 of the observer. At this time, the corneal reflection image d (virtual image) formed by a part of the infrared light reflected on the surface of the cornea 16 is condensed by the light receiving lens 12 and imaged at the position d ′ on the image sensor 14. Similarly light source 1
The infrared light emitted by 3a illuminates the cornea 16 of the eyeball 15. At this time, the corneal reflection image e formed by a part of the infrared light reflected on the surface of the cornea 16 is the light receiving lens 12
The light is condensed by the image sensor and forms an image at a position e ′ on the image sensor 14.

Light fluxes from the ends a and b of the iris 17 form images of the ends a and b at the positions a ′ and b ′ on the image sensor 14 via the light receiving lens 12. When the rotation angle θ of the optical axis of the eyeball 15 with respect to the optical axis of the light receiving lens 12 is small, the x coordinates of the ends a and b of the iris 17 are xa and xb.
Then, the coordinate xc of the center position c of the pupil 19 is expressed as xc≈ (xa + xb) / 2.

Further, the x-coordinate of the midpoint of the corneal reflection images d and e and the x-coordinate xo of the center of curvature o of the cornea 16 substantially coincide with each other. Therefore, the x-coordinates of the corneal reflection image generation positions d and e are set to x
Let d, xe, and the standard distance between the center of curvature o of the cornea 16 and the center c of the pupil 19 be OC, the optical axis 15 of the eyeball 15
The rotation angle θx of a substantially satisfies the relational expression of OC × SINθx≈ (xd + xe) / 2−xc (1). Therefore, as shown in FIG. 9A, the rotation of the optical axis 15a of the eyeball 15 is detected by detecting the position of each characteristic point of the eyeball 15 (corneal reflection image and the center of the pupil) projected on the image sensor 14. The angle θ can be obtained.

From the equation (1), the rotation angle of the optical axis 15a of the eyeball 15 is β × OC × SINθx≈ {(xpo-δx) -xic} × pitch (2) β × OC × SINθy≈ {(ypo-δy ) −yic} × pitch (3) Here, θx is the rotation angle of the eyeball optical axis in the zx plane, and θy is the rotation angle of the eyeball optical axis in the yz plane. (Xpo, ypo) is the coordinates of the midpoint of the two corneal reflection images on the image sensor 14, (xic, yic)
Are coordinates of the center of the pupil on the image sensor 14. p
pitch is the pixel pitch of the image sensor 14.
Further, β is an imaging magnification determined by the position of the eyeball 15 with respect to the light receiving lens 12, and is substantially obtained as a function of the interval between two corneal reflection images.

Δx and δy are correction terms for correcting the coordinates of the midpoint of the corneal reflection image, and a correction term for correcting an error caused by illuminating the eyeball of the photographer with divergent light instead of parallel light and , Δy also includes a correction term for correcting an offset component generated by illuminating the eyeball of the photographer with divergent light from the lower eyelid.

Rotation angle (θx, θy) of the optical axis of the eyeball of the photographer
Is calculated, the gazing point (x, y) on the observation surface (focus plate) of the photographer is x = m * (θx + Δ) (4a) y = m * θy when the camera is in the horizontal position. (5a) is required. Here, the x direction indicates the horizontal direction with respect to the photographer when the camera is in the horizontal position, and the y direction indicates the vertical direction with respect to the photographer when the camera is in the horizontal position. m is a conversion coefficient for converting the rotation angle of the eyeball into coordinates on the focus plate, and Δ is an angle formed by the eyeball optical axis 15a and the visual axis (gazing point). In general, it is known that the rotation angle of the eyeball deviates from the visual axis at which the observer is actually looking at about 5 ° in the horizontal direction and hardly deviates in the vertical direction with respect to the observer. Here, the operation of the focus detecting means conventionally provided in the camera will be described. This focus detection means has a plurality of focus detection operation modes, and generally, a focus detection operation mode for a stationary subject and a focus detection operation mode for a dynamic subject are set.

The focus detection operation mode for a stationary subject is
Once the in-focus state is detected, the focus detection operation is not performed thereafter.

On the other hand, the focus detection operation mode for a dynamic subject is an operation mode in which the focus detection operation continues after the focus state is detected.

The operation of the part relating to the line-of-sight detection and the focus detection of the camera having the line-of-sight detection means will be described below for each focus detection operation mode.

1) Focus Detection Operation Mode for Static Subject First, when the release button is half pressed (first stroke is pressed), the line-of-sight detection means operates prior to focus detection, and within the viewfinder field of the photographer. Seek attention. This point is represented by coordinates within the viewfinder field.

From the coordinates in the field of view of the finder, the focus detection area corresponding to this is determined. With respect to the focus detection area thus obtained, the focus state is detected by the focus detection means, and the photographing lens is driven to the in-focus state based on the information.

As described above, after the focus detection area is determined by the line-of-sight detecting means, the focusing operation is performed only once based on the focus state of the focus detection area.

2) Focus detection operation mode for dynamic subject When the release button is half pressed (first stroke is pressed), the line-of-sight detection means is operated to determine the focus detection area. After that, the lens drive is continued so as to maintain the in-focus state based only on the focus information of the focus detection area.

As another form of the focus detection operation mode for the dynamic subject, as the focus detection is linked with the movement of the subject and the position of the line of sight of the photographer, the focus detection is always performed before the focus detection. It is also possible to perform the focus detection after determining the focus detection area by detecting the line of sight.

[0021]

However, as in the above-mentioned conventional example, the line-of-sight detection is performed only in the first frame in the dynamic object orientation mode, and the focus detection area is determined from the result to perform focus adjustment. Then, even if the subject moves during continuous shooting and the gazing point of the photographer changes, the focus detection area does not change accordingly.

On the other hand, during continuous shooting, if the line-of-sight is detected for each shooting frame and the focus detection is performed based on the result, the shooting time for one frame is increased by the time for detecting the line-of-sight, as compared with the conventional example. Not only will the frame speed decrease, but the shooting interval during continuous shooting will not be a constant value, so focus detection that calculates the focus position of the subject in the next shooting frame in advance (dynamic prediction AF) There is a drawback that it is not suitable for.

Further, each time, based on the line-of-sight detection result,
With the configuration that determines the focus detection area, the user cannot keep an eye on the subject during continuous shooting, pay careful attention to framing every corner of the frame, and view the shooting information displayed in the viewfinder. You can't do that either. In view of such a problem, an object of the present invention is to improve the followability of focus detection in the dynamic subject orientation mode of the focus detection means, and to have a line-of-sight detection means that does not impair the consistency between the subject and the focus detection area. To provide a camera.

[0024]

[0025]

According to the invention described in claim 1 of the present application, the focus detection is performed based on the visual line detecting means for detecting the visual line position of the user looking into the finder and the detection result of the visual line detecting means. A camera having a line-of-sight detection means for performing focus detection means, a determination means for determining whether or not the focus detection result by the focus detection means is appropriate, and a camera control means for repeating the photographing operation while the release button is pressed. In the above, the focus detecting means repeats the focus detecting operation at a predetermined time interval to predict the position of the subject after a predetermined time has passed from the past focus detecting results, and the judging means determines the present focus detecting result and the past. The focus detection results are compared to determine if the subject position is continuous, and if the subject position is not continuous, it is determined to be inappropriate. When it is determined that the focus detection cannot be performed when it is determined that the detection result by the focus detection unit is not appropriate by performing the line-of-sight detection operation again, the line-of-sight detection unit executes the line-of-sight detection unit. Therefore, the followability of the focus detection means can be improved without reducing the frame speed.

[0026]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS FIG. 4 is a block diagram of a main part of an electric circuit built in a camera body of the present invention. A central processing unit (hereinafter referred to as CPU) 100 of a microcomputer, which is a camera control means built in the camera body, includes a visual axis detection circuit 101, a photometric circuit 102, an automatic focus detection circuit 103, a signal input circuit 104, an LCD. Drive circuit 1
05, LED drive circuit 106, IRED drive circuit 10
7, shutter control circuit 108, motor control circuit 10
9 is connected. Further, signals are transmitted to the lens drive circuit 110 and the diaphragm drive circuit 111 arranged in the photographing lens through the mount contact 37.

The EEPROM 100a as a storage means attached to the CPU 100 can store film information such as a film counter.

The visual axis detection circuit 101 outputs the eyeball image output from the image sensor 14 (CCD-EYE) to the CPU 1
To 00. The CPU 100 converts the eyeball image signal from the image sensor 14 to A by the A / D conversion means inside the CPU.
The D / D conversion is performed, the image information is used to extract each feature point of the eyeball image necessary for detecting the line of sight according to a predetermined algorithm, and the rotation angle of the photographer's eyeball is calculated from the position of each feature point. The photometric circuit 102 amplifies the output from the photometric sensor 10, logarithmically compresses it, and sends it to the CPU 100 as luminance information of each sensor. The photometric sensor 10 is divided into a plurality of surfaces on both sides, and is composed of a plurality of photodiodes that each output a photoelectric conversion output.

The line sensor 6f is composed of five sets of line sensor CCD-L2, CC corresponding to five distance measuring points on the screen.
It is a photoelectric conversion element array such as a known CCD line sensor composed of D-L1, CCD-C, CCD-R1, and CCD-R2. The automatic focus detection circuit 103 sends the voltage obtained from these line sensors 6f to the CPU 100, and CP
In U, the line sensor signal is sequentially A / D converted by the built-in A / D converter.

SW-1 is a photometric switch that is turned on by the first stroke of the release button to start photometry, AF, and line-of-sight detection operations, and SW-2 is a release switch that is turned on by the second stroke of the release button. SW-DIAL1 and SW-
DIAL 2 is an external operation dial switch of the camera, and the amount of rotation click input from the dial to the up / down counter of the signal input circuit is counted.

The LCD drive circuit is the LCD in the finder.
24, the monitor LCD 42 is driven. On the LCD 24 and the monitor LCD 42 in the finder, the shooting information of the camera and the line-of-sight input mark indicating that the line-of-sight detection operation is being performed are displayed.

The LED drive circuit includes an LED 21 (LED-L) for superimposing the focus detection area.
2, LED-L1, LED-C, LED-R1, LED
-R2) and the drive control of the F-LED 25 that illuminates as a backlight of the LCD in the finder.

Upon receiving a drive signal from the CPU 100, the lens drive circuit 110 activates the drive motor 33 to drive the photographing optical system. Driving amount is pulse plate 36, photo coupler 3
5, the driving amount is instructed by the CPU 100 and the driving is stopped. For this purpose CPU1
00 does not need to monitor the lens drive amount and transmit a stop signal after transmitting the drive amount to the lens drive circuit 110. The CPU 110 can communicate with the lens driving circuit 110 and receive data such as the driving state of the photographing optical system.

The IRED drive circuit 107 is an IRED-1
~ Performs lighting control of IRED-6. IRED-1 ~
The most suitable IRED-6 is selected depending on the posture of the camera and the distance between the eyeball of the photographer and the eyepiece. Two of the selected IRED-1 to IRED-6 (corresponding to the two selected IREDs 13a and 13b) illuminate the photographer's eyeball, and the reflected image is received by the CCD-EYE to detect the line of sight. I do.

The line-of-sight detection means, which is a component of the present invention,
Eye-gaze detection circuit 101, IRED drive circuit 107, CCD
EYE 14 and CPU 100.

The focus detecting means is the automatic focus detecting circuit 10.
3, the line sensor 6f and the CPU 100.

The selecting means is composed of the CPU 100.

The lens driving means is the lens driving circuit 11
0, a drive motor 33, and a drive amount includes a pulse plate 36 and a photocoupler 35.

The focus adjusting means is the lens drive circuit 110,
The drive motor 33 and the drive amount are composed of a pulse plate 36, a photocoupler 35, and a CPU 100.

The focus detection determination means, the storage means, the drive amount calculation means, and the continuity determination means are composed of the CPU 100.

Next, a flow chart of the operation of the camera having the visual axis detecting device is shown in FIG. 6 and will be described below based on these.

Next, the operation of the camera will be described with reference to the flowchart of FIG.

First, FIG. 6A will be described.

When power supply to the camera is started, execution is started from # 601.

In # 602, variables and flags are initialized.

The "gaze detection prohibition flag" is cleared.

The "AF prohibition flag" is cleared.

[Release & Feed Interrupt] is prohibited.

At # 604, the first release button is pressed.
The state of the switch SW1 which is turned on by pressing the stroke is detected. If it is off, the process proceeds to # 605, and if it is on, #
Proceed to 612.

# 605 Initialize variables and flags. Here, initialization is performed when SW1 is pushed once to perform a predetermined operation and then SW1 is released again.

Clear the "line-of-sight detection prohibition flag" to detect the line-of-sight, clear the "AF prohibition flag",
Perform AF. In addition, "Release & Feed Interrupt" is prohibited.

# 606 It is checked whether the current state of the camera is the feeding mode setting state. If the feeding mode is set, go to step # 608, otherwise, go to # 61
Go to 4.

In step # 608, it is checked whether the dial has changed. If the dial has not changed, the process proceeds to # 620 without switching the feeding mode, and if the dial has changed, the process proceeds to # 610.

At step # 610, the feeding mode is changed every time the dial is changed. The feeding modes include a "single-shot mode" in which only one image is shot when the release switch SW2 is pressed, and a "continuous-shot mode" in which shooting is continuously performed while the SW2 is held, and the dial changes. The "single shot mode" and "continuous shot mode" are switched each time. When the setting of the feeding mode is completed, the process proceeds to # 602.

# 614, here it is determined whether or not the camera is in the AF mode switching state. If the AF mode is switched, the process proceeds to step # 616. If not, the process proceeds to step # 618.

In # 616, the AF mode is switched. A
As described above, the F mode includes the "focus detection mode for static subjects" and the "focus detection mode for dynamic subjects". If the "focus detection mode for static subjects" is "focus for dynamic subjects". If the "detection mode" is the "focus detection mode for the dynamic subject", then it is switched to the "focus detection mode for the static subject". When the AF mode setting is completed, # 62
Go to 0.

In # 618, settings other than the feeding mode and the AF mode are set. Detailed description is omitted because it is not directly related to the present invention.

In step # 612, the [AE control] subroutine for photometry and camera status display is executed. In the subroutine [AE control], the photometric sensor is operated and the AE control value is calculated by a known algorithm for the photometric value. The calculated AE control value is displayed on the external liquid crystal of the camera. Then, the process proceeds to # 613.

Step # 613 is a subroutine of [visual axis detection and focus detection].

In step # 620, when the series of operations is completed, the process returns to step # 640, and the camera operation is repeated.

While SW1 is being pressed # 604 → #
The [AE control] and [visual axis detection and focus detection] subroutines are repeatedly executed in the order of 612 → # 613 → # 620 → # 604.

Next, the release operation of this embodiment will be described with reference to FIG. 6 (b). The release operation is executed by an interrupt routine.

# 686 When the release button is pushed to the second stroke and SW2 is turned on while interrupt is permitted, the "release &feed" subroutine is called by interrupt processing.

# 688 The interruption of "release feeding" is prohibited.

# 692 Here, the aperture control value and the shutter speed are calculated. The aperture and shutter speed are calculated by a predetermined algorithm according to the photographing mode of the camera, the photometric value or the set value.

# 693 The main mirror 2 and the sub-mirror of the camera are raised, and the diaphragm drive circuit 111 is communicated with to control the diaphragm 31 provided in the lens to the value calculated in # 692.

# 694 Next, the shutter magnet MG
-1 is energized to drive the front curtain of the shutter to start exposure. After the lapse of the predetermined time calculated in # 692, the shutter magnet MG-2 is energized to drive the rear curtain of the shutter to end the exposure.

# 695 The mirror is moved down to a predetermined position, the diaphragm drive circuit 111 is communicated with, and the diaphragm is opened again.

# 696 One frame of film is fed,
Charge the shutter springs.

# 697 The "release release" interrupt subroutine is terminated. The "release feeding" subroutine does not return to the program where the interrupt occurred, but returns to # 604 in FIG. 6 (a).

Next, the operation of the "line-of-sight detection &AF" subroutine of # 613 will be described with reference to the flowchart of FIG.

When "visual axis detection &AF" is called, # 6
The subroutine is executed from 31.

# 631 It is determined here whether line-of-sight detection is permitted. Prohibition of line-of-sight detection can be determined by the “line-of-sight detection prohibition flag”. At first, since it is permitted, the process proceeds to # 632. If the line-of-sight detection is prohibited, the line-of-sight detection operation is not performed and the process proceeds to # 636.

# 632 A line-of-sight input mark is displayed to indicate that the line-of-sight input is being performed prior to the line-of-sight detection. The lighting LED (F-LED) 25 is turned on,
LCD 2 in viewfinder via LCD drive circuit 105
The line-of-sight input mark 4 is turned on. This allows the photographer to confirm that the camera is performing line-of-sight detection outside the viewfinder screen 207.

# 633 Line-of-sight detection is performed according to a predetermined algorithm. First, the IRED drive circuit 107 turns on the IRED that is predetermined according to the position of the camera (the optimum two or four of IRED to IRED6).
The line-of-sight detection circuit 101 starts the accumulation operation of the CCD-EYE 14. When the storage is completed, CP the storage electrification
U100 is sequentially read, A / D conversion is performed, and processing is performed by a predetermined algorithm. All pixels of the CCD-EYE 14 are processed to obtain the coordinates of the corneal reflection images 52b and 52a of the eyeball illumination light source and the center coordinates of the pupil 51 shown in FIGS. 9A and 9B. The coordinates gazed at by the photographer are obtained by calculating these according to the algorithm described above.

# 634 Once the visual axis detection is performed, the visual axis detection is prohibited so that the visual axis detection is not repeated. Prohibition of line-of-sight detection is performed by setting a “line-of-sight detection prohibition flag”.

# 635 A focus detection area is selected from the gazing point coordinates obtained by the sight line detection. Here, the gazing point coordinates and the focus detection area will be described with reference to FIG. In Figure 2, L2, L
1, C, R1 and R2 are C of the line sensor 6f, respectively.
CD-L2, CCD-L1, CCD-C, CCD-R
1) Corresponding to CCD-R2, focus detection is performed at each place. x and y are coordinate axes when the gazing point on the finder is obtained, and the obtained gazing point coordinates are (xn, y
n).

If the gazing point coordinates are -x3 <xn≤-x2 and y1≤yn≤-yn, then L2-x2 <xn≤-x1 and if y1≤yn≤-yn, then L1-x1 <xn <x1 and y1≤. If yn≤-yn C x1≤xn <x2 and y1≤yn≤-yn If x1≤xn <x2 and y1≤yn≤-yn x2≤xn <x3 and if y1≤yn≤-yn Focus of R2 Select the detection area. When the gazing point is at the position shown in FIG. 2, x1 ≦ xn <x2 and y1 ≦ yn ≦ −yn, so the focus detection area is R1.

Next, in order to inform the photographer of the focus detection area selected by the line of sight, a signal is transmitted to the LED drive circuit 106 and the superimposing LED-21 is used to blink the focus detection area (light up). But good).

# 636 Here, it is determined whether or not the lens is being driven by communicating with the lens driving circuit 110. When the "line-of-sight detection &AF" subroutine is executed last time and the lens drive is performed, and the lens drive is not yet completed, the process proceeds to # 680 and returns. Before the lens is driven or after the lens is driven, the process proceeds to # 637.

# 637 Here, it is determined whether the AF mode is the "static subject orientation mode" or the "dynamic subject orientation mode". If it is "static subject mode", proceed to # 640,
If it is the "dynamic subject-oriented mode," proceed to # 660.

# 640 Whether or not focus detection is permitted is determined by the "AF prohibition flag". When the focus is once obtained and the SW1 is still held, the "AF prohibition flag" is set. In this case, therefore, the process proceeds to # 680 and returns.

# 642 Focus detection is performed. The focus detection of the focus detection area selected by the line-of-sight detection is performed from the plurality of focus detection areas.

# 644 It is determined whether the focus detection in the focus detection area performed in # 642 is impossible. If focus cannot be detected, proceed to # 654. If focus can be detected, #
Proceed to 646.

# 646 It is determined whether or not the focus detection result is in focus. If the defocus amount obtained by focus detection is within a predetermined amount, it is determined to be in focus. If in focus, proceed to # 650,
If it is not in focus, proceed to # 652.

# 648 In-focus display is performed to inform the photographer that the focus detection result is in-focus. LE for lighting
The D (F-LED) 25 is turned on, and the LCD drive circuit 10
The focus mark of the LCD 24 in the finder is turned on via the display 5.

# 650 Since focus has been achieved, "release release"
The interrupt is permitted, and when SW2 is pressed, the release operation is performed by the interrupt. Further, the "AF prohibition flag" is set so that the focus detection is not performed again.

The lens drive circuit 110 calculates the lens drive amount from the defocus amount detected in # 652 and # 642.
Communicate to. The lens drive circuit 110 drives the lens drive motor 33 while monitoring the pulse plate 36 to drive the communicated lens drive amount lens. After communicating data to the lens drive circuit 110, the CPU 100 does not need to monitor the lens drive amount and can perform another operation while driving the lens. Therefore, when the communication with the lens driving circuit is completed, the process proceeds to # 680.

# 654 An AF impossible display is performed to inform the photographer that the focus detection result is impossible. The AF disabled display is performed by blinking the focus mark on the LCD 24 in the viewfinder.

# 656 The "line-of-sight detection prohibition flag" is cleared to detect the line-of-sight again. As a result, when the "line-of-sight detection &AF" subroutine is called next, line-of-sight detection and focus detection are performed.

# 660 Focus detection is performed. The focus detection is performed on the focus detection area selected by the line-of-sight detection from the plurality of focus detection areas.

# 661 It is determined whether the focus detection performed in # 660 is impossible. If focus detection is not possible, the process proceeds to # 662, and if focus detection is possible, the process proceeds to # 664.

# 662 The "line-of-sight detection prohibition flag" is cleared to detect the line-of-sight again. As a result, when the "line-of-sight detection &AF" subroutine is called next, line-of-sight detection and focus detection are performed.

# 664 Here, the defocus data, the timing at which focus detection is performed, etc. are stored and updated in order to predict the movement of the subject.

# 668 The movement of the subject is calculated from a plurality of past focus detection data by a predetermined algorithm. Therefore, it is determined here whether or not there is sufficient past data. If there is enough data and the movement of the subject can be predicted, the process proceeds to # 670, and if there is not enough data, the process proceeds to # 678.

# 670 Defocus amount detected this time,
Defocus amount, lens drive amount, which was stored in the past,
From the focus detection interval and the release time lag, the defocus position of the subject when the shutter curtain actually travels is determined by a predetermined algorithm.

# 672 It is determined whether the calculated predicted defocus amount is appropriate. If the predicted defocus amount is larger than a certain threshold value or if the direction is reversed, the prediction result is not appropriate, so the process proceeds to # 676. If the lens drive amount is 0 for a plurality of times, it means that the subject is not moving, and the process similarly proceeds to # 676. If the prediction is correct #
Proceed to 674.

# 674 The predicted defocus amount is converted into the lens drive amount, and the value is communicated to the lens drive circuit 110 to drive the lens.

# 675 Release permission determination is performed. "Release feed interrupt" is enabled when the conditions that allow release are satisfied. When the lens is driven by the predicted amount, the lens is stopped inside this routine, and when the lens driving is completed, the “release feeding interrupt” is permitted. When the lens is driven by the defocus amount, when the lens driving amount is small, or when the lens driving amount is 0, the “release feeding interrupt” is permitted.

# 676 The prediction data is erased, and the prediction is initialized. When the prediction is inappropriate, either the data stored in the past or the defocus amount of this time is incorrect, so it is necessary to erase these data and store new prediction data.

# 678 When the prediction cannot be made, the lens is driven by the detected defocus amount. If the lens drive amount is smaller than a predetermined value, the lens drive is not performed in order to prevent the lens from slightly vibrating.

# 680 subroutine "line-of-sight detection &AF"
Ends and returns.

As described above, in the dynamic object orientation mode, the line-of-sight detection is performed again only when the focus detection becomes impossible. Therefore, when the line-of-sight position of the photographer changes as the subject moves, The focus detection area is changed accordingly, and a more improved dynamic object orientation mode can be realized.

As a result, in the dynamic object orientation mode, since the focus detection area is not determined and the focus adjustment is performed by the line-of-sight detection means only when the release button is first pressed, the subject moves during continuous shooting. Then, even if the gazing point of the photographer changes, the focus detection area changes accordingly, and a dynamic subject-oriented AF mode linked to the line of sight can be realized. In addition, the line-of-sight input is not always performed before focus detection, and the line-of-sight detection is performed only when focus detection is not possible.The focus detection point is linked to the gazing point, so focus detection is performed each time the line-of-sight detection is performed. You can take in-focus photos without spending extra time. In addition, if the line-of-sight detection is performed for each focus detection and the gaze point detected by the line-of-sight detection matches the focus detection area, it is impossible to keep an eye on the subject, and framing is performed carefully in every corner of the frame and in the viewfinder. It also eliminates the conventional drawback that you cannot see the shooting information displayed in.

(Other Embodiments) FIG. 2 illustrates the second embodiment.
Use the flow chart of.

What is different from the first embodiment is "line-of-sight detection &
Since the contents of the "AF" subroutine are different, only that part will be described.

When "visual axis detection &AF" is called, # 3
The subroutine is executed from 1.

# 31 It is determined here whether line-of-sight detection is permitted. Prohibition of line-of-sight detection can be determined by the “line-of-sight detection prohibition flag”. At first, since it is permitted, the process proceeds to # 32. If the line-of-sight detection is prohibited, the line-of-sight detection operation is not performed and the process proceeds to # 36.

# 32 The line-of-sight input mark is displayed to indicate that the line-of-sight input is being performed prior to the line-of-sight detection. Turn on the lighting LED (F-LED) 25, and
LCD 24 in viewfinder via CD drive circuit 105
Turn on the line-of-sight input mark. This allows the photographer to confirm that the camera is performing line-of-sight detection outside the viewfinder screen 207.

# 33 Line-of-sight detection is performed according to a predetermined algorithm. First, the IRED drive circuit 107 turns on the IRED that is predetermined according to the position of the camera (two or four of IRED1 to IRED6).
The line-of-sight detection circuit 101 starts the accumulation operation of the CCD-EYE 14. When the storage is completed, charge the storage by electing the CPU1
00 is sequentially read, A / D conversion is performed, and processing is performed by a predetermined algorithm. All pixels of the CCD-EYE 14 are processed, and the coordinates of the corneal reflection images 52b and 52a and the pupil 51 of the eyeball illumination light source shown in FIGS.
Get the center coordinates of. The coordinates gazed at by the photographer are obtained by calculating these according to the algorithm described above.

# 34 Once the line-of-sight is detected, the line-of-sight detection is prohibited so as not to repeat the line-of-sight detection. Prohibition of line-of-sight detection is performed by setting a “line-of-sight detection prohibition flag”.

# 35 A focus detection area is selected from the gaze point coordinates obtained by the sight line detection. Here, the gazing point coordinates and the focus detection area will be described with reference to FIG. In Figure 2, L2, L
1, C, R1 and R2 are C of the line sensor 6f, respectively.
CD-L2, CCD-L1, CCD-C, CCD-R
1, corresponding to CCD-R2, focus detection at each location is performed. x and y are coordinate axes when the gazing point on the finder is obtained, and the obtained gazing point coordinates are (xn, y
n).

If the gazing point coordinates are -x3 <xn≤-x2 and y1≤yn≤-yn, then L2-x2 <xn≤-x1 and y1≤yn≤-yn if L1-x1 <xn <x1 and y1≤. If yn≤-yn then C x1≤xn <x2 and if y1≤yn≤-yn then R1 x2≤xn <x3 and if y1≤yn≤-yn select the focus detection area of R2. When the gazing point is at the position shown in FIG. 3, x1 ≦ xn <x2 and y1 ≦ yn ≦ −yn, so the focus detection area is R1.

Next, in order to inform the photographer of the focus detection area selected by the line of sight, a signal is transmitted to the LED drive circuit 106 and the superimposing LED-21 is used to blink the focus detection area (light up). But good).

# 36 Here, it is determined whether or not the lens is being driven by communicating with the lens driving circuit 110. If the "line-of-sight detection &AF" subroutine was executed last time and the lens drive has been performed, and the lens drive has not been completed yet, the process proceeds to step # 80. Before the lens is driven or after the lens is driven, the process proceeds to # 37.

# 37 Here, it is determined whether the AF mode is the "static subject orientation mode" or the "dynamic subject orientation mode". If it is "static subject mode", go to # 40,
If it is the "dynamic subject-oriented mode," proceed to # 60.

# 40 It is determined by the "AF prohibition flag" that the focus detection is still permitted. When the focus is once achieved and the SW1 is still held, the "AF prohibition flag" is set. In this case, therefore, the process proceeds to # 80.

# 42 Focus detection is performed. The focus detection of the focus detection area selected by the line-of-sight detection is performed from the plurality of focus detection areas.

# 44 It is determined whether or not the focus detection in the focus detection area performed in # 42 is impossible. If the focus cannot be detected, the process proceeds to step # 54. If the focus can be detected, the process proceeds to step # 46.

# 46 It is determined whether or not the focus detection result is in focus. If the defocus amount obtained by focus detection is within a predetermined amount, it is determined to be in focus. If it is in focus, the process proceeds to step # 50, and if it is determined that it is not in focus, the process proceeds to step # 52.

# 48 Focusing display is performed to inform the photographer that the focus detection result is focused. LED for lighting
The (F-LED) 25 is turned on, and the LCD drive circuit 105
The focus mark on the LCD 24 in the viewfinder is turned on via.

# 50 Since the focus is achieved, the "release feed" interrupt is permitted, and when SW2 is pressed, the release operation is performed by the interrupt. Further, the "AF prohibition flag" is set so that the focus detection is not performed again.

In # 52 and # 46, if the lens is out of focus, the lens is driven. The drive amount of the lens is obtained from the defocus amount detected in # 42 and communicated to the lens drive circuit 110. The lens driving circuit 110 drives the lens driving motor 33 while monitoring the pulse plate 36 to drive the lens driving amount lens communicated. The CPU 100 does not need to monitor the lens driving amount after communicating the data to the lens driving circuit 110, and proceeds to # 80 when the communication with the lens driving circuit is completed.

# 54 An AF impossible display is performed to inform the photographer that the focus detection result is impossible. The AF disabled display is performed by blinking the focus mark on the LCD 24 in the viewfinder.

# 56 The "gaze detection prohibition flag" is cleared in order to detect the gaze again. As a result, when the "line-of-sight detection &AF" subroutine is called next, line-of-sight detection and focus detection are performed.

# 60 Focus detection is performed. The focus detection of the focus detection area selected by the line-of-sight detection is performed from the plurality of focus detection areas.

# 61 It is determined whether the focus detection performed in # 60 was impossible. If focus detection is not possible, the process proceeds to # 60 again, and focus detection is repeated. Although not shown in this flowchart, if the number of times of focus detection is limited and focus detection cannot be performed even after repeating the focus detection a fixed number of times, the focus detection may be stopped and this subroutine may be returned.

# 64 Here, the defocus data, the timing at which focus detection is performed, and the like are stored and updated in order to predict the movement of the subject.

# 68 The movement of the subject is calculated from a plurality of past focus detection data by a predetermined algorithm. Therefore, it is determined here whether or not there is sufficient past data. If there is enough data and the motion of the subject can be predicted, the process proceeds to # 70, and if there is not enough data, the process proceeds to # 69.

# 69 Here, the lens is driven by the detected defocus amount.

# 70 Here, from the defocus amount detected this time, the defocus amount stored in the past, the lens drive amount, the focus detection interval, and the release time lag,
The defocus position of the subject when the shutter curtain actually travels is calculated by a predetermined algorithm. Here, the principle of prediction will be briefly described with reference to FIG. Figure 5
Is a graph in which time is plotted on the horizontal axis and the image plane position of the subject is plotted on the vertical axis. LO is a curve showing the image plane position of the subject obtained from the detected defocus amount, and LL is a graph showing the position of the lens. The defocus amount obtained at time tA is DF1, the lens drive amount (image surface conversion) driven up to time tB is DL1, the defocus amount obtained at time tB is DF2, and the lens driven between time tC. Letting DL2 be the drive amount (image plane conversion) and DF3 be the defocus amount obtained at time tC, the points A, B, and C are represented by the coordinate system based on the position of the lens at time tA. D
F1), B (TM1, DL1 + DF2), C (TM1 +
TM2, DL1 + DL2 + DF3). Using these, a quadratic expression of the curve LO indicating the image plane movement of the subject can be obtained. If this is D = kt ² + 1t + m, the image plane position at time tX can be predicted by substituting t = TM1 + TM2 + TL into this equation. If the image plane position can be predicted, the lens drive amount DL can be obtained.

# 72 It is determined whether the calculated predicted defocus amount is appropriate. If the predicted defocus amount is larger than a certain threshold value or the direction is reversed, the prediction result is not appropriate, and the process branches to # 73. In addition to this, when the lens drive amount is 0 for a plurality of times, it means that the subject is not moving, and similarly, the process proceeds to # 73. If the prediction result is appropriate, proceed to # 74.

# 73 Continuity is determined and if there is no continuity, the process proceeds to # 75, otherwise the process proceeds to # 76. Here, the determination of continuity will be described with reference to FIGS. Similar to FIG. 5A, this figure is a graph in which the horizontal axis represents time and the vertical axis represents image plane position. Now, points A, B, and C in the graph are detected from the focus detection result of three times, the lens drive amount, and the focus detection interval. It is assumed that the actual subject is moving on a curve passing through A, B and C '. However, if the object is deviated from the focus detection area and the focus of a distant object is detected, the image plane position of point C in the figure will be detected. If the prediction calculation is performed using this point C, the subject will not be focused on the contrary to the subject. In this case, since the direction of the image plane moving speed from the point A to the point B and the direction of the image plane moving speed from the point B to the point C are opposite, it can be determined that there is no continuity.

FIG. 5 (c) shows a case in which a subject other than the subject enters the focus detection area. At this time point A-B
Since the image plane moving speed obtained at the point and the image plane moving speed obtained at the points B and C are suddenly changed, it can be determined that there is no continuity.

When photographing a moving subject, the photographer is photographing while looking at the subject. In such a case, the line-of-sight detection is performed again to determine the focus detection area. By performing the line-of-sight detection to determine the focus detection area, the prediction calculation can be correctly performed even when the subject is temporarily out of the focus detection area or another object is distance-measured.

# 75 Since there is no continuity of prediction, line-of-sight detection is permitted.

# 76 Here, only the defocus data detected this time and the parameter of the time interval between the previous time and this time are erased. The reason why all the prediction data is not erased is that when the line-of-sight detection is performed next and the focus detection of the correct subject is performed, the prediction can be performed using the past data. Next, the process proceeds to step # 80 and the subroutine is returned. If SW1 is still pressed after returning from this subroutine, the process returns to # 31 via # 620, # 604 and # 612 in FIG. 6 as described above. Since the line of sight is permitted here, the line of sight is detected.

# 74 The predicted defocus amount is converted into the lens drive amount, and the value is communicated to the lens drive circuit 110 to drive the lens.

# 75 Release permission is determined. "Release feed interrupt" is enabled when the conditions for permitting the release are satisfied. When the lens is driven by the predicted amount, the lens stop is awaited in this routine, and when the lens driving is completed, the "release feeding interrupt" is permitted. When the lens is driven with the defocus amount, the "release feeding interrupt" is permitted when the lens driving amount is small or 0.

# 76 The prediction is initialized. If the prediction is inappropriate, either the data stored in the past or the defocus amount this time is wrong, so it is necessary to erase these data and store new prediction data.

# 78 When the prediction cannot be made, the lens is driven with the detected defocus amount. If the lens drive amount is smaller than a predetermined value, the lens drive is not performed in order to prevent the lens from slightly vibrating.

# 80 Subroutine "Line-of-sight detection &AF"
Ends and returns.

In this way, when there is no continuity in the movement of the subject, it is possible to realize a dynamic subject orientation mode in which the subject is correctly followed by detecting the line of sight again and determining the focus detection area. This series of operations will be described with reference to FIG. Focus detection is started from FIG. First, focus detection is performed, the rightmost focus detection area is selected, and focus detection is performed. In FIG. 7B, focus detection is performed in the focus detection area determined first without performing line-of-sight detection. Therefore, at this time, it is possible to look away from the subject and view the shooting information in the viewfinder or the edge of the shooting screen. In FIG. 7C, since the subject has moved to the center, the subject has disappeared from the focus detection area that has been detected so far. Therefore, the detected image plane moving speed also changes rapidly. At this time, the line-of-sight is detected again to obtain the gazing point. If the gazing point is in the center, the focus detection area is in the center. 7 (d)) FIG.
Since the same subject as in (a) and (b) is detected, the prediction calculation can be performed using the past data and the current focus detection result.

As a result, in the dynamic subject orientation mode, the continuity of the subject is lost instead of determining the focus detection area by the line-of-sight detection means and performing focus adjustment only when the release button is first pressed. By performing line-of-sight detection, it is possible to realize a dynamic subject-oriented AF mode in which the focus detection area changes in association with the subject as the photographer wishes. Furthermore, since line-of-sight input is not performed each time before focus detection,
It is possible to take a focused photograph that follows a moving subject without requiring extra focus detection time each time for the time of sight line detection. In addition, since the line-of-sight detection is not performed every time, it is possible to look at the edge of the screen to perform framing while keeping the eyes away from the subject and to view the shooting information displayed in the viewfinder.

[0145]

[0146]

[0147]

[0148]

[0149]

According to the first aspect of the present invention, the visual axis detecting means for detecting the visual axis position of the user looking into the viewfinder, and the focus detecting means for performing the focus detection based on the detection result of the visual axis detecting means. And a camera having a line-of-sight detection unit having a determination unit that determines whether the focus detection result by the focus detection unit is appropriate, and a control unit of the camera that repeats the shooting operation while the release button is pressed,
The focus detection means repeats the focus detection operation at a predetermined time interval to predict the position of the subject after a predetermined time has passed from the past focus detection results, and the determination means is the current focus detection result and the past focus. The detection result is compared to determine that the continuity of the position of the subject is appropriate, and the continuity of the position of the subject is determined to be inappropriate. The line-of-sight detection means performs the line-of-sight detection operation again, so that the line-of-sight detection means is executed only when it is determined that the detection result by the focus detection means is not appropriate.
The followability of the focus detection means of the user can be improved.

The invention as set forth in claim 2 of the present application, a line-of-sight detection unit for detecting the line-of-sight position of the user looking into the finder, and a focus detection unit for performing focus detection based on the detection result of the line-of-sight detection unit, In a camera having a line-of-sight detection unit, which has a determination unit that determines whether or not the focus detection result by the focus detection unit is appropriate, and a control unit of the camera that performs continuous shooting while repeating the shooting operation while the release button is pressed. The focus detecting means repeats the focus detecting operation at a predetermined time interval to predict the position of the subject after a predetermined time has passed from the past focus detecting results, and the judging means determines the present focus detecting result and the past. The focus detection result is compared to determine that there is continuity in the position of the subject, and it is determined to be inappropriate when there is no continuity in the position of the subject. When the determination unit determines that the focus detection result is inappropriate during the first frame of the image capturing and the continuous image capturing, the line-of-sight detecting unit performs the line-of-sight detection operation again, so that the continuous image capturing is performed. The line-of-sight detection means is executed only when it is determined that the detection result by the first frame and the focus detection means is not appropriate, so that the followability of the focus detection means is improved without reducing the frame speed of continuous shooting. be able to.

[0151]

The invention described in claim 3 of the present application is to detect the visual axis of the user looking into the finder, and to perform focus detection based on the detection result of the visual axis detecting means. In a camera having a line-of-sight detection unit having a focus detection unit that continues the focus detection operation even after focusing and a determination unit that determines whether or not the detection result by the focus detection unit is appropriate, the focus detection unit is provided at predetermined time intervals. The focus detection operation is repeated to predict the position of the subject after a predetermined time has passed from the past focus detection results, and the determination means compares the focus detection result of this time with the past focus detection result of the subject. If there is continuity in the position, it is determined to be appropriate, and if there is no continuity in the position of the subject, it is determined to be inappropriate. By performing the line-of-sight detection operation, and the focus detection unit continues the focus detection operation based on the new detection result of the line-of-sight detection unit, the followability of the focus detection unit can be improved.

The invention according to claim 4 of the present application is to detect the visual axis of the user looking into the finder, and to perform focus detection based on the detection result of the visual axis detecting means. A focus detection unit that can continue the focus detection operation even after focusing, a determination unit that determines whether or not the focus detection result by the focus detection unit is appropriate, and a visual detection unit that performs the detection operation once each time. Inhibiting means for prohibiting the detecting operation of the line-of-sight detecting means, the focus detecting means repeats the focus detecting operation at a predetermined time interval, and predicts the position of the subject after a predetermined time has passed from past focus detection results, The determination means compares the focus detection result of this time with the past focus detection result and determines that there is continuity in the position of the subject, and determines that it is inappropriate when there is no continuity in the position of the subject. If there is something wrong,
And a release means for releasing the prohibition of the visual axis detection by the prohibiting means, and when the detection result of the focus detecting means based on the detection result of the visual axis detecting means is not appropriate, the visual axis detecting operation is performed again, and the visual axis detecting means is performed. By continuing the focus detection operation based on the new detection result of, the tracking performance of the focus detection means can be improved.

[Brief description of drawings]

FIG. 1 is a flowchart showing an operation of a “line-of-sight detection & AF” subroutine for explaining a first embodiment of the present invention.

FIG. 2 is a flowchart showing the operation of a “line-of-sight detection & AF” subroutine for explaining a second embodiment of the present invention.

FIG. 3 is a diagram for explaining a gaze point coordinate and a focus detection area.

FIG. 4 is a block diagram of a main part of an electric circuit of a camera having a line-of-sight detection unit.

FIG. 5 is a diagram illustrating the principle of a moving object predictive focus detection operation.

FIG. 6 is a flowchart of an operation of a camera having a line-of-sight detection unit.

FIG. 7 is a diagram for briefly explaining the operation of the present invention.

FIG. 8 is a schematic diagram of a line-of-sight detection optical system provided in the single-lens reflex camera.

FIG. 9 is a schematic view of an eyeball image.

FIG. 10 is a diagram illustrating the principle of line-of-sight detection.

[Explanation of symbols]

1 Shooting lens 2 primary mirror 7 focus plate 8 penta roof prism 11 eyepiece 12 Light receiving lens 13 Infrared emitting diode 14 image sensor 15 eyeballs 16 cornea 17 Iris 19 pupils 21 LED 100 CPU 101 line-of-sight detection circuit 102 Photometric circuit 103 Automatic focus detection circuit 104 signal input circuit 105 LCD drive circuit 107 IRED drive circuit 108 shutter control circuit 109 motor control circuit 110 lens drive circuit 111 Aperture drive circuit

Claims (4)

(57) [Claims] 1. The position of the line of sight of the user looking through the viewfinder
The line-of-sight detection means to detect and the detection result of the line-of-sight detection means
Focus detection means for performing focus detection based on the
Determining whether the result of focus detection by means is appropriate
Means and the shooting operation while the release button is pressed.
Repeating camera control means and line-of-sight detection means
In this camera, the focus detection means repeats the focus detection operation at predetermined time intervals.
Back, and after a certain time has passed from the past focus detection results,
This is to predict the position of the object,
The focus detection results of the
If there is continuity in the body position, it is judged to be appropriate and the position of the subject
If there is no continuity in the
When it is determined to be appropriate, the line-of-sight detection means again
Includes line-of-sight detection means characterized by performing line-of-sight detection
A camera to do. 2. A line-of-sight detection unit for detecting the line-of-sight position of a user looking through the viewfinder, a focus detection unit for performing focus detection based on a detection result of the line-of-sight detection unit, and a focus detection result by the focus detection unit. A camera having a line-of-sight detection means and a determination means for determining whether or not it is appropriate, and a camera control means for performing continuous shooting while repeating the shooting operation while the release button is pressed, wherein the focus detection means has a predetermined time interval. The focus detection operation is repeated in the above step to predict the position of the subject after a predetermined time has passed from the past focus detection results, and the determination means compares the current focus detection result with the past focus detection results to determine the subject. If the position of the subject has continuity, it is determined to be appropriate, and if the position of the subject has no continuity, it is determined to be inappropriate, and the first frame of the continuous shooting and the continuous shooting are determined. Wherein when the focus detection result is determined to be inappropriate by the determination unit, a camera having a sight line detecting means and performing the sight line detection unit again, visual axis detection operation during. 3. A line-of-sight detecting means for detecting a line-of-sight position of a user looking through a viewfinder, and focus detection based on a detection result of the line-of-sight detecting means, the focus detecting operation being continued even after focusing. In a camera having a line-of-sight detection unit having a focus detection unit and a determination unit that determines whether the focus detection result by the focus detection unit is appropriate, the focus detection unit repeats the focus detection operation at a predetermined time interval, This is for predicting the position of the subject after a predetermined time has elapsed from the focus detection result, and the determination means compares the present focus detection result and the past focus detection result and is appropriate if the subject position has continuity. If there is no continuity in the position of the subject, it is determined to be inappropriate. When it is determined to be inappropriate, the line-of-sight detection means performs the line-of-sight detection operation again, Detecting means camera having a sight line detecting means, characterized in that continued focus detection operation based on the new detection result of the visual axis detecting means. 4. A line-of-sight detecting means for detecting a line-of-sight position of a user looking through a finder, and focus detection based on a detection result of the line-of-sight detecting means, the focus detecting operation being continued even after focusing. Focus detecting means, a determining means for determining whether or not the focus detection result by the focus detecting means is appropriate, and a detecting operation of the visual axis detecting means is prohibited every time the visual axis detecting means performs the detecting operation once. The prohibition means and the focus detection means repeat the focus detection operation at a predetermined time interval, predict the position of the subject after a predetermined time has passed from the past focus detection results, and the determination means is the focus detection result of this time. And the past focus detection results are compared, it is judged to be appropriate if there is continuity in the position of the subject, and it is judged to be inappropriate if there is no continuity in the position of the subject. To Camera having a sight line detecting means and having a release means for releasing the inhibition of the line-of-sight detection by said inhibiting means.