JPH06153092A - Storage controller of photoelectric conversion element - Google Patents

Abstract

PURPOSE:To shorten a period of time taken for a series of storage control and to improve a function of a whole system through the use of a storage controller by delaying transfer speed only when transfer driving is executed within an area requiring signal quantity. CONSTITUTION:Plural unit image elements K and L consisting of a photoelectric converting part SIMG which executes photoelectric conversion in accordance with irradiate luminance and an electric charge storage part SSTR which stores the photoelectrically converted electric charge are continuously arrayed. Information for deciding the storage time of a sensor which transfers the photoelectrically converted electric charge within image elements to one of the adjacent image elements by a transfer signal is sensor-driven in the unit of block before the storage and transfer speed is delayed only when transfer is executed within the block area which requires the information. Thus, an image detection time does not take long without relying on the number of the image element and the storage time of the censor is correctly controlled so that the efficiency of the whole system is improved.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a photoelectric conversion element which uses a sensor composed of a photoelectric conversion element and which controls accumulation of photoelectric conversion elements suitable for use in, for example, a visual axis detection device or a focus detection device of a camera. Storage controller.

[0002]

2. Description of the Related Art A photoelectric conversion element can obtain an electric signal corresponding to the amount of light to be irradiated, but is not affected by an output called dark current which does not depend on the amount of light to be irradiated, within a limited storable electric signal capacity. In addition, it is necessary to control the storage time according to the amount of light emitted so that the optimum electric signal amount that can obtain the contrast component to be obtained is obtained.

Therefore, in order to obtain information for controlling the storage time for obtaining an optimum signal amount, the following preprocessing operation is generally performed.

First, the amount of signal is obtained by previously accumulating for a certain amount of accumulation time, and the amount of signal for each pixel obtained at this time is read from the photoelectric conversion element.

Next, since the amount of light applied to the photoelectric conversion element for controlling storage is not uniform, all pixel outputs or signal processing is performed for pixels forming a required partial area.

Then, within the area of the optimum signal amount, the pixel output based on the result of the signal processing performed on a pixel-by-pixel basis for the optimum accumulation time at which the required electric signal output of the pixel will be obtained by photoelectric conversion. Is obtained from the average value and the accumulation time, and the accumulation control is performed according to the obtained accumulation time to obtain an electric signal.

Further, in order to obtain the information for controlling the accumulation time, it is conceivable to control the accumulation time from the luminance information of the object in an optical device having a photometric means such as a camera capable of measuring the luminance of the object.

[0008]

In the conventional storage control device as described above, in the storage control of the photoelectric conversion element performed to obtain the optimum signal amount, the area necessary for obtaining the information for controlling the storage time is controlled. Since the average value of pixel output is obtained by reading drive and signal processing for each pixel, 1
In order to obtain the optimum image signal of two times, the accumulation is performed twice, each pixel is read, and signal processing is performed. The time required is enormous and the number of pixels is large, so that the number of pixels is increased. It is difficult to deal with.

In the case of use in a visual axis detection device of a camera, the time required for visual axis detection occupies a very large rate as compared with the other series of processing operation time of the camera, and the information from the photoelectric conversion element output is simply used. Not only the time required for the existing device but also the function deterioration in the entire system using the storage control device, the user's discomfort becomes significant.

The present invention has been made to solve the above problems, and it is intended to shorten the time required for a series of storage control and to improve the function of the entire system using a storage control device.

Alternatively, as described above, the control of the storage time by the photometric means requires a short time, but only a very rough control can be performed, which is insufficient for performing image processing such as visual axis detection with high accuracy.

[0012]

Therefore, the transfer speed is slowed only when transfer driving is performed in a region requiring a signal amount, so that the charges photoelectrically converted at the pixel position are accumulated. At the same time, the amount of signal processing after reading is reduced by the cumulative accumulation output of the charges accumulated at each pixel position during transfer.

[0013]

DESCRIPTION OF THE PREFERRED EMBODIMENTS The contents will be described in detail by taking the case where the present invention is applied to a visual axis detecting device as an example, but before that, the visual axis detecting method will be briefly described first.

FIG. 20 is an explanatory view of the principle of the line-of-sight detection method, FIG.
1A and 1B are explanatory diagrams of the eyeball image projected on the surface of the image sensor 14 in FIG. 20 and the output intensity from the image sensor 14.

Next, a line-of-sight detection method will be described with reference to FIGS. 20, 21 (A) and 21 (B). The infrared light emitting diodes 13a and 13b as light sources are arranged substantially symmetrically in the Z direction with respect to the optical axis A of the light receiving lens 12, and divergently illuminate the photographer's eyes.

The infrared light emitted from the infrared light emitting diode 13b illuminates the cornea 16 of the eyeball 15. At this time, the corneal reflection image d by a part of the infrared light reflected on the surface of the cornea 16
Is collected by the light receiving lens 12 and the image sensor 14
The image is re-imaged at the upper position d '.

Similarly, the infrared light emitted from the infrared light emitting diode 13a illuminates the cornea 16 of the eyeball. At this time, the cornea reflection image e by a part of the infrared light reflected on the surface of the cornea 16 is condensed by the light receiving lens 12 and the image sensor 1
The image is re-imaged at the position e'on the position 4.

The light beams from the ends a and b of the iris 17 form the images of the ends a and b at the positions a ′ and b ′ on the image sensor 14 via the light receiving lens 12. When the rotation angle θ of the optical axis a of the eyeball 15 with respect to the optical axis of the light receiving lens 12 (optical axis a) is small, the Z coordinate of the ends a and b of the iris 17 is set to X.
Assuming a and Xb, the coordinate Xc of the center position c of the pupil 19 is expressed as Xc≈ (Xa + Xb) / 2.

Further, since the Z coordinate of the midpoint of the corneal reflection images d and e and the Z coordinate Zo of the center of curvature O of the cornea 16 coincide with each other,
The X-coordinates of the corneal reflection image generation positions d and e are Xd and Xe, the standard distance from the center of curvature O of the cornea 16 to the center C of the pupil 19 is L _OC, and a coefficient that considers individual differences with respect to the distance L _OC Where A1 is A1, the rotation angle θ of the eyeball optical axis B substantially satisfies the relational expression of (A1 * L _OC ) * sin θ≈Xc− (Xd + Xe) / 2 (1). Therefore, by detecting the positions of the respective feature points (corneal reflection images d and e and the edges a and b of the iris) projected on a part of the image sensor in the visual line processing device as shown in FIG. The rotation angle θ of the optical axis a of the eyeball can be obtained. At this time, the expression (1) can be rewritten as β (A1 * L _OC ) * sin θ≈ (Xa ′ + Xb ′) / 2− (Xd ′ + Xe ′) / 2 (2). However, β is a magnification determined by the position of the eyeball with respect to the light receiving lens 12, and is substantially obtained as a function of the interval | Xd′−Xe ′ | of corneal reflection images. *
Represents multiplication. The rotation angle θ of the eyeball 15 can be rewritten as θ≈ARCSIN {(Xc′−Xf ′) / β / (A1 * L _OC )} (3). However, Xc′≈ (Xa ′ + Xb ′) / 2 Xf′≈ (Xd ′ + Xe ′) / 2. By the way, since the optical axis a of the photographer's eye does not coincide with the visual axis, the horizontal rotation angle θ of the optical axis a of the photographer's eye
When is calculated, the angle of view δ between the optical axis of the eyeball and the visual axis is corrected to obtain the horizontal line of sight θH of the photographer.
Letting B1 be a coefficient considering the individual difference with respect to the correction angle δ between the optical axis a of the eyeball and the visual axis, the horizontal line of sight θH of the photographer
Is calculated as θH = θ ± (B1 * δ) (4). Here, if the angle of rotation to the right of the photographer is positive, the sign ± is selected as + when the eye of the photographer looking through the observation device is the left eye, and when the eye of the photographer is the right eye.

Further, in the figure, the eyeball of the photographer is Z-X.
It shows an example of rotation in a plane (eg horizontal plane),
The same can be detected when the photographer's eyeball rotates in the Z-Y plane (for example, a vertical plane). However, since the vertical component of the line of sight of the photographer coincides with the vertical component θ ′ of the optical axis of the eyeball, the vertical line of sight θV is θV = θ ′. Further, from the line-of-sight data θH, θV, the position (Xn, Y
n) is calculated as Xn≈m * θH≈m * [ARCSIN {(Xc′−Xf ′) / β / (A1 * L _OC )} ± (B1 * δ)] (5) Yn≈m * θV To be However, m is a constant determined by the finder optical system of the camera.

Here, the values of the coefficients A1 and B1 for correcting the individual differences of the eyeballs of the photographer have the photographer fixate the index arranged at a predetermined position in the viewfinder of the camera, and the position of the index. And the position of the fixation point produced according to equation (5).

The calculation for obtaining the line-of-sight and gazing point of the photographer in this embodiment is executed by the software of the microcomputer of the line-of-sight calculation processing device based on the above equations.

Further, since the coefficient for correcting the individual difference of the line of sight usually corresponds to the horizontal rotation of the eyeball of the observer, the two indexes provided in the viewfinder of the camera are for the observer. It is set to be horizontal.

The coefficient for correcting the individual difference of the line of sight is obtained, and the position of the line of sight of the observer looking into the viewfinder of the camera is calculated using the formula (5), and the line-of-sight information is used for focus adjustment or exposure of the photographing lens. It is used for control.

FIG. 1 is a layout view of the essential parts of Embodiment 1 when the present invention is applied to a single-lens reflex camera provided with a visual axis detection device, and FIG. 2 is a rear view when the present invention is applied to a single-lens reflex camera.
FIG. 3 is an explanatory view in the viewfinder field of FIG.

In each of the drawings, reference numeral 1 is a taking lens, which is 2 for convenience.
Although shown with one lens, it is actually composed of a larger number of lenses. Reference numeral 2 denotes a main mirror, which is obliquely installed or retreated in the photographing optical path according to the observation state of the subject image by the finder system and the photographing state of the subject image. Reference numeral 3 denotes a sub-mirror, which reflects the light flux transmitted through the main mirror 2 toward a focus detection device 6 described below below the camera body.

Reference numeral 4 is a shutter, and 5 is a photosensitive member, which is composed of a silver salt film, a CCD or MOS type solid-state image pickup device, or an image pickup tube such as a vidicon.

Reference numeral 6 denotes a focus detection device, which is a field lens 6a and reflection mirrors 6b and 6 arranged near the image plane.
c, a secondary imaging lens 6d, a diaphragm 6e, a line sensor 6f including a plurality of CCDs, and the like.

The focus detecting device 6 in this embodiment uses a well-known phase difference method, and as shown in FIG. 3, a plurality of regions (5 places) in the observation screen (in the viewfinder field) are used as distance measuring points. The focus detection point is configured to be capable of focus detection.

Reference numeral 7 is a focusing plate arranged on the planned image forming surface of the taking lens 1, 8 is a pentaprism for changing the finder optical path, and 9 and 10 are image forming lenses for measuring the brightness of the subject in the observation screen. It is a photometric sensor. The imaging lens 9 conjugately connects the focusing plate 7 and the photometric sensor 10 via the reflection optical path in the penta roof prism 8.

Next, an eyepiece lens 11 having a light splitter 11a is arranged behind the exit surface of the penta roof prism 8 and is used for observation of the focusing plate 7 by the photographer's eye 15. The light splitter 11a is composed of, for example, a dichroic mirror that transmits visible light and reflects infrared light.

Reference numeral 12 is a light-receiving lens, and 14 is an image sensor in which a photoelectric element array such as a CCD is two-dimensionally arranged so as to be conjugated with the vicinity of the pupil of the photographer's eye 15 at a predetermined position with respect to the light-receiving lens 12. ing. 13a to 13f
Are infrared light emitting diodes, which are the illumination light sources,
As shown in FIG. 2, it is arranged around the eyepiece lens 11.

Reference numeral 21 is a high-intensity superimposing LED that can be visually recognized even in a bright subject, and the emitted light is reflected by the main mirror 2 via the projection prism 22 and is provided on the display portion of the focusing plate 7. It is bent in the vertical direction by the micro prism array 7a, passes through the penta prism 8 and the eyepiece lens 11 and reaches the photographer's eye 15.

Therefore, the fine prism array 7a is provided at a plurality of positions (distance measuring points) corresponding to the focus detection area of the focusing plate 7.
Are formed in a frame shape, and five superimposing LEDs 21 (each of which are LED-L1 and LED
-L2, LED-C, LED-R1, LED-R2).

As a result, as can be seen from the field of view of the finder shown in FIG.
Numerals 1, 202, 203 and 204 illuminate in the finder field to display a focus detection area (distance measuring point) (hereinafter referred to as superimpose display).

Reference numeral 23 is a field mask for forming a finder field area. Reference numeral 24 denotes an LCD in the finder for displaying photographing information outside the finder field of view.
-LED) 25.

The light transmitted through the LCD 24 is the triangular prism 2
6 is guided into the field of view of the finder, and 207 of FIG.
It is displayed outside the field of view of the viewfinder as shown by, and the photographer can know the photographing information. Reference numeral 27 is a known mercury switch for detecting the attitude of the camera.

Reference numeral 31 denotes an aperture provided in the taking lens 1, 32
Is a diaphragm driving device including a diaphragm driving circuit 111 described later, 3
Reference numeral 3 is a lens driving motor, 34 is a lens driving member including a driving gear, and 35 is a photocoupler which detects rotation of the pulse plate 36 interlocking with the lens driving member 34 and transmits it to the lens focus adjusting circuit 110. Focus adjustment circuit 110
On the basis of this information and the information on the lens driving amount from the camera side, the lens driving motor is driven by a predetermined amount to move the taking lens 1 to the in-focus position. 37
Is a mount contact that serves as an interface between a known camera and a lens.

FIG. 4 is an electric circuit diagram built in the camera.
The same members as those in FIG. 1 are given the same numbers.

A central processing unit (hereinafter referred to as a CPU) 100 of a microcomputer incorporated in the camera body includes a visual axis detection circuit 101, a photometric circuit 102, and an automatic focus detection circuit 10.
3, signal input circuit 104, LCD drive circuit 105, LE
The D drive circuit 106, the IRED drive circuit 107, the shutter control circuit 108, and the motor control circuit 109 are connected. Further, a focus adjustment circuit 1 arranged in the photographing lens
10. Signals are transmitted to and from the diaphragm drive circuit 111 via the mount contact 37 shown in FIG.

EEPROM 10 attached to CPU 100
Reference numeral 0a has a function of storing the line-of-sight correction data as a storage unit for correcting individual differences in line-of-sight.

The line-of-sight detection circuit 101 A / D converts the output of the eyeball image from the image sensor 14 (CCD-EYE) and sends this image information to the CPU 100. CPU1
As will be described later, 00 extracts each feature point of the eyeball image required for sight line detection according to a predetermined algorithm, and further calculates the photographer's sight line from the position of each feature point.

The photometric circuit 102, after amplifying the output from the photometric sensor 10, performs logarithmic compression and A / D conversion, and sends it to the CPU 100 as luminance information of each sensor. The photometric sensor 10 is the left focus point 2 in the viewfinder field shown in FIG.
SPC-L for photometry of the left area 210 including 00 and 201
SPC-C for measuring the central area 211 including the center distance measuring point 202, SPC-R for measuring the right area 212 including the right distance measuring points 203 and 204, and their peripheral areas 21.
SPC-A for photometry of 3 and a photodiode for photometry of four areas.

As shown in FIG. 3, the line sensor 6f in FIG. 4 includes five sets of line sensors CCD-L2 and CCD-L corresponding to the five distance measuring points 200 to 204 in the screen.
This is a known CCD line sensor composed of 1, CCD-C, CCD-R1, and CCD-R2.

The automatic focus detection circuit 103 A / D-converts the voltage obtained from these line sensors 6f, and the CPU 10
Send to 0. SW-1 is O at the first stroke of the release button
N, switch for starting photometry, AF, line-of-sight detection operation, SW-2 is a release switch that is turned on by the second stroke of the release button, SW-ANG is an attitude detection switch that is detected by the mercury switch 27, SW -A
EL is an AE lock switch, SW-DIAL1 and SW-DIAL2, which is turned on by pressing the AE lock button.
Is input to the up / down counter of the signal input circuit 104 by a dial switch provided in the electronic dial (not shown) to count the rotation click amount of the electronic dial.

Reference numeral 105 denotes a known LCD drive circuit for driving the liquid crystal display device LCD to display the aperture value, the shutter speed, the set photographing mode, etc. according to a signal from the CPU 100 on the monitor LCD 42 and the viewfinder LCD 24. Both can be displayed at the same time. L
The ED drive circuit 106 is a lighting LED (F-LED) 25.
Then, the superimposing LED 21 is turned on and controlled to blink. The IRED drive circuit 107 is an infrared light emitting diode (I
RED1 to 6) 13a to 13f are selectively turned on according to the situation.

When the shutter control circuit 108 is energized, it controls the magnet MG-1 for moving the front curtain and the magnet MG-2 for moving the rear curtain to expose the photosensitive member with a predetermined amount of light. The motor control circuit 109 is for controlling the motor M1 for winding and rewinding the film and the motor M2 for charging the main mirror 2 and the shutter 4. These shutter control circuits 108,
A series of camera release sequences operate by the motor control circuit 109.

Next, a flow chart of the operation of the camera having the visual axis detecting device will be described below with reference to FIG.

When the mode dial (not shown) is rotated to set the camera from the inoperative state to the predetermined photographing mode (this embodiment will be described based on the case where the shutter priority AE is set), the power of the camera is turned on. Yes (# 100), C
The variable used for the sight line detection of the PU 100 is reset (# 101).

Then, the camera waits until the switch SW1 is turned on by pressing the release button (# 10).
2). When the signal input circuit 104 detects that the release button is pressed and the switch SW1 is turned on, the CPU1
00 is confirmed with the line-of-sight detection circuit 101 (# 103).

At this time, if the line-of-sight prohibition mode is set, a specific range-finding point is selected by the range-finding point automatic selection subroutine (# 116) without executing line-of-sight detection, that is, without using line-of-sight information. At this focus detection point, the automatic focus detection circuit 103 performs focus detection operation (# 107).

Although several methods can be considered as algorithms for automatically selecting distance measuring points, a near point priority algorithm in which weighting is applied to a central distance measuring point is effective. Here, the contents of the present invention will be described. Since it is not directly related, the description is omitted.

When the visual axis detection mode is set, visual axis detection is executed (# 104). Here, the line-of-sight detection circuit 1
The line of sight detected at 01 is converted into the gazing point coordinates on the focus plate 7. The CPU 100 selects a distance measuring point close to the gazing point coordinates, transmits a signal to the LED driving circuit 106, and causes the distance measuring point mark to blink using the superimposing LED 21 (# 105).

When the photographer sees the distance measuring point selected by the photographer's line of sight and recognizes that the distance measuring point is not correct, he releases his hand from the release button and turns off the switch SW1 ( # 106), the camera waits until the switch SW1 is turned on (# 102).

As described above, the fact that the distance measuring point is selected by the line-of-sight information is displayed by blinking the distance measuring point mark in the viewfinder to notify the photographer, so that the photographer can arbitrarily select the distance measuring point. You can check whether or not.

Further, seeing that the distance measuring point selected by the photographer is displayed by the photographer, the switch SW1 is continuously turned on.
If continued (# 106), the automatic focus detection circuit 103
Performs focus detection of one or more focus detection points using the detected line-of-sight information (# 107).

It is determined whether or not the distance measuring point selected here cannot be measured (# 108). If it is impossible, the CPU 100 sends a signal to the LCD drive circuit 105 to set L in the finder.
The focus mark on the CD24 blinks, and distance measurement is NG (impossible)
That is, the photographer is warned (# 118) and continues until SW1 is released (# 119).

Distance measurement is possible, and the focus adjustment state of the distance measurement point selected by the predetermined algorithm is not in focus (#
109), the CPU 100 sends a signal to the lens focus adjustment circuit 110 to drive the taking lens 1 by a predetermined amount (# 11).
7). After driving the lens, the automatic focus detection circuit 103 performs focus detection again (# 107), and determines whether or not the taking lens 1 is in focus (# 109).

If the taking lens 1 is in focus at a predetermined distance measuring point, the CPU 100 causes the LCD drive circuit 105 to operate.
Is sent to turn on the focus mark on the LCD 24 in the finder, and also the signal is sent to the LED drive circuit 106 to display the focus on the focusing point 201 in focus (# 110).

At this time, the blinking display of the distance measuring point selected by the line of sight is turned off, but the distance measuring point displayed in focus is often coincident with the distance measuring point selected by the line of sight. The focus range-finding point is set to a lighting state so that the photographer can recognize that the focus has been achieved. When the photographer sees that the focusing point is displayed in the viewfinder and recognizes that the focusing point is incorrect, he releases the release button and turns off the switch SW1 (# 111). Waits until switch SW1 is turned on (# 10
2).

If the photographer looks at the focus detection point displayed in focus and continues to turn on the switch SW1 (# 1
11), the CPU 100 sends a signal to the photometry circuit 102 to perform photometry (# 112). At this time, the exposure value is calculated by weighting the photometric areas 210 to 213 including the in-focus distance measuring points.

In the case of the present embodiment, the known photometry calculation is performed by weighting the photometry area 210 including the distance measuring point 201.

Further, the release button is pressed and the switch S
It is determined whether or not W2 is turned on (# 11
3) If the switch SW2 is OFF, the state of the switch SW1 is checked again (# 111). When the switch SW2 is turned on, the CPU 100 sends signals to the shutter control circuit 108, the motor control circuit 109, and the diaphragm drive circuit 111, respectively.

First, M2 is energized to raise the main mirror 2 and the diaphragm 31 is narrowed down. Then, MG1 is energized to open the front curtain of the shutter 4. The aperture value of the aperture 31 and the shutter speed of the shutter 4 are determined from the exposure value detected by the photometric circuit 102 and the sensitivity of the film 5.
M after a predetermined shutter time (1/250 second, for example)
Energize G2 and close the rear curtain of the shutter 4. When the exposure of the film 5 is completed, the M2 is energized again, the mirror is down, the shutter is charged, and the M1 is also energized, the film is advanced by a frame, and the series of shutter release sequence operations is completed. (# 114) After that, the camera waits until the switch SW1 is turned on again (# 102).

FIG. 6 is a flow chart of the visual axis detection.
As described above, the line-of-sight detection circuit 101 executes the line-of-sight detection upon receiving a signal from the CPU 100 (# 104).

In the case of detecting the line of sight in the photographing mode, the line-of-sight detection circuit 101 first detects the posture of the camera via the signal input circuit 104 (#
201). The signal input circuit 104 is a mercury switch 27 (S
W-ANG) output signal is processed to determine whether the camera is in the horizontal position or the vertical position, and in the case of the vertical position, for example, whether the release button is in the top direction or the ground (plane) direction. to decide.

Next, from the previously detected camera posture information and the photographer's eyeglass information included in the calibration data, the infrared light emitting diode (hereinafter referred to as IRED) 13a.about.
13f is selected (# 202). That is, the camera is held in the horizontal position, and the photographer puts on the glasses and selects 13a and 13b. Also, if the camera is in the horizontal position and the photographer wears glasses, the IRED 13 that is far from the viewfinder optical axis
c and 13d are selected. At this time, a part of the illumination light reflected by the photographer's spectacles reaches a region other than a predetermined region on the image sensor 14 on which the eyeball image is projected, so that the analysis of the eyeball image does not occur.

Further, if the camera is held in the vertical position, an IRED that illuminates the photographer's eyeball from below.
Combinations of IRED13a, 13e or IRED13
Either combination of b and 13f is selected.

Next, based on the luminance information (brightness) of the eyeball, C
Set the accumulation time and amplification characteristics of the CD-EYE 14 (#
203). The eyeball image detected in order to know the brightness information of the eyeball is divided into a plurality of areas, based on each brightness value, for example, outside light is hitting the eyeball, or in a dark room, The illumination conditions of the eyeballs due to the external light are determined, and the optimum accumulation time and the amplification characteristic of the sensor output are set so that the Purkinje image and the feature points of the pupil can be easily extracted.

The driving method of the CCD-EYE for dividing the eyeball image into a plurality of areas and obtaining the luminance information of the eyeball is the main point of the present invention, which will be described in detail later.

When the CCD-EYE storage time is set,
The CPU 100 sends the IRE through the IRED drive circuit 107.
D is turned on with a predetermined power (# 204), and the line-of-sight detection circuit 101 starts accumulating CCD-EYE (# 205).

The CCD-EYE finishes the accumulation according to the previously set accumulation time of the CCD-EYE, and at the same time, the IRED is turned off. The CPU 100 reads out the eyeball image of the photographer from the CCD-EYE whose accumulation has been completed, and at the same time, performs a corneal reflection image (hereinafter, Purkinje image) and a feature extraction process of the pupil portion (# 206).

Since the Purkinje image appears as a bright spot having a strong light intensity, it is possible to set a predetermined threshold value for the light intensity and to extract a Purkinje image candidate exceeding the threshold value as a Purkinje image candidate. A plurality of boundary points between the pupil 19 and the iris 17 are extracted.

After the feature extraction of the Purkinje image pupil is completed, the position of one set of Purkinje images and the position of the center of the pupil are detected based on these information (# 207). CC
A set of IREDs located almost on one line of D-EYE14
Position (Xd ', Yd') of Purkinje's image which is a virtual image of
(Xe ′, Ye ′) is detected, and the center position of the pupil (X
c ', Yc') is detected by performing the least squares approximation of the circle based on the extracted boundary points. At this time, pupil diameter r
p is also calculated, and the distance is calculated from the positions of the two Purkinje images (# 207).

Further, the reliability of the calculated position of the Purkinje image and the center of the pupil is judged from the contrast of the eyeball image or the size of the pupil (# 208).

Next, the distance between the eyepiece lens 11 of the camera and the eyeball 15 of the photographer is calculated from the distance between the Purkinje images, and further, the distance between the eyepiece lens 11 and the eyeball 15 of the photographer is projected on the CCD-EYE. The imaging magnification β of the eyeball image is calculated (# 209). From the above calculated values, the rotation angle θ of the optical axis of the eyeball 15 is calculated from (3) (# 210).

When the rotation angles θ _X and θ _Y of the eyeball of the photographer are obtained, the position of the line of sight on the focusing plate 7 is obtained from the equation (5) (#
211). Thus, the visual axis detection routine is returned (# 212).

Next, the detection area of the CCD-EYE 14 is divided into a plurality of areas, and the CCD-E for obtaining the luminance information of each area.
The driving method of the YE 14 will be described.

The line-of-sight detection circuit 101 has a sensor driving device SDR (not shown) built therein. When the sensor is driven, timing signals φVI and φHI are supplied from the CPU 101, and the sensor driving signal φVO is supplied based on these two timing signals. , ΦHO are output to drive the image sensor CCD-EYE14. Image sensor CC
D-EYE will be described with reference to FIG.

FIG. 7 shows vertical K pixels and horizontal L pixels.
It is a block diagram showing a CCD area sensor of a vertical frame transfer system. Since the internal structure of the CCD sensor and the mechanism of charge transfer are not directly related to the present invention, the description thereof will be omitted.

SIMG is a photosensitive area, and is composed of pixels IMPXL arranged in K in the vertical direction and L in the horizontal direction. The IMPXL is a unit pixel having a photoelectric conversion function such as a photodiode and a charge transfer function, and a signal generated here is applied with a vertical transfer clock φVO to generate φVO inside the CCD-EYE 14. It is driven by a plurality of drive signals generated on the basis.

SSTR is a storage area for temporarily storing the charges generated in the photosensitive area SIMG, and the unit storage section STPXL forming the SSTR is a unit pixel I forming the SIMG.
The number is the same as MPXL, and the transfer clock is also the vertical transfer clock φVO.

HSHIFT is a horizontal transfer register, which is driven by a plurality of drive signals generated based on φHO in the CCD-EYE 14 by applying a horizontal transfer clock φHO input from the sensor driving device SDR. It

A signal is transferred to φRO by HSHIFT. In synchronization with this, when a signal is output from SOUT, a signal that initializes the output for each pixel in the signal amplifier circuit SAMP so that the influence of the output of the previous pixel does not affect the next output pixel in the output order of SOUT. is there.

A voltage for controlling the saturation level of the pixel IMPXL is applied to VOF. The OFD applies a charge of a predetermined level or higher so that the charge generated according to the amount of light applied to the pixel IMPXL reaches a predetermined level controlled by the voltage applied to the VOF, that is, when the charge does not become oversaturated. It is a discharge groove that is provided for discharging and is generally called an overflow drain.

Next, a method of obtaining the luminance information of the eyeball for setting the accumulation time of the image sensor CCD-EYE shown in # 203 of FIG. 6 will be described with reference to FIGS.

Here, the image sensor CCD-EYE
As shown in FIG. 8, 14 two-dimensionally arranged K × L pixels are divided into three equal parts in the vertical direction and four equal parts in the horizontal direction. It is what you want. At this time, the area divided into 12 is (K / 3)
It is composed of × (L / 4) pixels.

FIG. 9 shows horizontal components extracted from the 12 regions shown in FIG. SIMGBV1 in the figure,
SIMGBV2 and SIMGBV3 are areas obtained by dividing the photosensitive area SIMG into three equal parts in the vertical direction, and attention is paid to the line placed at the uppermost position in each of these areas. That is,
SIMGBV1 is the first line, SIMGBV2 is the (K / 3) +1 line, and SIMGBV3 is (2K /
3) It is the + 1st line. The drive timing will be described below.

First, clear driving for discharging unnecessary charges is performed. To clear in Fig. 11, φVO is K times, φ
FIG. 9 is a timing diagram in which V-TRANS, which is a set in which HO is repeated L times, is performed for two cycles to drive SIMG and SSTR to perform clear driving which is initialization. The description of V-TRANS will be given later.

Next, an AGC sequence is performed. 11A
The GC sequence is composed of three sub-sequences AGC1, AGC2 and AGC3. Further, each sub-sequence is composed of driving for accumulation, V-TRANS, and reading. Of these, V-TRANS is the same drive as V-TRANS during the clear operation.

First, the sub-sequence AGC1 will be described. AGC1 is the drive timing when the accumulation operation is performed for the area of SIMGBV1 in FIG.
FIG. 12 is a timing diagram showing this in detail.

In the accumulation shown in FIG. 12, DRV-B and DRV
The drive is performed by two drive patterns of -A. DRV
-A, P (1), C (2,1), C in DRV-B
Each of (1, 1) corresponds to one transfer driving operation, whereby the charges are transferred to one adjacent pixel.

First, by setting φHO = HI, the switch of the overflow drain arranged along the horizontal transfer register HSHIFT (not shown) is turned on, and the excess charge transferred to HSHIFT is discharged by the vertical transfer clock φVO. To be able to. Next, vertical transfer is performed by φVO.

First, during DRV-B, φVO drives P (1) to P ((K / 3) -1) at equal intervals. Here, during P (1), the first line in FIG. 9 remains on the first line, and the second φV
It is transferred to the second line by the O output and remains on the second line until P (2) until the third φVO output is performed. This is repeated K / 3 times to transfer to the K / 3 line which is the lower end of SIMGBV1.

Here, each interval from P (1) to P (K / 3) is a transfer interval in the vertical direction of the area SIMGB1, and the charge generated in each pixel that has remained during the transfer is accumulated. It is the accumulation time.

That is, in the area from the 1st line to the K / 3rd line, accumulation is performed for each pixel in the vertical direction at a time of P (1), which is equal to the total sum of these accumulated charges in the vertical direction. Will be things.

Following this, a drive operation called DRV-A is performed. The charges transferred to the K / 3 line in the DRV-B are transferred to the 2K / 3 line in the first DRV-A, and further transferred to the K line in the second DRV-A. . Here, the times C (2,1) and C (1,1) in the figure are the same, and both the first and second DRV-A are C.
ΦVO is repeatedly output at intervals of (2,1) and C (1,1). Even during this period, charge generation occurs according to the amount of light irradiated, but since it is minute compared to the time indicated by P (1) to P (K / 3), the amount of generated charge also becomes minute and DRV- There is almost no effect on the amount of charge generated in B. K
The charge transferred to the line line is continuously V-TR
The storage is completed by being transferred to the storage unit by the ANS.

Next, V-TRANS will be described. VT
RANS transfers the accumulated charge to STLLOW which is the lowermost line in SSTR by outputting φVO K times, and further adds and transfers the charge up to immediately before the charge transfer of the first line by L times φHO output. Has been HS
The HIFT is driven to perform the clear operation. And V-TR
The charges transferred to STLLOW by the ANS are read by the read operation.

First, S is stored in the horizontal transfer register by φVO.
Charge transfer from TLLOW is performed, and then φ
The first pixel at R (1) in the figure that outputs HO, R (2)
The second pixel is read at, and the L pixel is read at R (L).
At this time, the charges horizontally transferred by φHO are sequentially voltage-converted and amplified by the signal amplifier SAMP in synchronization with the horizontal transfer, and further the analog value is converted to the digital value by the A / D converter incorporated in the CPU 100. Is converted to. Here, since there are L pixels in the horizontal direction, the CPU 100 reads L digital values corresponding to R (1) to R (L) in FIG.

The digital value corresponding to R (1) stored in the CPU 100 is D (1), the digital value corresponding to R (2) is D (2), and the digital value corresponding to R (L) is D.
(L), SIMGBH1, SIMGB in FIG.
The digital value corresponding to each area of H2, SIMGBH3, and SIMGBH4 is SIMGBH1: D (1) to D (L / 4) SIMGBH2: D ((L / 4) +1) to D (2L /
4) SIMGBH3: D ((2L / 4) +1) to D (3L /
4) SIMGBH4: D ((3L / 4) +1) to D (L), and the luminance information is

[0101]

[Outer 1] The calculation result is obtained.

At this time, the value SIMGBH1 is the average value per pixel of the charges photoelectrically converted by the amount of light applied to the area indicated by SIMGB1 in FIG. 8. By performing this for each area, It is used as information as the brightness of each area.

FIG. 13 is a diagram showing a drive timing for obtaining the luminance information about SIMGBV2, and FIG. 14 is an S timing chart.
It is a figure which shows the drive timing for calculating | requiring the brightness | luminance information about IMGBV3.

Also at these two timings, the luminance information of the driving region DRV-B which is being accumulated can be obtained. That is, from FIG. 13, the drive called DRV-B is S
Since it is performed by IMGBV2, SI is started from this drive.
The brightness information of the MGBV2 area is obtained. However, at this time, the drive of DRV-A is performed in advance before the drive of DRV-B, and the charge on the first line before the accumulation is changed to C (1,
1) The transfer to the K / 3 + 1th line is performed by the φVO output of K / 3 times at the interval.

Then, according to the luminance information from the SIMGBV2 area of FIG. 13, SIMGB5, SIMGB6 of FIG.
The brightness information of SIMGB7 and SIMGB8 is calculated.
Similarly, in FIG. 14, the luminance information of the SIMGBV3 area is obtained, and SIMGB9, SIMGB10, and
The brightness information of SIMGB11 and SIMGB12 is calculated.

SIMGB thus shown in FIG.
The brightness information of 12 areas from 1 to SIMGB 12 is obtained.

Next, regarding the method for setting the accumulation time of 14 and the amplification characteristic of the sensor output in the CCD-EYE based on the luminance information of the eyeball image obtained by dividing the detection area of the CCD-EYE 12 into 12 in this way explain.

FIG. 15 shows a state of an eyeball image for each of the 12 divided detection areas. The detection areas SIMGB1 to 12 in FIG. 8 correspond to I to XII in FIG. Here, the regions VI and V which are the central portion A of the detection region
The brightness value of II is A ₁ , A ₂ or the brightness values of areas II, III, X, and XI which are intermediate portions B above and below the central portion A are B
₁ , ₁ , B ₂ , B ₃ , B _4, or the brightness values of the regions I, IV, V, VIII, IX, and XII which are the peripheral portions C on the left and right of the detection region are C ₁ , C ₂ , C ₃ , and C _4. , C ₅ , and C ₆ .

FIG. 16 is a flow chart of "setting of CCD accumulation time and amplification characteristic" in # 203 of the flow chart of FIG. First, block driving of the above-mentioned CCD (# 30
1) is performed, and the brightness value of each of the 12 divided areas is determined (# 302).

FIG. 17 shows the EEPROM 10 of the CPU 100.
It is an AGC table showing the accumulation time and the setting value of the amplification characteristic stored in 0a. Next, each detection area I to XII
The brightness value Be of the eyeball image is calculated by a predetermined algorithm based on the brightness value of (No. 303), and one of the AGC tables 1 to 8 of the EEPROM 100a is selected based on this Be value, and it is selected on each address. The stored accumulation time, amplification factor and amplification characteristic setting values are read (#
304). Then, when the setting is completed, it returns (# 3
05). MPU100 is C according to the read setting value
The line-of-sight detection circuit is controlled so as to accumulate CD-EYE, amplify the sensor output, and output an image signal.

Next, the A / D conversion of this image signal is performed to extract the features of the Purkinje image and the pupil portion, as already described in the flow charts # 205 and # 206 of FIG. The image signal output at this time is equal to or greater than the threshold value W for determining that the Purkinje image is the Purkinje image as shown in FIG. The accumulation time is set so that the image output level of the iris part 17 is equal to or lower than the threshold value W and the difference threshold value of the image output level between the iris part 17 and the pupil 19 is equal to or higher than V. And the amplification characteristics are set.

Here, the Purkinje image for explaining the non-linear amplification and the linear amplification of the amplification characteristics set in the AG tables 1 to 8 appears as a bright spot having a very strong light intensity due to its nature, and therefore is less affected by outside light. When shooting indoors or in the shade, the output difference between the sensor output of the iris and the Purkinje image is very large, and the sensor output of the Purkinje image is saturated.

Further, since the output of the Purkinje image itself has a very large variation, it has been difficult to detect the iris portion and the Purkinje image at the same eyeball image sensor output level at the same time.

Therefore, FIG. 18 is a characteristic diagram showing how the output of the image sensor is amplified. As shown in this figure, the sensor output is non-linearly amplified, and the Purkinje image detection threshold level for the amplified image signal is shown. The problem is solved by setting W to the point where the amplification factor change is large.

Specifically, in FIG. 18, the amplification factor is set large in the D region where the sensor output is small, for example, about 10 to 20 times, and the amplification factor is set small in the E region where the sensor output is large. However, for example, it is set to about 1 to 2 times. This makes the output of the image sensor PI.
Even if it changes from 1 to PI2, the detected image output changes only from PO1 to PO2. That is, even if the sensor output changes in the PI2 / PI1 ratio, the image output is PO2 / PO1.
It becomes a ratio and the relative fluctuation ratio becomes small. By setting the detection threshold level of the Purkinje image in the vicinity of the conversion point between the D region and the E region, if the sensor output of the Purkinje image is PI3 or more, an image signal output with little fluctuation can be obtained and a stable Purkinje image can be obtained. Can be detected. The constant chain line f shows a linear characteristic without a conversion point, and the amplification factor is always constant.

The characteristic of the AGC tables 1 to 8 is that the accumulation time, the amplification factor (Gain), and the amplification characteristic are determined as one set, which is useful for reducing the number of AGC tables. . Furthermore, the above-mentioned non-linear characteristic is switched to a linear characteristic from a certain level or more (12 or more in the embodiment) of the eyeball brightness Be. This is because the luminance of the eyeball image becomes considerably bright, and when the accumulation time is short and the amplification factor is small, the output ratio of the image signal relative to the iris portion of the Purkinje image becomes smaller than that when the luminance of the eyeball image is dark. In order to compensate for this, the characteristic is switched to a linear characteristic, and the image signal output of the Purkinje image is made relatively high to facilitate detection.

Next, an algorithm for obtaining the brightness value Be of the eyeball image (# 303) based on the brightness values of the divided areas I to XII (# 303) will be described with reference to FIG. First, the brightness values A ₁ , A
₂ , B ₁ , B ₂ , B ₃ , B ₄ , C ₁ , C ₂ , C ₃ , C
_{Of 4} , C ₅ , and C ₆ , those having extremely bright brightness are determined to be ghosts due to external light, and those having a value larger than the threshold F are excluded (# 401). Similarly, if the brightness is extremely dark, it is determined that the eyeball does not exist (the image does not exist) in the dark room, and the one smaller than the threshold G is excluded (# 402).

Next, as can be seen from FIG. 15, since the finder can be seen most easily by placing the pupil in the center when looking into the finder, the iris portion is the largest in the central portion A of the general detection area. In addition, the iris part is often hung on the middle part B. Therefore, in this embodiment, # 40
1, the brightness value average Ai of the A part excluding the brightness information excluded in # 402, the brightness value average Bi of the B part, and the brightness value average Ci of the C part are

[0119]

[Outside 2] A weighting operation is performed in which the central portion A is heavily weighted, the intermediate portion B is slightly large, and the peripheral portion C is slightly weighted (# 403). As a result, the brightness value Be of the eyeball portion is determined, and the algorithm of # 303 ends (#
404). Here, in # 401 and # 402, those larger than the threshold value F and those smaller than the threshold value G are excluded, but instead of eliminating them, the threshold values F and G are replaced as respective luminance values. Is also good.

As a result, the brightness of the eyeball centering on the iris part can be determined without erroneous brightness of the eyeball image due to external light noise on the eyeball part.

[0121]

As described above, the unit pixel composed of the photoelectric conversion means for performing the photoelectric conversion according to the brightness to be irradiated and the charge storage means for storing the photoelectrically converted charges is arranged continuously and the transfer signal is transmitted. A sensor that transfers the photoelectrically converted charge in one pixel to one of the adjacent pixels (a so-called CCD
For each pixel by driving the sensor for the information which determines the storage time of (etc.) in block units before the main storage and slowing down the transfer speed only when transferring the above information in the block area that requires it. It is required in a shorter time than when driving, and it is possible to shorten the time required for a series of storage control and accurately control the storage time of the sensor device, which improves the function of the entire system using this storage control device. There is. Further, since the time required to determine the accumulation time is short, there is an effect that it is not necessary to provide a dedicated photoelectric sensor.

Further, as compared with performing storage control using a subject brightness measuring means such as a camera, it is possible to perform dramatically more accurate storage control, and it is possible to perform accurate image processing even when applied to a visual axis detection device. effective.

[Brief description of drawings]

FIG. 1 is a layout view of a single-lens reflex camera.

FIG. 2 is a rear view of the single-lens reflex camera.

FIG. 3 is a viewfinder view.

FIG. 4 is an electric circuit diagram of the camera.

FIG. 5 is a flowchart of the operation of a camera having a visual line detection device.

FIG. 6 is a flowchart of eye gaze detection.

FIG. 7 is a block diagram of an image sensor CCD-EYE.

FIG. 8 is a divided area diagram of an image sensor CCD-EYE.

FIG. 9 is an explanatory diagram of a horizontal area of the image sensor CCD-EYE.

FIG. 10 is an explanatory diagram of a vertical area of an image sensor CCD-EYE.

FIG. 11 is a drive timing chart of the image sensor CCD-EYE.

FIG. 12 is a drive timing chart of the image sensor CCD-EYE.

FIG. 13 is a driving timing chart of the image sensor CCD-EYE.

FIG. 14 is a drive timing chart of the image sensor CCD-EYE.

FIG. 15 is a schematic diagram of an eyeball image for a detection region.

FIG. 16 is a flowchart for setting a storage time and an amplification characteristic.

FIG. 17 is an AGC table.

FIG. 18 is an amplification characteristic diagram of an image sensor output.

FIG. 19 is a flowchart showing an algorithm for obtaining eyeball luminance.

FIG. 20 is an explanatory view of the principle of the eye gaze detection method.

FIG. 21 is an output diagram of an eyeball image and an image sensor.

[Explanation of symbols]

 1 Photographic Lens 2 Main Mirror 6 Focus Detection Device 6f Image Sensor 7 Focus Plate 10 Photometric Sensor 11 Eyepiece Lens 13 Infrared Light Emitting Diode (IRED) 14 Image Sensor (CCD-EYE) 15 Eyeball 16 Cornea 17 Iris 19 Pupil 21 Superimpose LED 24 Finder LCD 25 Illumination LED 27 Mercury switch 31 Aperture 100 CPU 101 Line-of-sight detection circuit 103 Focus detection circuit 104 Signal input circuit 105 LCD drive circuit 106 LED drive circuit 107 IRED drive circuit 110 Focus adjustment circuit 200 Focusing point mark (Calibration target) 201 Distance measuring point mark (Calibration target) 203 Distance measuring point mark (Calibration target) 204 Distance measuring point mark (Calibration target)

Claims (1)

[Claims] 1. A plurality of unit pixels each including a photoelectric conversion unit for accumulating photoelectrically converted charges according to an incident light beam and a reading unit for reading out photoelectric conversion output accumulated in the photoelectric conversion unit. In the photoelectric conversion element that has the unit pixels and is continuously arranged adjacent to each other and transfers the photoelectric conversion signal output accumulated in the pixel to the adjacent pixel by one transfer signal output, the output interval of the transfer signal. And a transfer signal output varying means for varying the photoelectric conversion signal output varying means for accumulating the photoelectric conversion signal output according to the amount of light irradiated during the transfer. .