JP2014157338A - Distance measurement device, distance measurement method, and imaging apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve a problem associated with a distance measurement operation on an imaging apparatus, where the measurement of a distance is sometimes difficult depending on the direction of image separation for a distance-measuring image such as a high-frequency striped object and a dynamic object.SOLUTION: A distance measurement device, for use in measuring a distance of an object by using an output of an imaging element having a two-dimensional array of unit pixel cells constituted by a given number of plural pixels arrayed correspondingly to pupil division means, performs: obtaining, for each unit pixel cell, image separation data of an object produced by synthesizing outputs of pixels arrayed in a given direction of a two-dimensional array; performing a distance measurement computation using the image separation data to output the computation result; switching over given direction at a given cycle; obtaining image separation data of a switched-to direction in accordance with the switchover; and performing a distance measurement computation using the obtained image separation data.

Description

  The present invention relates to a distance measuring device applicable to an imaging device using an imaging element such as a CMOS image sensor, and more particularly to a distance measuring device and an imaging device having a distance measuring function using a phase difference method.

In recent years, there has been a demand for downsizing and high accuracy in imaging devices such as digital cameras.
In order to obtain an image with high focus accuracy, attention has been focused on one of them. An image acquisition pixel and an information acquisition pixel (for example, a distance measurement pixel) are provided in the image sensor, and distance measurement is performed on the same screen. There is a method for obtaining a highly accurate image. (See Patent Document 1.)

Since the information acquisition pixel in the above proposal has a structure different from that of the image acquisition pixel, it has been mainly considered that correction is performed in the same manner as a defective pixel when an image is created. However, in the future, in order to obtain a high-accuracy image, it is preferable to expand the information acquisition region over most of the image sensor and use both the information acquisition pixel and the image acquisition pixel.
For example, as shown in Patent Document 2, a configuration in which pixels in which pixels are divided into pupils is arranged on the entire surface and used for both information acquisition and image acquisition is preferable.

  Examples of using an image sensor having a composite pixel structure that can be used for both information acquisition and image acquisition will be given below.

  FIG. 22 is a conceptual diagram (top view) of an image sensor formed by pixels obtained by dividing a unit cell into pupils, so-called composite pixels. In the figure, reference numeral 1 denotes a unit cell of an image sensor, and reference numerals 1a to 1d denote pixels each having a known image pickup pixel structure including a photoelectric conversion element. Each pixel is configured to be output individually. Further, a well-known color filter of the same color is disposed on the upper surfaces of 1a to 1d. Each pixel 2 is a known microlens.

  The image sensor having the above-described configuration measures and compares the output of each or a plurality of pixels in the divided state in each unit cell in the column direction (or the row direction) during information acquisition (ranging operation), and the correlation calculation result The well-known phase difference ranging is performed to obtain the phase difference. On the other hand, at the time of image acquisition, an image is created using the output of each unit cell (in this example, four pixels of the same color under the microlens).

  An example when the image sensor described in FIG. 22 is used for the distance measuring operation will be described with reference to FIG.

  FIG. 23A is an example of using the output of the unit cell, and is an example of using the imaging device for distance measurement in the horizontal direction. When used in the horizontal direction, since the pupil division direction is used as the horizontal direction, the pixels 1a and 1b and the pixels 1c and 1d in FIG. 22 are respectively combined and output as A output (left output) and B output (right output). It is output for ranging of unit cell.

  FIG. 23B shows an A image (left image) and a B image using A output (left output) and B output (right output) output from a plurality of cells in the usage example of FIG. In this example, a right image is generated, and a correlation calculation between them is performed to obtain a shift amount (defocus amount).

  A plurality of pixels having the same configuration as in FIG. 23 (a) are arranged in the horizontal direction (specifically, because adjacent cells are unit cells of different colors, cells having the same configuration are arranged for each cell), A plurality of unit cells having the same configuration are used to generate and compare an A image (left image) and a B image (right image). By comparing the A image and the B image (correlation calculation), a focus shift amount (defocus amount) is calculated. Basically, the higher the correlation value, the more focused, and the lower the correlation value, the more out of focus.

  In the above description, distance measurement is performed based on the amount of deviation between the left and right images. However, depending on the subject (for example, a horizontal stripe-like subject), distance measurement may not be performed well with left and right image separation. Therefore, the distance separation performance can be further improved by using the data separated in the vertical direction as well as the image separation direction used in the distance measurement.

  In Patent Document 3, in order to improve the distance measurement performance, distance measurement pixels having different pupil division directions are arranged in a distributed manner so as not to concentrate on the screen.

JP 2003-244712 A JP 2007-325139 A JP 2008-017116 A

  However, even if distance measurement pixels having different pupil division directions are distributed and arranged as in Patent Document 3, the pupil division direction is fixed. Although the performance is improved, the problem of distance measurement performance cannot be avoided if the subject position is shifted. In particular, in the case of continuous shooting operations such as moving image shooting in which there is a relatively high possibility that the subject is moving, there is a high possibility that the subject position will shift.

  Japanese Patent Application Laid-Open No. 2004-228561 proposes a configuration in which the image separation direction can be switched between the horizontal direction and the vertical direction, but does not describe details about under what conditions or in what case.

  In order to solve the above-described problem, according to the present invention, the output of an image sensor having a two-dimensional array of unit pixel cells composed of a predetermined number of a plurality of pixels arranged corresponding to the pupil dividing means is used. A distance measuring device for measuring a distance of an object includes an acquisition unit that acquires image separation data of an object obtained by combining outputs of pixels arranged in a predetermined direction of a two-dimensional array for each unit pixel cell. A calculation unit that performs distance measurement using the image separation data and outputs a calculation result; and a control unit that switches a predetermined direction at a predetermined cycle, and the acquisition unit is switched according to switching by the control unit. The image separation data in the selected direction is acquired, and the calculation means performs a distance measurement calculation using the image separation data acquired by the acquisition means.

  According to the present invention, it is possible to improve distance measurement accuracy during a continuous shooting operation of a moving image or the like, and it is possible to appropriately capture an image.

1 is a block diagram of an image pickup apparatus having a distance measuring apparatus according to a first embodiment of the present invention. 1 is a schematic configuration diagram of pixels that constitute an imaging device used by an imaging apparatus according to a first embodiment of the present invention. 1 is a diagram illustrating a circuit configuration of a unit pixel cell of an image sensor used in a first embodiment of the present invention. 1 is a block diagram of an image sensor used in a first embodiment of the present invention. 1 is a configuration diagram of a readout circuit of an image sensor used in a first embodiment of the present invention. FIG. 4 is a diagram illustrating a timing chart of the driving operation of the image sensor used in the first embodiment of the present invention. FIG. 3 is a diagram schematically showing the operation of the image sensor used in the first embodiment of the present invention. The figure which shows the flowchart of operation | movement of the ranging apparatus which concerns on 1st Example of this invention. The figure which shows the usage example of the pixel output at the time of operation | movement of the ranging apparatus which concerns on 1st Example of this invention. The figure for demonstrating the calculation at the time of operation | movement of the ranging apparatus which concerns on 1st Example of this invention. The circuit block diagram of the unit pixel cell of the image pick-up element used in the 1st modification of the 1st Example of this invention. The schematic block diagram of the image pick-up element used in the 1st modification of the 1st Example of this invention. The figure which shows the timing chart of the drive operation of the image pick-up element used in the 1st modification of the 1st Example of this invention. The figure which shows the drive timing chart of the image pick-up element used in the 1st modification of the 1st Example of this invention. The figure which shows typically operation | movement of the image pick-up element used in the 1st modification of the 1st Example of this invention. The figure which shows the flowchart of operation | movement of the ranging apparatus which concerns on the 1st modification of 1st Example of this invention. The figure which shows typically operation | movement of the image pick-up element used in the 1st modification of the 1st Example of this invention. The circuit block diagram of the unit pixel cell of the image pick-up element used in the 2nd modification of the 1st Example of this invention. The schematic block diagram of the image pick-up element used in the 2nd modification of the 1st Example of this invention. The figure which shows the drive timing chart of the image pick-up element used in the 2nd modification of the 1st Example of this invention. The figure which shows the drive timing chart of the image pick-up element used in the 2nd modification of the 1st Example of this invention. The conceptual diagram (top view) when the unit pixel cell of an image sensor is divided into pupils. The usage example at the time of the ranging operation | movement of the image pick-up element which has the unit pixel cell which carried out the pupil division.

  Preferred embodiments of the present invention will be described below in detail with reference to the accompanying drawings.

  FIG. 1 is a block diagram showing the configuration of an imaging apparatus such as an electronic camera having a distance measuring apparatus according to the first embodiment of the present invention.

  In the figure, reference numeral 110 denotes a lens constituting an optical system that forms an optical image on the image sensor 114, and 111 denotes a lens control unit that controls the lens 110. Reference numeral 112 denotes a shutter (SH) that controls the exposure amount of the image sensor 114 which is a CMOS sensor or the like. Reference numeral 114 denotes an image sensor such as a CMOS sensor that converts an optical image into an electrical signal. The image sensor 114 has a configuration in which a predetermined number of pixels are separately arranged in a unit pixel cell of one microlens unit, and the image divided into pupils every two pixels from the unit pixel cell is different. An image signal can be output. As a result, an image-separated output (image separation data) for ranging operation can be obtained. Details will be described later with reference to FIGS.

  Reference numeral 116 denotes an output signal processing unit that performs processing such as OB clamping for matching the optical black level to the reference level for the analog signal output from the image sensor 114. This output signal processing unit also has an analog front end (AFE) that converts it into a digital signal, and a digital front end (DFE) that receives the digital output of each pixel and digitally processes various corrections and rearrangements. Including.

  Reference numeral 117 denotes a timing generation circuit (TG) that supplies a control signal. Reference numeral 118 denotes a memory for temporarily storing the output of the output signal processing unit 116 in order to perform subsequent ranging calculation and image processing.

  A distance measurement calculation unit 119 performs a distance measurement process based on the output signal from the image sensor 114 digitally processed by the output signal processing unit 116. The distance measurement calculation unit 119 obtains the distance of the subject from the defocus amount (image shift amount) in the direction output by separating the image from the image sensor 114, and transfers the information to the system control unit 150 described later. The ranging calculation unit 119 also performs a vertical or horizontal combining process for each pixel for performing the ranging calculation. The ranging calculation unit 119 and the system control unit 150 constitute a ranging device according to the present embodiment.

  An image processing unit 120 performs image processing such as predetermined color conversion on data from the image output digitally processed via the output signal processing unit 116. Note that the image processing unit 120 also performs a process of generating an image by outputting one pixel signal every unit pixel cell by synthesizing all the outputs of each pixel separated in the unit pixel cell of the image sensor 114. .

  A memory control unit 122 performs control such as transferring image data generated by the image processing unit 120 to a memory 130 described later. Reference numeral 128 denotes an image display unit composed of a TFT LCD.

  Reference numeral 130 denotes a memory for storing captured still images and moving images. A system control unit 150 controls the entire image processing apparatus. The system control unit 150 includes a well-known CPU and the like, and controls the operation of each unit of the imaging apparatus by executing a control program stored in a memory (not shown). For example, the control related to the operation of the distance measuring device according to the present invention and the imaging control for controlling the driving of the optical system and the imaging device are performed. Note that an actual image pickup apparatus generally includes a photometric means, a power supply control means, a switch operation unit, and the like, but since it is not a main element according to the present invention, description thereof is omitted here.

  2 to 3 show a schematic structure of a unit pixel cell and a circuit configuration of the unit pixel cell in the image sensor 114 included in the imaging apparatus according to the present embodiment.

  FIG. 2 is a schematic diagram (top view) of a unit pixel cell of an image sensor used in the imaging apparatus according to the present embodiment. In FIG. 2A, reference numeral 200 denotes an array microlens serving as pupil dividing means. Pixels 201 to 204 each having a photodiode (photoelectric conversion element) under the microlens 200 (in this embodiment, four-divided pixels). ) Is arranged.

  The pixels 201 to 204 can output signals of the respective pixels as synthesized signals by appropriately using an appropriate driving method. FIG. 2B illustrates the pixels 201 and 202 and the pixels 203 and 204. It is a conceptual diagram which shows synthesizing each output and outputting. By performing output in such a combination, it is possible to acquire two signals obtained by image division in the left and right (horizontal direction) from the signal in the unit pixel cell.

  Similarly, FIG. 2C is a conceptual diagram showing that the outputs of the pixels 201 and 203 and the pixels 202 and 204 are combined and output. By performing output in a combination as shown in the drawing, two signals obtained by image division in the vertical direction (vertical direction) can be acquired from the signal in the unit pixel cell.

  FIG. 2D is a conceptual diagram showing that all the outputs of the pixels 201 to 204 are combined and output. By outputting in such a combination, all signals in the unit pixel cell are output. The signal of one pixel added can be acquired. The combining process in FIG. 2D is a process of adding four individual output pixels of each pixel in FIG. 2A, or the two image-divided outputs in FIG. 2B or FIG. Furthermore, the process which adds and produces | generates a synthesized signal may be sufficient.

  FIG. 3 shows a circuit configuration of the unit pixel cell described in FIG. Since each of the pixels 201 to 204 has the same structure, only the configuration of the pixel 201 is illustrated and described here. In the figure, the same parts as those in FIG. 2 are denoted by the same reference numerals.

  Reference numeral 301 denotes a photoelectric conversion unit made of a photodiode or the like. 331 is a transfer transistor for transferring the charge of the photodiode 301, 333 is a floating diffusion (FD) as a predetermined part for transferring the charge, 335 is an amplifying unit for outputting a signal corresponding to the charge amount of the FD 333 It is an amplification transistor. Further, reference numeral 337 denotes a selection transistor for outputting the charge of the FD 333 from the pixel portion to a subsequent circuit portion, and reference numeral 339 denotes a reset transistor for resetting the charge of the FD 333.

  Reference numeral 301 denotes a vertical output line for outputting the charge stored in the FD 333 to the subsequent circuit via the amplification transistor 335 when the selection transistor 337 is turned on. One column of pixels arranged vertically (pixels 201 and 202, pixels 203 and 204) is connected to a common output line (301-1, 301-2).

  FIG. 4 is a block diagram showing a schematic configuration of an image sensor in which a plurality of unit pixel cells shown in FIG. 3 are arranged. A pixel array in which a plurality of unit pixel cells 200 described in FIG. The drive unit is configured to read out signals. In the figure, the same parts as those in FIGS. 2 and 3 are denoted by the same reference numerals.

  Reference numeral 402 denotes a readout circuit for reading out an electrical signal output from each pixel via the vertical output line 301 (301-1, 301-2, etc.) shown in FIG. The configuration of the reading circuit 402 will be described later with reference to FIG.

  Reference numerals 503 and 404 respectively denote MOS transistors that output optical signal components and noise components input from the pixels via the readout circuit 402 to the differential amplifier 405 at the subsequent stage, and reference numeral 405 receives a signal from the MOS transistor and outputs a difference. This is a differential amplifier.

  Reference numeral 406 denotes a horizontal shift register that controls ON / OFF of the MOS transistors 403 and 404 of each unit cell and sends a signal to the differential amplifier 405. A vertical shift register 407 performs charge accumulation / transfer control of each pixel. Reference numeral 408 denotes a constant current source for controlling the current of the vertical output line 301.

  FIG. 5 shows a circuit configuration of the readout circuit 402 in FIG. 4, and shows only a circuit configuration of a readout block of pixel signals output to one vertical output line 301. The pixel signal read from the unit pixel cell to each vertical signal output line 301 is read to the differential amplifier 405 through the corresponding read block. The same parts as those in FIG. 3 or FIG.

  In the figure, reference numeral 501 denotes a clamp capacitor having one end connected to the vertical output line 301 (301-1 or 301-2) and the other connected to the inverting input side of the operational amplifier 502. Reference numeral 502 denotes an operational amplifier, and reference numeral 503 denotes a feedback capacitor for determining the amplification degree of the inverting amplifier 502 based on the ratio to the clamp capacitor 501. Reference numeral 504 denotes a clamp switch for clamping operation. Reference numerals 505 and 506 denote transfer switches for transferring the optical signal component and the dark signal component to holding capacitors 507 and 508, which will be described later, through the operational amplifier 502, respectively.

  Reference numerals 507 and 508 denote holding capacitors for holding the optical signal component and the dark signal component via the operational amplifier 502, respectively. The signals transferred to the holding capacitors 507 and 508 are output to the differential amplifier 405 via the MOS transistors 403 and 404 described with reference to FIG.

  FIG. 6 is a diagram illustrating a timing chart of the driving operation of the image sensor 114 used by the imaging apparatus according to the present embodiment. Each drive signal is supplied from the timing generation circuit 117 under the control of the system control unit 150.

  Prior to reading of the optical signal charge from the photodiode 301, the gate line φRES (the first row is φRES1-1) of the reset transistor 339 is set to the high level. As a result, the gate of the amplification transistor 335 is reset to the reset power supply voltage. After the gate line φRES of the reset transistor 339 returns to the low level and the gate line PC0R of the clamp switch 504 becomes the high level, the gate line φSEL of the selection transistor 337 (the first row is φSEL1-1) becomes the high level. . As a result, a reset signal (noise signal) on which reset noise is superimposed is read out to the vertical output line 301 and clamped to the clamp capacitors 501 in each column.

  Next, after the gate line PC0R of the clamp switch 504 returns to the low level, the gate line PTN of the noise signal side transfer switch becomes the high level, and the reset signal is held in the dark signal holding capacitor 508 provided in each column. The Next, after the gate line PTS of the pixel signal side transfer switch is set to the high level, the gate line φTX (φTX1-1 in the first row) of the transfer transistor 331 is set to the high level. As a result, the optical signal charge of the photodiode 301 is transferred to the gate of the amplification transistor 335, and at the same time, the optical signal is read out to the vertical output line 301.

  Next, after the gate line φTX of the transfer transistor 331 returns to the low level, the gate line PTS of the pixel signal side transfer switch becomes the low level. As a result, the amount of change (optical signal) from the reset signal is read out to the optical signal component holding capacitor 507 provided in each column. With the operations so far, the optical signal and dark signal of the photoelectric conversion unit 301 connected to the first row are held in the holding capacitors 507 to 508 connected to the respective columns.

  Thereafter, the horizontal shift register 406 is driven by the signal CSL, and the horizontal transfer switch gates in each column are sequentially set to the high level. The voltages held in the holding capacitors 507 to 508 are sequentially read out to the capacitors Chs and Chn, subjected to differential processing by the differential amplifier 405, and sequentially output. Capacitors Chs and Chn are reset to reset voltages VCHRS and VCHRN by a reset switch between signal readings of each column. Thus, reading of the pixels connected to the first row of the unit pixel cell is completed.

  Similarly, the signals from the photoelectric conversion units connected to the second and subsequent rows are sequentially read out by signals from the vertical shift register 407, and all pixels (signals accumulated in all the photoelectric conversion units) are read out. Complete. That is, in this embodiment in which the unit pixel cell has a horizontal / vertical two-pixel configuration, reading of one horizontal line (1H) of the unit pixel cell is completed when reading of two vertical rows is completed.

  As described above, the imaging device used in the imaging apparatus according to the present embodiment has a configuration in which two pixels (pixels 201 to 204) are arranged in the horizontal direction and in the vertical direction for each unit pixel cell. ing. In this embodiment, the electric charges accumulated in the four photoelectric conversion units are read out and stored in the memory 118 via the output signal processing unit 116 at the subsequent stage. The ranging calculation unit 119 performs ranging calculation using the output stored in the memory 118. At this time, the ranging calculation is performed after performing necessary pixel addition processing.

  In addition, the image processing circuit 120 performs a process of generating an image using an output obtained by synthesizing the outputs of the four pixels of the unit pixel cell using the output stored in the memory 118 as a pixel signal.

  FIG. 7 is a diagram schematically showing driving according to the timing chart of FIG. 6, and shows the elapsed time when accumulation-reading of each row of the two-dimensional array of unit pixel cells is performed from the upper part to the lower part of the screen. It is visually represented.

  A horizontal line in FIG. 7 indicates an accumulation-read operation per unit pixel cell. The outputs of the pixels 201 and 203 are read in the period h1 on one line, and the outputs of the pixels 202 and 204 are read in the period h2. Reading out. This operation is sequentially repeated to complete readout for the number of unit pixel cells (1 V) in the vertical direction.

  FIG. 8 is a diagram illustrating a flowchart of the distance measuring operation in the imaging apparatus according to the present embodiment. This operation is started when a continuous imaging operation such as a moving image is instructed by a shooting start switch, a power switch, etc. (not shown). In practice, the imaging operation described with reference to FIG. 6 is repeatedly performed, and this operation is started in a situation where the captured data is sequentially sent to the memory 118. The operation in a certain case is omitted because the explanation becomes complicated. In parallel with this operation, the image processing circuit 120 performs all pixel addition processing and the like for image generation. However, since it is not directly related to the distance measuring operation of the present invention, detailed description here. Will be omitted.

  In step S800, the system control unit 150 resets a counter FC (not shown) and determines whether it is the first frame of the imaging operation. In step S801, the distance measuring unit 119 checks the count value of the counter FC. If it is an even number, the process proceeds to step S802, and if it is an odd number, the process proceeds to step S803.

  In step S <b> 802, the ranging calculation unit 119 selects pixels (pixels 201 and 202, pixels 203 and 204) arranged in the vertical direction for each unit pixel cell in the pixel data stored in the memory 118. Add the outputs. As a result, the calculation data as the A image (pixel 201 + 202) and the B image (pixel 203 + 204) for distance measurement are acquired (see FIG. 2B).

  In step S803, the outputs of the pixels arranged in the horizontal direction (pixels 201 and 203, pixels 202 and 204) are added for each unit pixel in the pixel data stored in the memory 118. As a result, calculation data is obtained as an A image (pixel 201 + 203) and a B image (pixel 202 + 204) for distance measurement (see FIG. 2C).

  In step S804, a well-known distance measurement calculation such as a correlation calculation is performed using the calculation data (A image / B image) acquired in step S802 or step S803. A detailed example of the distance measurement calculation will be described later with reference to FIGS. 9 and 10.

  In step S805, the distance measurement calculation result obtained in step S804 is transferred to the system control unit 150. The system control unit 150 performs focus adjustment by driving the lens 110 through the lens control unit 111 based on the distance measurement calculation result. As described above, the distance measurement calculation result is also used for the feedback operation of the system. Regarding focus driving and the like, it is ideal to determine the target focus amount by combining the vertical direction addition (left and right image separation) in step S802 and the horizontal direction addition (upper and lower image separation) data in step S802 and drive the lens. Is.

  In step S806, the system control unit 150 determines whether or not the end of the shooting operation has been instructed by a shooting start switch, a power switch, or the like (not shown) when the distance measurement data transfer for one frame is completed. As a result, if the end operation has not been performed, the process proceeds to step S807, the counter FC (not shown) that has started counting in step S800 is incremented (FC = FC + 1), and the process returns to step S801. If an end operation has been performed in step S806, the distance measuring operation sequence ends.

  An example of use of the image sensor in the distance measuring operation described in steps S802 to S804 will be described with reference to FIG. In the figure, for convenience of explanation, only one horizontal or vertical line is shown. However, the present embodiment is not limited to this, and a system that performs distance measurement using a plurality of lines (ideally, the entire surface of the image sensor). It is desirable that

  FIG. 9A-1 schematically illustrates a case in which when a chart including two bright lines is captured in the dark, pixels are added in the vertical direction in step S802 to perform distance measurement in the horizontal direction. FIG. In the case of distance measurement in the horizontal direction, A output (left output) and B output (right output) are used as distance measurement outputs of the unit cell. As shown in the figure, the distance measurement pattern is a subject having a high contrast in the horizontal direction, and therefore it is possible to perform distance measurement with high accuracy.

  FIG. 9A-2 illustrates a case where the distance is measured in the vertical direction by adding pixels in the horizontal direction in step S803 when photographing a chart including two bright lines in the dark. FIG. In the case of distance measurement in the horizontal direction, the A output (upper output) and the B output (lower output) are used as the unit cell distance measurement outputs. As shown in the figure, this distance measurement pattern is a subject with high contrast in the vertical direction, and therefore it is possible to perform distance measurement with high accuracy.

  In FIGS. 9A-1 and 9A-2, the pixels used for distance measurement are shown every other pixel on the assumption that they are the same color.

  FIG. 9 (b) acquires A output and B output output in the format of FIG. 9 (a-1) or FIG. 9 (a-2) from a plurality of cells, and A image (left image or upper image) and An example is shown in which a B image (right image or lower image) is generated and correlation calculation is performed to obtain a shift amount (defocus amount). By comparing the A image and the B image (correlation calculation), a focus shift amount (defocus amount) is calculated. Basically, the higher the correlation value, the more focused, and the lower the correlation value, the more out of focus.

  FIG. 9B-1 shows an A image and a B image that are out of focus, and the A image (pixel 201 + 202) and the B image (pixel 203 + 204) do not match. Therefore, FIG. 9B-2 showing the degree of correlation coincidence (corresponding to the difference between the A image and the B image) is not constant, and the output of the shifted portion is high.

  On the other hand, FIG. 9B-3 shows an A image and a B image that are substantially in focus, and both are substantially coincident. Therefore, FIG. 5 (b-4) showing the correlation coincidence (corresponding to the difference between the A image and the B image) is almost constant.

  FIG. 10 is a diagram illustrating an example of a method for calculating the correlation value d by the correlation calculation. As in FIG. 9A, this figure shows the output when a chart including two bright lines in the dark is observed.

  FIG. 10A shows a state in which the A image and the B image at the time of observation are shifted. That is, the focus deviation amount = the defocus amount is large. A correlation value d (corresponding to the defocus amount) is obtained by calculating a correlation between the output values of the two images at the time of observation.

で求められる。ここでｍｉｎ（Ａ，Ｂ）はＡ、Ｂの小さい方の値のことであり、ｎはＡ像およびＢ像の画素数（生成に使用された複数の単位画素セルの数）である。まず、このＸ０を計算する。 Specifically, the output data of the A image signal is A [1] to A [n], the output data of the B image signal is B [1] to B [n], and the correlation amount X0 is

Is required. Here, min (A, B) is the smaller value of A and B, and n is the number of pixels of the A and B images (the number of unit pixel cells used for generation). First, this X0 is calculated.

で求められる。このような手順でＡ像を１ビット（１画素）ずつシフトした相関量を繰り返し計算し、相関量が最も大きくなるＸｍａｘを求める。（図１０（ｃ）） Next, as shown in FIG. 10B, a correlation amount X1 between data obtained by shifting the A image signal by 1 bit (one pixel) of the signal voltage and data of the B image signal is calculated.
This X1 is

Is required. In such a procedure, the correlation amount obtained by shifting the A image by 1 bit (one pixel) is repeatedly calculated to obtain Xmax that maximizes the correlation amount. (Fig. 10 (c))

  Since the correlation amount Xmax corresponds to a state in which the two-image coincidence between the A image and the B image is high (position closest to the in-focus state) as shown in FIG. 10C, the shift amount up to the correlation amount Xmax is represented by the correlation value. It is stored as d (corresponding to the defocus amount) and used for the distance measuring operation.

  The above is an example of the correlation calculation (MIN algorithm). The correlation calculation described here is merely an example, and there is no problem even if the correlation calculation is performed by another method.

  As described above, in this embodiment, the distance measurement operation based on vertical pixel addition (horizontal image separation distance measurement) and the distance measurement operation based on horizontal pixel addition (vertical image separation distance measurement) are captured. This is done according to the frame (specifically, alternately). Thereby, even when there is a subject that is not good at the conventional distance measuring operation, it is possible to perform distance measurement with higher accuracy.

  In this embodiment, vertical addition and horizontal addition are switched for each frame. However, the present invention is not limited to this. For example, at least two or more frame periods such as horizontal addition performed once in 60 frames. Therefore, there is no problem even if the configuration is switched appropriately.

[Modification]

(Modification 1)
In the first embodiment of the present invention, the configuration in which all the outputs of the pixels 201 to 2304 in the unit pixel cell 200 are individually acquired and the ranging calculation processing is performed has been described. However, the processing / circuit configuration of each frame In consideration of the above, it is effective to perform addition within the image sensor. Therefore, as a first modification of the first embodiment, an example in which a distance measurement operation is performed by obtaining a pixel output obtained by adding pixel outputs in an image sensor will be described. Note that the basic configuration of the imaging apparatus is the same as that of the first embodiment (FIG. 1), and a description thereof will be omitted here.

  FIG. 11 shows a circuit configuration of a unit pixel cell of an image sensor used in the image pickup apparatus according to the first modification. 2 and 3 are denoted by the same reference numerals.

  The structure of each pixel 201 and 202 will be described with reference to FIG. Note that the pixels 203 and 204 have the same structure as the pixels 201 and 202, and thus description thereof is omitted here. Reference numerals 301 and 302 denote photoelectric conversion units corresponding to the pixels 201 and 202. Reference numerals 331 and 1132 denote transfer transistors for transferring charges of the photodiodes 301 and 1102, and 333 denotes a floating diffusion (FD) which is a predetermined portion for transferring charges, and is used in common by the two pixels 201 and 202. It is a part. Reference numeral 335 denotes an amplification transistor that is an amplification unit that outputs a signal corresponding to the amount of charge of the FD 333, 337 is a selection transistor that outputs the charge of the FD 333 from the pixel unit to the circuit unit at the subsequent stage, and 339 resets the charge of the FD 333 This is a reset transistor. The amplification transistor 335, the selection transistor 337, and the reset transistor 339 are also common parts of the two pixels.

  301-1 (301-2 in the case of the pixels 203 and 204) is a vertical for outputting the electric charge stored in the FD 333 to the subsequent circuit via the amplification transistor 335 when the selection transistor 337 is turned on. Output line. A row of pixels arranged in the vertical direction (pixels 201 and 202, pixels 203 and 204) are connected to a common output line (301-1, 301-2).

  FIG. 12 is a schematic configuration diagram of an image sensor in which a plurality of unit pixel cells shown in FIG. 11 are arranged. A plurality of unit pixel cells 200 described in FIG. 11 are two-dimensionally arranged. Since this configuration is basically the same as FIG. 4 described in the first embodiment, the same parts as those in FIG. 4 are denoted by the same reference numerals, and the description thereof is omitted. Here, only the parts different from the first embodiment will be described.

  1211 and 1212 in FIG. 12 are connection switches for connecting the holding capacitors 507 and 508 (see FIG. 5) in the readout circuit 402 in the horizontal direction in the unit pixel cell. By turning on this connection switch, it is possible to add the light signal and dark signal of the pixels 201 and 203 adjacent in the horizontal direction.

  13 and 14 are timing charts of the driving operation of the image sensor included in the imaging apparatus according to the first modification. Since this is basically the same as the timing chart described with reference to FIG.

  FIG. 13 is a timing chart of the driving operation when the pixels in the vertical direction in the unit pixel cell are added and read. Reset of amplification transistor 335 (φRESx is high level), clamp to clamp capacitor 501 of each column (φRES is low level, PC0R is high, φSEL is high), dark signal holding capacity transfer (PC0R is low, PTN is high) The operations up to are the same as those in FIG.

  At the time of the next optical signal transfer, the gate line PTS of the pixel signal side transfer switch is set to the high level, and then the gate lines φTXx−1, φTXx-2 of the transfer transistors 331, 1322 (TX1-1, φTX1- in the first row). 2) is simultaneously set to the high level. As a result, the optical signal charges of the photodiodes 301 and 1102 are simultaneously supplied to the FD 333, and an addition operation is performed on the FD 333.

  The subsequent operation is the same as the movement described in FIG. However, since the addition in the unit pixel cell in the vertical direction has been completed, each row is read in the same operation in FIG. 6, whereas in the operation in FIG. 13, reading is performed while thinning out every other row in the vertical direction. . As a result, the readout time for one frame can be shortened compared to the first embodiment in which all pixels are read out.

  FIG. 14 is a timing chart of the driving operation when the horizontal pixels in the unit pixel cell are added and read. The operations up to resetting the amplification transistor 335 (φRES is high level), clamping to the clamp capacitor 501 of each column (φRES is low, PC0R is high level, and φSEL is high level) are the same as those in FIG.

  Subsequently, PSHORT is controlled to a high level when the dark signal holding capacity is transferred (PC0R is low and PTN is high), and the holding capacity of the readout circuit adjacent in the horizontal direction in the unit pixel cell is connected. As a result, the dark signal component obtained by adding the pixels adjacent in the horizontal direction is held on the holding capacitor 508.

  Further, the gate line PTS of the pixel signal side transfer switch is set high, and the gate line φTX1-1 of the transfer transistor 331 is set high. As a result, the optical signal charges of the photodiodes 301 and 303 (and 1102 and 1104) are held on the holding capacitor 507 as the added signal component.

  After that, by returning PSHORT to low, signals that have been added and averaged in the storage capacitors 507 and 508 remain. For this reason, since the addition in the unit pixel cell in the horizontal direction is performed by the storage capacitor, the reading may be performed by thinning out one column even in the horizontal addition reading. That is, reading is sequentially performed by driving every other column by the signal CSL. As a result, the readout time for one frame can be shortened compared to the first embodiment in which all pixels are read out.

  By the way, when a continuous shooting operation such as a moving image is assumed, control at a stable frame rate is necessary, but the time for reading out the entire screen does not necessarily match an appropriate frame rate. Therefore, it is common to obtain an appropriate frame rate by shortening the reading time of one frame with respect to the frame rate and using the remaining time as the adjustment time.

  An example in which various controls are performed using time for adjusting the frame rate will be described with reference to FIGS.

  FIG. 15 is a diagram schematically showing the flow of the driving operation in FIGS. 13 and 14, and visually shows the elapsed time when each row performs accumulation-reading from the upper part of the screen toward the lower part. It shows.

  When considering a continuous shooting operation such as a moving image, it is necessary to control at a stable frame rate. Therefore, in the present modification, as shown in the figure, the time for one frame is set to T0 (for example, 60 fps), and control for realizing a stable frame rate is performed using the actual read time for one frame + adjustment time.

  FIG. 13A shows an example of the driving operation at the time of the vertical addition reading shown in FIG. 13. In the reading of one row (1H) of unit pixel cells, first, the pixel 201 and the pixel 201 and h1 ′ on one line are read. The pixel 203 is read as an output added with the pixel 202 and the pixel 204, respectively. This h1 'is the same as h1 in FIG. Subsequently, since the readout of the pixel 202 and the pixel 204 has already been performed in the state of addition at h1 ′, the operation corresponding to the readout of h2 in FIG. 7 becomes unnecessary. Therefore, control for thinning out the reading operation of the pixels 202 and 204 is performed (thinning control is defined as h2 ').

  FIG. 15B shows an example of the horizontal addition readout driving operation shown in FIG. 14. The readout of one row (1H) of unit pixel cells is first performed at h1 ″ on one line, and the pixel 201 is a pixel. As shown in Fig. 14, the readout time can be shortened by not outputting the control signal CSLx-2 as shown in Fig. 14. This h1 "is 1 of h1 in Fig. 7. / 2.

  Subsequently, at h2 ″ on one line, the pixel 202 is read as an output added to the pixel 204. Here again, similarly to h1 ″, the control signal CSLx-2 is not output, thereby shortening the readout time. be able to. This h2 ″ is 1/2 of h2 in FIG.

As described above, there may be a difference in the adjustment time for one frame period T0 due to the difference between the vertical addition and horizontal addition readout methods. In this example, the adjustment time T1 during vertical addition and the horizontal addition are different. The relationship of the adjustment time T2 at the time is T1 <T2. It should be noted that this relationship is merely an example, and that T1> T2 and T1 = T2 may be possible depending on the imaging system and method. For example, T1 = T2 when the configuration does not require time for performing thinning control during vertical addition.
Here, an example in which the time difference of T1 <T2 is used effectively and different control is performed will be described with reference to FIGS.

  FIG. 16 is a flowchart of the imaging / ranging operation of the imaging apparatus according to the first modification, and FIG. 17 is a diagram schematically illustrating the driving operation of the imaging device according to the first modification. Note that FIG. 17 is a schematic diagram when the vertical addition reading of FIG. 15A and the horizontal addition reading of FIG. 15B are alternately performed. Therefore, the individual detailed movements are the same as those in FIG. 15, and the imaging / ranging operation according to the flowchart of FIG. 16 will be described below.

  The basic operation of the imaging / ranging operation is started when a continuous imaging operation of a moving image or the like is instructed by a shooting start switch / power switch (not shown).

  In step S1600, the system control unit 150 resets a counter FC (not shown) and determines whether it is the first frame of the imaging operation. In step S1601, the distance measurement calculation unit 119 checks the count value of the counter FC. If it is an even number, the process proceeds to step S202. If it is an odd number, the process proceeds to step S1603.

  In step S1602, the driving of the image sensor is controlled, and as shown in FIG. 13, the pixels (pixels 201 and 202, pixels 203 and 204) arranged in the vertical direction for each unit pixel cell are added and read. The pixel signals read by addition are output as A images (pixels 201 + 202) and B images (pixels 203 + 204) for distance measurement. (Operation of the vertical addition region in FIG. 17; see FIG. 2B.)

  In step S1603, using the output data (A image / B image) output in step S1602, a known distance measurement calculation such as a correlation calculation is performed. The distance measurement calculation has already been described with reference to FIGS. 9 and 10 in the first embodiment, and therefore detailed description thereof will be omitted.

  In step S <b> 1604, the distance measurement calculation unit 119 transfers the distance measurement calculation result obtained in step S <b> 1603 to the system control circuit 150. Steps S1603 and S1604 are performed within the shorter adjustment time T1 of the adjustment times T1 and T2 in FIG.

  In step S1605, the drive of the image sensor is added and output from the pixels (pixels 201 and 203, pixels 202 and 204) arranged in the horizontal direction for each unit pixel, as shown in FIG. (Operation of the horizontal addition area in FIG. 17; see FIG. 2C.)

  In step S1606, using the output data (A image / B image) output in step S1605, a well-known distance measurement calculation such as a correlation calculation is performed in the same manner as in step 1604.

  In step S 1607, the distance measurement calculation result obtained in step S 1606 is transferred to the system control circuit 150. In step S1608, based on the distance measurement calculation result transferred to the system control circuit 150 in steps S1606 and S1607, focus adjustment for performing focus driving of the lens 110 is performed via the lens control unit 111.

  Steps S1606 to S1608 are performed within the period of adjustment time T2 in FIG.

  In step S1609, when the driving of one frame is completed, the system control unit 150 determines whether or not the photographing operation is instructed by a photographing start switch, a power switch, or the like (not shown). As a result, if the end operation has not been performed, the process proceeds to step S1610, the counter FC (not shown) that has started counting in step S1600 is incremented (FC = FC + 1), and the process returns to step S1601. If an end operation has been performed in step S1609, the distance measuring operation is ended.

  As described above, in the first modification of the first embodiment of the present invention, the readout speed can be increased by using the added pixel output in the image sensor, and the distance measurement The operation can be performed quickly. Further, by setting an adjustment time suitable for each of horizontal addition reading and vertical addition reading, a stable frame rate can be obtained even when the reading drive of the image sensor is switched. Furthermore, it is possible to drive efficiently by changing the calculation contents / driving contents using the difference in adjustment time between horizontal addition reading and vertical addition reading.

  In the first modification, it is assumed that the adjustment time varies depending on the addition operation. However, the present invention is not limited to this, and for example, the adjustment time is stable. Different control may be performed for even frames and odd frames. Of course, there is no problem even if the control during the adjustment time is controlled as the same operation every time.

(Modification 2 of the first embodiment)
In the first modification, an example of the addition method in the image sensor has been described. However, since the vertical and horizontal addition configurations are different, each drive must be switched significantly, and the frame rate is changed. The control such as the adjustment becomes complicated. For this reason, as a second modification of the first embodiment, an example of an addition readout configuration in which there is no difference in frame rate adjustment time depending on the addition direction will be described. Note that the basic configuration of the imaging apparatus is the same as that of the first embodiment (FIG. 1), and a description thereof will be omitted here.

  FIG. 18 shows a circuit configuration of a unit pixel cell of the image sensor in the second modification. The same parts as those in FIG. 3 are denoted by the same reference numerals.

  In FIG. 18, reference numerals 301, 1802, 1803, and 1804 denote photoelectric conversion units, and a plurality of photodiodes having the same characteristics (four in this embodiment) are arranged two-dimensionally. Reference numerals 331 and 1832 denote transfer transistors that transfer charges of the photodiodes 301, 1802, 1803, and 1804, and reference numerals 333 and 1834 denote floating diffusions (FD) which are predetermined portions for charge transfer. Reference numerals 335 and 336 denote amplification transistors that output signals in accordance with the charge amounts of the FDs 333 and 1834. Select transistor. Reference numerals 339 and 1840 denote reset transistors for resetting the electric charges of the FDs 333 and 1834.

  Reference numeral 301-1 denotes a first vertical output line for outputting the charge stored in the FD 333 to the subsequent circuit via the amplification transistor 335 when the selection transistor 337 is turned on. Reference numeral 301-2 denotes a second vertical output line for outputting the charge stored in the FD 1834 to the subsequent circuit via the amplification transistor 1836 when the selection transistor 1838 is turned on.

  Reference numerals 1811, 1812, 1821, and 1822 denote connection transistors for connecting the photodiodes. When the connection transistor 1811 is on, the photodiodes 301 and 1802 are connected. On the other hand, when the connection transistor 1812 is on, the photodiodes 803 and 1804 are connected. When the connection transistor 1821 is on, the photodiodes 301 and 1803 are connected. When the connection transistor 1822 is on, the photodiodes 1802 and 1804 are connected.

  The connection transistors 1811 and 1812 are controlled at a common timing by a control signal φADDV. The connection transistors 1821 and 1822 are controlled at a common timing by the control signal φADDH. That is, when the control signal φADDV is on and the control signal φADDH is off, photodiodes arranged in the vertical direction (photodiodes 301 and 1802 and photodiodes 1803 and 1804) are connected in the layout. For this reason, the outputs of the two photodiodes in the vertical direction are combined, and an optical signal output separated in the horizontal direction can be obtained. When the control signal φADDH is on and the control signal φADDV is off, the photodiodes (photodiodes 301 and 1803 and photodiodes 1802 and 1804) arranged in the horizontal direction are connected in the layout. Therefore, the outputs of the two horizontal photodiodes can be combined to obtain an optical signal output separated in the vertical direction.

  FIG. 19 is a schematic block diagram of an imaging device in which a plurality of unit pixel cells shown in FIG. 18 are arranged, and a plurality of unit pixel cells 200 described in FIG. 18 are two-dimensionally arranged. The configuration is basically the same as that of the first embodiment of the present invention described with reference to FIG. However, in this modification, a control signal φADDH for controlling addition reading of horizontal pixels and a control signal φADDV for controlling addition reading of vertical pixels are connected to each unit pixel cell.

  FIG. 20 is a diagram illustrating a timing chart of the pixel addition reading operation in the vertical direction of the image sensor according to the second modification.

  The amplification transistors 335 and 1836 are reset (φRES is high level), and then φADDV (φADDV1 in the first row) is set to high. By control, the photodiodes 301 and 1802 or the photodiodes 1803 and 1804 are connected via the connection transistor 1811 or 1812. Thereafter, the operations of the clamp capacitor 501 of each column (φRES is low, PC0R is high level, φSEL is high level), and the dark signal holding capacitor transfer (PC0R is low, PTN is high) are the same as in FIG. is there. In addition, the storage capacity transfer of the optical signal (PTN is low, PTS is high, and φTX is high) and the reading operation are the same as those in FIG.

  FIG. 21 is a timing chart of an example of an addition reading operation of pixels in the horizontal direction of the image sensor according to the second modification. Since this operation is a movement in which φADDV of the operation for adding and reading pixels in the vertical direction described with reference to FIG. 20 is changed to φADDH (φADDH1 in the first row), detailed description thereof is omitted.

  By adopting the configuration as shown in FIG. 19 to FIG. 21, pixel addition within the image sensor can be performed in the same control time regardless of the addition direction, so that control can be prevented from becoming complicated. . Of course, since it is necessary to control the frame rate stably, it is also possible to perform distance measurement processing and lens drive (focus drive) using the time of one frame and the adjustment time caused by the difference in frame rate. The invention is intended.

  In the description of the present invention, since it is assumed that a general rolling shutter drive is used, a time-series shift in the vertical direction may occur (see, for example, FIG. 7). However, regarding this, on the premise that the addition is performed in the vertical direction in the unit pixel cell, it is assumed that there is no problem in practical use.

  Of course, ideally, use a configuration that corrects time-series deviations by vertical image processing, or uses a configuration that has a memory that can temporarily store accumulated charges in the image sensor, and transfers them sequentially in a batch. It is desirable to correct / suppress time-series deviations, such as by adopting a method of reading.

  According to the above-described embodiments of the present invention and the modifications thereof, it is possible to perform a distance measuring operation with high accuracy regardless of whether the subject has a horizontal or vertical direction, such as a moving image. It is possible to appropriately adjust the focus in continuous shooting.

[Other embodiments]
Each means constituting the imaging apparatus and each step of the distance measuring operation in the embodiment of the present invention described above can be realized by operating a program stored in a RAM or a ROM of a computer. This program and a computer-readable storage medium storing the program are included in the present invention.

  In addition, the present invention can be implemented as, for example, a system, apparatus, method, program, storage medium, or the like. Specifically, the present invention may be applied to a system including a plurality of devices. The present invention may be applied to an apparatus composed of a single device.

  In the present invention, a software program that realizes the functions of the above-described embodiments (in the embodiment, a program corresponding to the flowcharts shown in FIGS. 8 and 16) may be directly or remotely supplied to the system or apparatus. Including. This includes the case where the system or the computer of the apparatus is also achieved by reading and executing the supplied program code.

  Accordingly, since the functions of the present invention are implemented by computer, the program code installed in the computer also implements the present invention. In other words, the present invention includes a computer program itself for realizing the functional processing of the present invention. In that case, as long as it has the function of a program, it may be in the form of object code, a program executed by an interpreter, script data supplied to the OS, and the like.

  Examples of the storage medium for supplying the program include a flexible disk, a hard disk, an optical disk, and a magneto-optical disk. Further, there are MO, CD-ROM, CD-R, CD-RW, magnetic tape, nonvolatile memory card, ROM, DVD (DVD-ROM, DVD-R) and the like.

  As another program supply method, there is a method of connecting to a homepage on the Internet using a browser of a client computer. It can also be supplied by downloading the computer program itself of the present invention or a compressed file including an automatic installation function from a homepage to a storage medium such as a hard disk.

  It can also be realized by dividing the program code constituting the program of the present invention into a plurality of files and downloading each file from a different homepage. That is, a WWW server that allows a plurality of users to download a program file for realizing the functional processing of the present invention on a computer is also included in the present invention.

  As another method, the program of the present invention is encrypted, stored in a storage medium such as a CD-ROM, distributed to users, and encrypted from a homepage via the Internet to users who have cleared predetermined conditions. Download the key information to be solved. It is also possible to execute the encrypted program by using the key information and install the program on a computer.

  Further, the functions of the above-described embodiments are realized by the computer executing the read program. Furthermore, based on the instructions of the program, an OS or the like running on the computer performs part or all of the actual processing, and the functions of the above-described embodiments can be realized by the processing.

  As another method, a program read from a storage medium is first written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer. Then, based on the instructions of the program, the CPU or the like provided in the function expansion board or function expansion unit performs part or all of the actual processing, and the functions of the above-described embodiments are realized by the processing.

  The above-described embodiments are merely examples of implementation in carrying out the present invention, and the technical scope of the present invention should not be construed as being limited thereto. That is, the present invention can be implemented in various forms without departing from the technical idea or the main features thereof.

Claims (17)

In a distance measuring device for measuring a subject using an output of an image sensor having a two-dimensional array of unit pixel cells composed of a predetermined number of pixels arranged corresponding to pupil dividing means,
Obtaining means for obtaining image separation data of the subject obtained by synthesizing outputs of pixels arranged in a predetermined direction of the two-dimensional array for each unit pixel cell;
A calculation means for performing a ranging calculation using the image separation data and outputting a calculation result;
Control means for switching the predetermined direction at a predetermined period;
With
The acquisition unit acquires image separation data in the switched direction according to switching by the control unit, and the calculation unit performs distance measurement using the image separation data acquired by the acquisition unit. Ranging device.   The control unit switches the predetermined direction at a predetermined frame period of the output of the image sensor, and the acquisition unit is a pixel arranged in the predetermined direction in the unit pixel cell. The distance measuring apparatus according to claim 1, wherein image separation data is generated by combining the two.   The control means to perform generates and outputs a control signal for switching the predetermined direction at a predetermined frame period of the output of the image sensor, and the acquisition means is arranged in the predetermined direction in the unit pixel cell. The distance measuring apparatus according to claim 1, wherein output data synthesized with respect to each pixel is acquired from the image sensor.   The predetermined direction is one of a vertical direction and a horizontal direction of the two-dimensional array of the unit pixel cells, and the predetermined period is a frame period of an output of the imaging element or at least two or more frame periods. The distance measuring apparatus according to any one of claims 1 to 3, wherein An optical system for forming an optical image of a subject;
An imaging means having a two-dimensional array of unit pixel cells composed of a predetermined number of pixels arranged corresponding to the pupil dividing means, and photoelectrically converting the optical image of the subject to generate a pixel signal;
A distance measuring device according to any one of claims 1 to 4,
Drive means for reading the pixel signal by driving the image sensor;
An imaging apparatus comprising:   The image pickup device includes combining means for combining and reading pixel signals of a plurality of pixels constituting the unit pixel cell in different directions of the two-dimensional array, and the driving means is a control output by the control means. The imaging apparatus according to claim 5, wherein the synthesis unit is controlled according to a signal to switch the synthesis direction.   The image capturing apparatus further includes an image capturing control unit that controls an image capturing operation of the image capturing apparatus, the image capturing control unit controlling the distance measuring device, and combining and reading out the pixel signal from the two-dimensional array of the unit pixel cells. The imaging apparatus according to claim 6, wherein the calculation unit performs the distance measurement calculation and outputs the calculation result to the imaging control unit in an adjustment time corresponding to a frame period.   The image processing apparatus further includes a focus adjustment unit that drives the optical system to adjust the focus of the optical image of the subject, and the imaging control unit controls the focus adjustment unit during the adjustment time, and the calculation result output by the calculation unit The image pickup apparatus according to claim 7, wherein the optical system is driven on the basis of the information.   The synthesizing unit has a different configuration corresponding to the direction of the synthesis, and the imaging control unit is short when the adjustment time differs corresponding to the direction of the synthesis because of the different configuration of the synthesizing unit. The imaging apparatus according to claim 8, wherein the calculation unit and the focus adjustment unit are controlled during an adjustment time.   The imaging apparatus according to claim 8, wherein the synthesizing unit performs the same configuration in each synthesis direction.   The imaging apparatus according to claim 5, wherein the imaging element does not include a combining unit that combines and outputs the outputs of the pixels constituting the unit pixel cell.   The image processing unit further includes an image processing unit that processes a pixel signal read from the imaging element by the driving unit, and the image processing unit combines the outputs of a plurality of pixels that form the unit pixel cell to form the subject. The image pickup apparatus according to claim 5, wherein the image is generated and processed. In the control of a distance measuring device that performs distance measurement of a subject using the output of an image sensor having a two-dimensional array of unit pixel cells composed of a predetermined number of pixels arranged corresponding to pupil dividing means,
An acquisition step of acquiring image separation data of the subject obtained by combining outputs of pixels arranged in a predetermined direction of the two-dimensional array for each unit pixel cell;
A calculation step of performing a ranging calculation using the image separation data and outputting a calculation result;
A control step of switching the predetermined direction at a predetermined period;
With
The acquisition step acquires the image separation data in the switched direction according to the switching in the control step, and the calculation step performs a distance measurement calculation using the image separation data acquired in the acquisition step. Characteristic control method. To control a distance measuring device that performs distance measurement of an object using an output of an image sensor having a two-dimensional array of unit pixel cells composed of a predetermined number of pixels arranged corresponding to pupil dividing means The program of
Computer
Acquisition means for acquiring image separation data of the subject obtained by synthesizing outputs of pixels arranged in a predetermined direction of the two-dimensional array for each unit pixel cell;
A calculation means for performing a ranging calculation using the image separation data and outputting a calculation result;
Control means for switching the predetermined direction at a predetermined cycle;
Function as
The acquisition unit acquires image separation data in the switched direction according to the switching by the control unit, and the calculation unit is controlled to perform a distance measurement calculation using the image separation data acquired by the acquisition unit. Program to do.   A computer-readable storage medium storing the program according to claim 14.   The program which makes a computer function as each means of the ranging apparatus as described in any one of Claims 1 thru | or 4.   A storage medium storing a program for causing a computer to function as each unit of the distance measuring device according to any one of claims 1 to 4.

Images