JP7020463B2 - Imaging device - Google Patents

Description

The present invention relates to an image pickup apparatus.

In recent years, an image pickup device including a solid-state image pickup device having a CCD (Charge Coupled Device) or the like has become widespread. As such an image pickup device, there is known an image pickup device that performs image pickup control such as AF (autofocus) based on a pixel value read from a solid-state image sensor (see, for example, Patent Document 1).

Japanese Unexamined Patent Publication No. 2010-200196

By the way, in the image pickup device as described above, in order to improve the processing speed when performing image pickup control such as AF, for example, a frame indicating a speed of reading a pixel value for one screen (one frame) from a solid-state image sensor. May increase the rate. In such a case, the image pickup apparatus, for example, thins out the pixels to increase the frame rate.
However, in the image pickup apparatus as described above, for example, when the brightness is low, the pixel value becomes low if the exposure time is not sufficient, and therefore the SN ratio may decrease. In this case, the image pickup device as described above may reduce the accuracy of image pickup control such as AF.

The present invention has been made to solve the above problems, and an object of the present invention is to provide an image pickup apparatus capable of improving the accuracy of image pickup control.

In order to solve the above problem, the first signal having a plurality of pixels and generated by at least the first pixel among the plurality of pixels is read out at the first frame rate, and the first pixel is read at the first frame rate. Image pickup in which the second signal generated by at least the second pixel of the plurality of pixels is read out at a second frame rate higher than the first frame rate during the period in which the first signal is read out . It is an image pickup apparatus including an element and an image pickup control unit that performs image pickup control using a signal selected from the first signal and the second signal based on the brightness information based on the second signal.

According to the present invention, the accuracy of image pickup control can be improved.

It is a block diagram which shows the image pickup apparatus by this embodiment. It is a figure which shows an example of the arrangement structure of the pixel of the image pickup part in the same embodiment. It is a block diagram which shows the structure of the image processing part in the same embodiment. It is a 1st time chart which shows the reading operation of a pixel value in the same embodiment. 2 is a second time chart showing a pixel value reading operation in the same embodiment. It is a figure which shows another example of the arrangement structure of the pixel of the image pickup part in the same embodiment.

Hereinafter, the solid-state image pickup device and the image pickup apparatus according to the embodiment of the present invention will be described with reference to the drawings.
[First Embodiment]
FIG. 1 is a schematic block diagram showing an image pickup apparatus 1 according to the present embodiment.
In FIG. 1, the image pickup apparatus 1 includes a lens 10, an aperture mechanism 11, an image pickup unit 12, a TG (timing generator) 13, an analog front end unit (hereinafter referred to as “AFE”) 14, an image processing unit 15, and a RAM (Random Access). It includes a Memory) 16, a recording I / F (recording interface) 17, a display unit 18, an operation unit 19, a ROM (Read Only Memory) 21, a bus 22, and a control unit 30. The image processing unit 15, the RAM 16, the recording I / F 17, the display unit 18, the ROM 21, and the control unit 30 are connected to each other via the bus 22. Further, the operation unit 19 is connected to the control unit 30.

Further, in the present embodiment, the image pickup unit 12, the TG13, and the AFE14 correspond to the solid-state image pickup element 100. Further, the control unit 30 and the image processing unit 15 correspond to the image pickup control unit 40.

The lens 10 is composed of a plurality of lens groups including a focus lens and a zoom lens. In FIG. 1, the lens 10 is illustrated as a single lens. The lens 10 is controlled by a lens driving device (not shown).

The aperture mechanism 11 adjusts the amount of light incident on the image pickup unit 12. This adjustment is performed by the aperture drive unit (not shown) according to the instruction of the control unit 30.

The image pickup unit 12 has a plurality of pixels 120 (FIG. 2) arranged in rows and columns, and generates an analog pixel signal of the image by photoelectrically converting a subject image formed on the image pickup surface. That is, the image pickup unit 12 captures a subject image and generates pixel data of the image to be captured. In the present embodiment, for example, CMOS (Complementary Metal-Oxide Semiconductor) of the XY address method is used for the image pickup unit 12. On the light receiving surface of the image pickup unit 12, a plurality of light receiving elements constituting the pixels 120 arranged in a matrix are arranged. The image pickup unit 12 outputs the analog pixel signal generated by receiving light received by these light receiving elements to the AFE 14 as pixel data under the control of the TG 13 based on the instruction of the control unit 30. The image pickup unit 12 is controlled by the TG 13 to perform photoelectric conversion with a different exposure time for each line. The details of the exposure time control and the readout control for each line in the image pickup unit 12 will be described later.

The AFE 14 is an analog front-end circuit that performs signal processing on an analog pixel signal generated by the image pickup unit 12. The AFE 14 adjusts the gain of the analog pixel signal, performs A / D (analog / digital) conversion of the analog pixel signal, and the like. The AFE 14 outputs the converted digital pixel signal as pixel data to the image processing unit 15.

The TG (timing generator) 13 (control unit) supplies a drive signal to each of the image pickup unit 12 and the AFE 14 according to an instruction from the control unit 30, and controls the drive timing of both by the supplied drive signal. The TG 13 has an all-pixel readout mode in which pixel data of all lines of the imaging unit 12 is read out as image data, and a thinning-out readout mode in which pixel data obtained by thinning out pixels of a predetermined line among all lines of the imaging unit 12 is read out as image data. And have. In the present embodiment, in the thinned-out readout mode, the TG 13 reads the pixel value based on the exposure time of each line of the image pickup unit 12 described above from the image pickup unit 12 as information used for the image pickup control. Here, the imaging control is, for example, autofocus control (hereinafter referred to as AF control), automatic exposure control (hereinafter referred to as AE control), and auto white balance control (hereinafter referred to as AWB control). And so on. The image pickup control may include a control for displaying a live view image, a through image, or the like on the display unit 18 for confirming the subject.

For example, the TG 13 controls to read the pixel value based on the exposure time for each line read from the image pickup unit 12 from the image pickup unit 12 as information for calculating the evaluation value in the autofocus control (AF control). Further, for example, the TG 13 controls to read the pixel value based on the exposure time for each line read from the image pickup unit 12 from the image pickup unit 12 as information for calculating the evaluation value in the automatic exposure control (AE control). Further, for example, the TG 13 controls to read out the pixel value based on the exposure time for each line read from the image pickup unit 12 from the image pickup unit 12 as information for calculating the evaluation value in the auto white balance control (AWB control). Further, for example, the TG 13 reads out a pixel value based on a predetermined exposure time among the exposure times for each line as a display image. The TG 13 outputs the pixel value read from the image pickup unit 12 to the image processing unit 15 as pixel data via the AFE 14.

Further, the TG 13 changes the exposure time for each line by, for example, resetting the pixel value stored in the pixel for each line of the image pickup unit 12. Further, the TG 13 changes the line for reading the pixel value from the image pickup unit 12 according to the brightness (brightness) of the subject. For example, when the brightness of the subject is lower than a predetermined value (dark), the TG 13 selects and reads the pixel value of the line having a longer exposure time than when the brightness of the subject is equal to or higher than the predetermined value (bright). The luminance information of the subject is supplied from, for example, the control unit 30.
Further, the TG 13 changes the line for reading the pixel value based on the exposure time for each line from the image pickup unit 12 according to the movement of the subject. For example, when the moving speed of the subject is faster than the predetermined value, the TG 13 selects and reads the pixel value of the line whose exposure time is shorter than when the moving speed of the subject is equal to or less than the predetermined value (when it is bright). Information on the moving speed of the subject is supplied from, for example, the control unit 30.

The display unit 18 is, for example, a liquid crystal display and displays various images in response to instructions from the control unit 30. The display unit 18 displays, for example, image data captured by the image pickup unit 12, an operation screen, or the like.
The operation unit 19 is, for example, a release button, a cross key, a command dial, a touch panel, other operation keys, and the like, and supplies an operation input signal to the control unit 30 according to the operation content by the user. The operation unit 19 gives an imaging instruction to the control unit 30, for example, when the user fully presses the release button.

The ROM 21 stores a control program that controls the image pickup apparatus 1. For example, a sequence program for an imaging operation described later and a sequence program for correction processing described later are stored in the ROM 21 in advance.
The RAM 16 stores image data captured by the image pickup unit 12, various setting information in the image pickup process, and the like.

The recording I / F 17 is connected to a removable recording medium 23 such as a card memory, and writes, reads, or erases image data on the recording medium 23.
The recording medium 23 is a storage unit that is detachably connected to the image pickup device 1, and stores (records) image data or the like image-processed by the image processing unit 15, for example.

The bus 22 is connected to the image processing unit 15, the RAM 16, the recording I / F 17, the display unit 18, the ROM 21, and the control unit 30, and transfers image data, control signals, and the like output from each unit.

The image processing unit 15 stores the image data (pixel data) output by the AFE 14 in the RAM 16. In addition, the image processing unit 15 executes various image processing such as correction processing, evaluation value calculation processing, and white balance, which will be described later. The image processing unit 15 calculates, for example, the AF evaluation value, the AE evaluation value, and the AWB evaluation value based on the pixel values for each line corresponding to the different exposure times read by the TG 13 in the thinning-out reading mode described above. do. The image processing unit 15 executes auto white balance processing based on the calculated AWB evaluation value. Further, the image processing unit 15 outputs the calculated AF evaluation value and the AE evaluation value to the control unit 30 via the bus 22. The detailed configuration of the image processing unit 15 will be described later.

The control unit 30 includes a processor such as a CPU (Central processing unit), and performs overall control of the image pickup apparatus 1. The control unit 30 controls each unit of the image pickup apparatus 1 by executing a control program stored in advance in the ROM 21. For example, when the control unit 30 receives an image pickup instruction via the operation unit 19, the control unit 30 stores the image data obtained via the image pickup unit 12 and the AFE 14 in the recording medium 23 as an image captured. Further, the control unit 30 provides information (for example, luminance information, movement of the subject) for selecting the exposure time for each line corresponding to the pixel value used for calculating the evaluation value described above by the image processing unit 15 in the thinning-out reading mode. (Speed information) is generated based on the image data acquired from the image processing unit 15. The control unit 30 outputs information for selecting the generated exposure time to the TG3.

Further, the control unit 30 performs AF control based on, for example, the AF evaluation value calculated by the image processing unit 15. Specifically, the control unit 30 performs distance measurement processing of the subject based on the AF evaluation value, and controls the focus lens of the lens 10 based on the result of this distance measurement processing.
Further, the control unit 30 performs AE control based on, for example, the AE evaluation value calculated by the image processing unit 15. Specifically, the control unit 30 changes the sensitivity and exposure time of the solid-state image sensor 100 and controls the aperture mechanism 11 based on the AE evaluation value.

Further, the control unit 30 includes an AF control unit 31 and an AE control unit 32.
The AF control unit 31 performs the AF control described above based on the AF evaluation value calculated by the image processing unit 15.
The AE control unit 32 performs the above-mentioned AE control based on the AE evaluation value calculated by the image processing unit 15.

As described above, in the present embodiment, the image pickup control unit 40 corresponding to the control unit 30 and the image processing unit 15 performs the above-mentioned image pickup control based on the pixel value output from the solid-state image sensor 100.

Next, the configuration of the 12 pixels (light receiving element) in this embodiment will be described.
FIG. 2 is a diagram showing an example of the arrangement configuration of the pixels 120 of the imaging unit 12 in the present embodiment.
In FIG. 2, the image pickup unit 12 has a plurality of pixels 120, and the pixel 120, which is a light receiving element, is formed by a two-dimensional arrangement of rows (lines) and columns. Here, the plurality of pixels 120 have a plurality of lines (rows). Each line is classified into five groups (ROWA, ROWB, ROWC, ROWD, and Rowe) with different exposure times in the thinning-out mode.

Here, the lines A1 to A4 correspond to ROWA, and are controlled by the TG 13 so that each pixel 120 is stored and read out by an exposure time corresponding to, for example, a frame rate of 120 fps (Frames Per Second). Further, the lines B1 to B4 correspond to Rowb, and are controlled by the TG 13 so that each pixel 120 is stored and read out by an exposure time corresponding to, for example, a frame rate of 60 fps. Further, the lines C1 to C4 correspond to ROWC, and are controlled by the TG 13 so that each pixel 120 is stored and read out by an exposure time corresponding to, for example, a frame rate of 40 fps. Further, the lines D1 to D4 correspond to ROWD, and are controlled by the TG 13 so that each pixel 120 is stored and read out by an exposure time corresponding to, for example, a frame rate of 30 fps. Further, the lines E1 to E4 correspond to Rowe, and are controlled by the TG 13 so that each pixel 120 is stored and read out by an exposure time corresponding to, for example, a frame rate of 24 fps.

Further, the plurality of pixels 120 have a reset function for resetting (discharging) the pixel value stored for each line (for example, a reset function using a reset transistor) and a shutter function for controlling whether or not the stored pixel value is stored for each line. Have. Further, the image pickup unit 12 has a function of reading out a pixel value for each line. The TG 13 uses these functions to control reset, exposure (shutter), and pixel value reading for each line.

Next, the configuration of the image processing unit 15 will be described.
FIG. 3 is a block diagram showing the configuration of the image processing unit 15 in the present embodiment.
In FIG. 3, the image processing unit 15 includes a black level correction unit 151, a pixel defect correction unit 152, an evaluation value calculation unit 153, and an AWB processing unit 154. The image processing unit 15 is, for example, a DSP (Digital Signal Processor).

The black level correction unit 151 corrects pixel data read from the image pickup unit 12 via the AFE 14 based on dark current information detected by an optical black element (not shown) included in the image pickup unit 12, for example. The black level correction unit 151 outputs the pixel data obtained by performing this black level correction to the pixel defect correction unit 152.

The pixel defect correction unit 152 corrects pixel defects in the pixel data output from the black level correction unit 151 based on the pixel defect information possessed by the image pickup unit 12. The pixel defect correction unit 152 outputs the pixel data for which pixel defect correction has been performed to the evaluation value calculation unit 153 and the AWB processing unit 154.

The evaluation value calculation unit 153 calculates the AF evaluation value, the AE evaluation value, and the AWB evaluation value based on the pixel data output from the pixel defect correction unit 152 (for example, the pixel value obtained by the thinning-out reading mode). .. The evaluation value calculation unit 153 outputs the calculated AF evaluation value and the AE evaluation value to the control unit 30 via the bus 22. Further, the evaluation value calculation unit 153 outputs the calculated AWB evaluation value to the AWB processing unit 154.

The AWB processing unit 154 executes auto white balance processing based on the pixel data output from the pixel defect correction unit 152 and the AWB evaluation value calculated by the evaluation value calculation unit 153.

Next, the operation of the solid-state image pickup device 100 and the image pickup apparatus 1 in the present embodiment will be described.
FIG. 4 is a time chart showing a pixel value reading operation by the TG 13 of the solid-state image sensor 100.
In FIG. 4, first, the TG 13 reads out the pixel value of one pixel (120) of the above-mentioned ROWA line (for example, the line A1 in FIG. 2) in synchronization with the fall of the horizontal synchronization signal at the time T1. ..

At time T2, the reading of the pixel value of the ROWA in the imaging unit 12 is completed, and the TG 13 next reads the pixel value of one pixel (120) of the ROW * line. Here, "*" indicates any of B to E. That is, the TG 13 selects one line (for example, line B1 in FIG. 2) from the above-mentioned RowB to Rowe, and reads out the pixel value of one pixel (120) of the selected line.
Here, the period R1 is a read-out period of the pixel value of the rowb, and the period R2 is a read-out period of the pixel value in one line of Rowb to Rowe. This period R2 corresponds to a horizontal blanking (BLK) period in the prior art. In the present embodiment, the TG 13 reads out the pixel value of the stored line with an exposure time different from that of the ROWA by utilizing the horizontal blanking period in which the reading is not performed in the prior art.

Further, at time T3, in synchronization with the falling edge of the horizontal synchronization signal again, the TG 13 similarly has a pixel value of one pixel (120) next to the rowb line (for example, line A1 in FIG. 2). Then, the pixel value of the next one pixel (120) in one line (for example, line B1 in FIG. 2) of Rowb to Rowe is read out.
By repeating this operation, for example, when the reading of the pixel values of the line A1 and the line B1 is completed, the TG 13 similarly reads the pixel values of the next line (for example, the line A2 and the line B2), and finally. , The pixel value for one screen (one frame) corresponding to ROWA and ROWB is read out.

FIG. 5 is a time chart showing an example of a pixel value reading operation for one screen (one frame) in the present embodiment.
In FIG. 5, the diagonal solid line indicates the read timing of each line synchronized with the above-mentioned horizontal synchronization signal. Further, the diagonal wavy line indicates the shutter timing (and reset timing) of each line. Here, the shutter function is, for example, a rolling shutter.

As shown in FIG. 5, the TG 13 controls the exposure time and pixel value reading of the Rowa to Rowe in synchronization with the vertical synchronization signal.
For example, the TG 13 controls to store the line corresponding to the ROWA (frame A) in the image pickup unit 12 with an exposure time corresponding to 120 fps and read out the pixel value. Further, the TG 13 controls to store the line corresponding to the rowb (frame B) in the image pickup unit 12 with an exposure time corresponding to 60 fps and read out the pixel value. Here, the line corresponding to ROWB (frame B) has twice the exposure time (accumulation time) as the line corresponding to ROWA (frame A).

Similarly, the TG 13 controls to store the line corresponding to the ROWC (frame C) in the image pickup unit 12 with an exposure time corresponding to 40 fps and read out the pixel value. Further, the TG 13 controls to store the line corresponding to the ROWD (frame D) in the image pickup unit 12 with an exposure time corresponding to 30 fps and read out the pixel value. Further, the TG 13 controls to store the line corresponding to the Rowe (frame E) in the image pickup unit 12 with an exposure time corresponding to 24 fps and read out the pixel value.

For example, when reading the pixel values corresponding to the two types of exposure times of the rowb pixel value and the rowb pixel value, the TG 13 synchronizes with the horizontal synchronization signal as shown in FIG. 4 described above. Then, the pixel value of ROWA and the pixel value of ROWB are read out alternately.
Further, in FIG. 5, for example, when the timings for reading out the pixel values corresponding to the three types of exposure times overlap in one frame period, the TG 13 has the pixel value of the ROWA and the remaining two excluding the ROWA. Pixel values corresponding to any one of the exposure times may be alternately read out in synchronization with the horizontal synchronization signal. In this case, the TG 13 may reset the ROW * that does not read the pixel value and discharge the pixel value. Further, when the horizontal blanking period is long, the TG 13 may alternately read out the pixel values corresponding to the three types of exposure times in synchronization with the horizontal synchronization signal, or when the horizontal blanking period is short. May perform window reading (cropping) such as reading the central portion of one screen (one frame).

In this way, by controlling the image pickup unit 12 as shown in FIG. 5, the image pickup unit 12 generates image data (pixel data) corresponding to a plurality of exposure times in the thinning / reading mode. The TG 13 controls to select and read image data corresponding to the plurality of exposure times, and supplies the read image data to the image processing unit 15 via the AFE 14.

For example, the control unit 30 generates the luminance information of the image based on the image data (pixel data) acquired from the image pickup unit 12 via the image processing unit 15, and outputs the generated luminance information to the TG 13. The TG 13 selects pixel data for calculating an evaluation value used for AF control, AE control, and AWB control from among ROWA to Rowe according to the luminance information output from the control unit 30. For example, the TG 13 selects a group (frame) having a longer exposure time as the luminance value is lower (darker image), and selects a group (frame) having a shorter exposure time as the luminance value is higher (brighter image). .. That is, the TG 13 selects a group of exposure times sufficient to obtain a pixel value suitable for calculating the evaluation value from among ROWA to Rowe according to the luminance value. The TG 13 reads out the pixel data corresponding to the selected exposure time and supplies it to the image processing unit 15.

The evaluation value calculation unit 153 of the image processing unit 15 calculates evaluation values such as AF control, AE control, and AWB control based on the supplied pixel data. For example, the WB processing unit 154 executes the auto white balance process based on the AWB evaluation value calculated by the evaluation value calculation unit 153. Further, for example, the AF control unit 31 of the control unit 30 performs AF control based on the AF evaluation value calculated by the evaluation value calculation unit 153, and the AE control unit 32 performs the AE evaluation value calculated by the evaluation value calculation unit 153. AE control is performed based on.
Further, the TG 13 reads out a pixel value (pixel data) based on a predetermined exposure time (for example, an exposure time corresponding to 30 fps) among the exposure times for each line as a display image (for example, a live view image). In this case, the image processing unit 15 stores the pixel data acquired from the image pickup unit 12 in the RAM 16. Then, the display unit 18 reads out the pixel data acquired by the image pickup unit 12 from the RAM 16 and displays it as a display image. Here, the display image is an image based on the thinned-out pixel data.

As described above, in the solid-state image pickup device 100 in the present embodiment, the image pickup unit 12 has a plurality of pixels 120 arranged in rows and columns. The TG 13 reads a pixel value based on the exposure time for each line from the image pickup unit 12 as information used for image pickup control (eg, AF control, AE control, AWB control, etc.).
As a result, the TG 13 can select and read the pixel value corresponding to the exposure time appropriate for the image pickup control. Therefore, since the solid-state image sensor 100 in the present embodiment performs image pickup control based on an appropriate pixel value, the accuracy of the image pickup control can be improved.

For example, when the brightness of the subject is low (when the image is dark), the TG 13 selects a line having a long exposure time (for example, ROWE) and reads out the pixel value. As a result, the solid-state image sensor 100 in the present embodiment can obtain a sufficient pixel value for image pickup control, so that the SN ratio can be improved. Therefore, the solid-state image sensor 100 in the present embodiment can cause the image pickup device 1 to perform image pickup control based on the pixel value in which the error due to noise is reduced. Therefore, the solid-state image pickup device 100 in the present embodiment can improve the accuracy of image pickup control.

Further, for example, when the brightness of the subject is high (when the image is high), the TG 13 selects a line having a short exposure time (for example, ROWA) and reads out the pixel value. As a result, the solid-state image sensor 100 in the present embodiment can reduce the saturation of the pixel value and put the pixel value within an appropriate range of the dynamic range. Therefore, the solid-state image sensor 100 in the present embodiment can cause the image pickup device 1 to control the image pickup based on the pixel values within an appropriate range of the dynamic range. Therefore, the solid-state image pickup device 100 in the present embodiment can improve the accuracy of image pickup control.

Further, in the present embodiment, the TG 13 changes the exposure time for each line by resetting the pixel value stored in the pixel 120 for each line.
Thereby, the solid-state image sensor 100 in the present embodiment can appropriately change the exposure time for each line by a simple means.

Further, in the present embodiment, the TG 13 changes the line (line) for reading the pixel value based on the exposure time for each line (line) from the image pickup unit 12 according to the brightness of the subject.
As a result, the level of the pixel value can be appropriately changed according to the brightness of the subject, so that the solid-state image pickup device 100 in the present embodiment can improve the accuracy of the image pickup control.

Further, in the present embodiment, the TG 13 changes the line (line) for reading the pixel value based on the exposure time for each line (line) from the image pickup unit 12 according to the movement of the subject.
For example, the TG 13 reads a pixel value from a line having a short exposure time (for example, ROWA) when the subject moves fast (moves fast). As a result, subject blur can be reduced, so that the solid-state image pickup device 100 in the present embodiment can improve the accuracy of image pickup control.

Further, in the present embodiment, the image pickup control includes, for example, autofocus control (AF control). The TG 13 reads a pixel value based on the exposure time for each line read from the image pickup unit 12 from the image pickup unit 12 as information for calculating an evaluation value (AF evaluation value) in autofocus control.
Thereby, the solid-state image sensor 100 in the present embodiment can improve the accuracy of AF control, for example.

Further, in the present embodiment, the imaging control includes, for example, automatic exposure control (AE control). The TG 13 reads a pixel value based on the exposure time for each line read from the image pickup unit 12 from the image pickup unit 12 as information for calculating an evaluation value (AE evaluation value) in automatic exposure control.
Thereby, the solid-state image sensor 100 in the present embodiment can improve the accuracy of AE control, for example.

Further, in the present embodiment, the image pickup control includes, for example, auto white balance control (AWB control). The TG 13 reads a pixel value based on the exposure time for each line read from the image pickup unit 12 from the image pickup unit 12 as information for calculating an evaluation value (AWB evaluation value) in the control of auto white balance.
Thereby, the solid-state image sensor 100 in the present embodiment can improve the accuracy of AWB control, for example.

Further, in the present embodiment, the TG 13 reads out a pixel value based on a predetermined exposure time among the exposure times for each line as a display image.
Thereby, the solid-state image pickup device 100 in the present embodiment can generate a display image such as a live view image together with the information used for the image pickup control (eg, AF control, AE control, AWB control, etc.).

The image pickup device 1 in the present embodiment controls image pickup (eg, AF control) based on the above-mentioned solid-state image sensor 100 and pixel values based on the exposure time for each line output from the solid-state image sensor 100. , AE control, AWB control, etc.).
As a result, the image pickup device 1 in the present embodiment has the same effect as the solid-state image pickup device 100, and the accuracy of the image pickup control can be improved.

Next, the second embodiment will be described.
[Second Embodiment]
In the second embodiment, an example will be described in which the imaging unit 12 includes a plurality of readout circuits and can read out a plurality of pixel values in parallel. The other configurations and operations are the same as those in the first embodiment.

FIG. 6 is a diagram showing another example of the arrangement configuration of the pixels 120 of the imaging unit 12.
In this figure, as shown in groups G1 to G10, the image pickup unit 12 is grouped by four pixels and shared by pixels. Each of the groups G1 to G10 can read the pixel value of one of the four pixels. Further, the imaging unit 12 has a configuration capable of reading out the pixel values of two groups from the groups G1 to G10 in parallel.
In this case, the TG 13 can read, for example, the above-mentioned rowa pixel value and the ROWC pixel value in parallel. Thereby, the solid-state image sensor 100 according to the present embodiment can shorten the period for reading the pixel value from the image pickup unit 12. For example, even when the above-mentioned horizontal blanking period R2 is shorter than the pixel value readout period, the solid-state image sensor 100 according to the present embodiment has the above-mentioned ROWA pixel value and ROWC pixel value within one frame period. Both pixel values of and can be output.

The present invention is not limited to each of the above embodiments, and can be modified without departing from the spirit of the present invention.
In each of the above embodiments, as an example, as shown in FIG. 5, RowB, Rowb, Rowb, and Rowe correspond to frame rates of 2, 3, 4, and 5 times that of ROWA. The case where the exposure time is used has been described, but the present invention is not limited to this. For example, RowB, ROWC, RowD, and Rowe may be in the form of exposure times corresponding to frame rates of 2, 4, 8, and 16 times that of ROWA.

Further, in each of the above embodiments, the pixel value for generating the AF evaluation value may be generated based on the pixel values having different exposure times. For example, the image pickup apparatus 1 generates an AF evaluation value from pixel values acquired with a short exposure time, performs coarse AF control, and then generates an AF evaluation value from pixel values acquired with an exposure time longer than a short exposure time. Then, the AF control may be performed accurately. In this case, the image pickup apparatus 1 can shorten the time (period) until focusing.

Further, in each of the above embodiments, the image pickup apparatus 1 has described a mode in which the pixel value corresponding to any one of ROWA, ROWB, Rowb, ROWD, and Rowe is used for image pickup control, but a plurality of them are used in combination. The form may be used, for example, a form using the summed value or the average value of the pixel values corresponding to a plurality of the rowab, rowb, rowb, rowb, and rowe.
Further, the image pickup apparatus 1 may be used by adding the evaluation value of the previous frame and the evaluation value of the next frame. In this case, the dynamic range can be widened, and the image pickup apparatus 1 can improve the accuracy of AE control in, for example, a high-brightness subject.

Further, in each of the above embodiments, the form in which the image pickup device 1 includes the image processing unit 15 has been described, but the solid-state image pickup element 100 may include a form in which the image processing unit 15 or a part of the image processing unit 15 is provided.
Further, the TG 13 and the image pickup control unit 40 in each of the above embodiments may be realized by dedicated hardware, or may be realized by a memory and a CPU (central processing unit). ..

The image pickup device 1 or the solid-state image pickup device 100 described above has a computer system inside. The processing process of the image pickup device 1 or the solid-state image sensor 100 described above is stored in a computer-readable recording medium in the form of a program, and the process is performed by the computer reading and executing this program. .. Here, the computer-readable recording medium means a magnetic disk, a magneto-optical disk, a CD-ROM, a DVD-ROM, a semiconductor memory, or the like. Further, the computer program may be distributed to the computer via a communication line, and the computer receiving the distribution may execute the program.

1 ... Image pickup device, 12 ... Image pickup unit, 13 ... TG (control unit), 40 ... Image pickup control unit, 100 ... Solid-state image sensor, 120 ... Pixel

Claims (10)

A first signal having a plurality of pixels and generated by at least the first pixel among the plurality of pixels is read at a first frame rate, and the first signal is read from the first pixel at the first frame rate. An image pickup element that reads out a second signal generated by at least the second pixel of the plurality of pixels at a second frame rate having a frame rate higher than that of the first frame rate.
An image pickup control unit that performs image pickup control using a signal selected from the first signal and the second signal based on the luminance information based on the second signal.
An image pickup device equipped with. In the image pickup apparatus according to claim 1,
Light from an optical system including a focus lens is incident on the image sensor.
The image pickup control unit is an image pickup device that controls the focus lens by using a signal selected from the first signal and the second signal based on the luminance information based on the second signal. In the image pickup apparatus according to claim 2,
The image pickup control unit calculates an evaluation value for controlling the focus lens by using a signal selected from the first signal and the second signal based on the luminance information based on the second signal. .. The imaging device according to any one of claims 1 to 3.
The image pickup control unit is an image pickup device that controls exposure using a signal selected from the first signal and the second signal based on luminance information based on the second signal. In the image pickup apparatus according to claim 4,
Light from the diaphragm mechanism is incident on the image sensor.
The image pickup control unit is an image pickup device that controls the aperture mechanism by using a signal selected from the first signal and the second signal based on the luminance information based on the second signal. In the image pickup apparatus according to claim 4 or 5.
The image pickup control unit is an image pickup device that calculates an evaluation value for controlling exposure using a signal selected from the first signal and the second signal based on luminance information based on the second signal. The imaging device according to any one of claims 1 to 6.
The image pickup control unit is an image pickup device that performs white balance processing using a signal selected from the first signal and the second signal based on luminance information based on the second signal. In the image pickup apparatus according to claim 7,
The image pickup control unit is an image pickup device that calculates an evaluation value for white balance processing using a signal selected from the first signal and the second signal based on luminance information based on the second signal. The imaging device according to any one of claims 1 to 8.
The image pickup control unit is an image pickup device that displays an image generated by using a signal selected from the first signal and the second signal based on the luminance information based on the second signal on the display unit. The imaging device according to any one of claims 1 to 9.
The image pickup control unit is an image pickup device that causes a recording unit to record image data generated by using a signal selected from the first signal and the second signal based on luminance information based on the second signal.

Images