JP6056572B2 - Imaging device - Google Patents

Description

  The present invention relates to an imaging apparatus.

There is an imaging apparatus that divides an effective pixel area into a plurality of small areas and calculates a focus evaluation value for each small area for image data obtained by imaging (see, for example, Patent Document 1). In this imaging apparatus, imaging is performed at a plurality of step positions of the focus lens. Furthermore, based on the focus evaluation value for each small area, image data corresponding to a plurality of small areas at different step positions or the same step position are integrated to generate an image.
[Prior art documents]
[Patent Literature]
[Patent Document 1] JP 2009-260800 A

  However, in the above imaging device, since the image data of the pixels in the effective pixel area is obtained after being imaged at each step position, the data is divided into a plurality of small areas, so that the amount of data is large and the processing is slow. There is.

  According to the first aspect of the present invention, there is provided an imaging apparatus, wherein each of a plurality of areas obtained by dividing an imaging area in which a plurality of pixels are two-dimensionally arranged into a plurality of areas includes pixels included in each of the plurality of areas. A readout unit that controls the accumulation and reads out the pixel signal; an arithmetic unit that calculates an evaluation value based on the pixel signal and determines whether or not the evaluation value satisfies a predetermined determination condition for each of a plurality of regions; The pixel value based on the pixel signal is output to the region determined to satisfy the determination condition, and the imaging condition corresponding to the evaluation value is determined for the region determined not to satisfy the determination condition. Corresponding to an imaging region using a control unit that changes and repeats accumulation control by the reading unit and reading and determination by the calculation unit, and pixel values that are determined to satisfy the determination condition for each of a plurality of regions Picture And an image generation unit for generating a.

  It should be noted that the above summary of the invention does not enumerate all the necessary features of the present invention. In addition, a sub-combination of these feature groups can also be an invention.

1 is a cross-sectional view of a backside illuminating type MOS imaging device according to the present embodiment. It is a figure explaining the pixel arrangement | sequence and unit block of an imaging chip. It is a circuit diagram corresponding to the unit block of an imaging chip. It is a block diagram which shows the structure of the imaging device which concerns on this embodiment. It is a block diagram which shows the specific structure as an example of a signal processing chip. The functional block of an arithmetic circuit is shown. An example of a subject used to explain the operation of the imaging apparatus is shown. The positional relationship of each subject included in the subject is shown. It is an example of the flowchart which shows operation | movement of an imaging device. An example of the imaged image is shown. An example of the imaged image is shown. An example of the imaged image is shown. An example of the imaged image is shown. An example of the imaged image is shown.

  Hereinafter, the present invention will be described through embodiments of the invention, but the following embodiments do not limit the invention according to the claims. In addition, not all the combinations of features described in the embodiments are essential for the solving means of the invention.

  FIG. 1 is a cross-sectional view of a back-illuminated image sensor 100 according to this embodiment. The imaging device 100 includes an imaging chip 113 that outputs a pixel signal corresponding to incident light, a signal processing chip 111 that processes the pixel signal, and a memory chip 112 that stores the pixel signal. The imaging chip 113, the signal processing chip 111, and the memory chip 112 are stacked, and are electrically connected to each other by a conductive bump 109 such as Cu.

  As shown in the figure, incident light is incident mainly in the positive direction of the Z-axis indicated by a white arrow. In the present embodiment, in the imaging chip 113, the surface on the side where incident light is incident is referred to as a back surface. Further, as shown in the coordinate axes, the left direction of the paper orthogonal to the Z axis is the X axis plus direction, and the front side of the paper orthogonal to the Z axis and the X axis is the Y axis plus direction. In the following several figures, the coordinate axes are displayed so that the orientation of each figure can be understood with reference to the coordinate axes of FIG.

  An example of the imaging chip 113 is a back-illuminated MOS image sensor. The PD layer 106 is disposed on the back side of the wiring layer 108. The PD layer 106 includes a plurality of PDs (photodiodes) 104 that are two-dimensionally arranged and store charges corresponding to incident light, and transistors 105 that are provided corresponding to the PDs 104.

  A color filter 102 is provided on the incident side of incident light in the PD layer 106 via a passivation film 103. The color filter 102 has a plurality of types that transmit different wavelength regions, and has a specific arrangement corresponding to each of the PDs 104. The arrangement of the color filter 102 will be described later. A set of the color filter 102, the PD 104, and the transistor 105 forms one pixel.

  On the incident light incident side of the color filter 102, a microlens 101 is provided corresponding to each pixel. The microlens 101 condenses incident light toward the corresponding PD 104.

  The wiring layer 108 includes a wiring 107 that transmits the pixel signal from the PD layer 106 to the signal processing chip 111. The wiring 107 may be multilayer, and a passive element and an active element may be provided.

  A plurality of bumps 109 are disposed on the surface of the wiring layer 108. The plurality of bumps 109 are aligned with the plurality of bumps 109 provided on the opposing surfaces of the signal processing chip 111, and the imaging chip 113 and the signal processing chip 111 are pressed and aligned. The bumps 109 are joined and electrically connected.

  Similarly, a plurality of bumps 109 are disposed on the mutually facing surfaces of the signal processing chip 111 and the memory chip 112. The bumps 109 are aligned with each other, and the signal processing chip 111 and the memory chip 112 are pressurized, so that the aligned bumps 109 are joined and electrically connected.

  The bonding between the bumps 109 is not limited to Cu bump bonding by solid phase diffusion, and micro bump bonding by solder melting may be employed. Further, for example, about one bump 109 may be provided for one unit block described later. Therefore, the size of the bump 109 may be larger than the pitch of the PD 104. Further, a bump larger than the bump 109 corresponding to the imaging region may be provided in a peripheral region other than the imaging region where the pixels are arranged.

  The signal processing chip 111 has a TSV (silicon through electrode) 110 that connects circuits provided on the front and back surfaces to each other. The TSV 110 is preferably provided in the peripheral area. The TSV 110 may also be provided in the peripheral area of the imaging chip 113 and the memory chip 112.

  FIG. 2 is a diagram for explaining the pixel array and the unit block 131 of the imaging chip 113. In particular, a state where the imaging chip 113 is observed from the back side is shown. In the imaging area, 20 million or more pixels are arranged in a matrix. The imaging area is divided into a plurality of unit blocks 131.

  In the example of FIG. 2, adjacent 16 pixels of 4 × 4 pixels form one unit block 131. The grid lines in the figure indicate the concept that adjacent pixels are grouped to form a unit block 131. The number of pixels forming the unit block 131 is not limited to this, and may be about 1000, for example, 32 pixels × 64 pixels, or more or less. The unit blocks 131 are two-dimensionally arranged to form the imaging region.

  As shown in the partially enlarged view of the pixel region, the unit block 131 includes four so-called Bayer arrays, which are composed of four pixels of green pixels Gb, Gr, blue pixels B, and red pixels R, in the vertical and horizontal directions. The green pixel is a pixel having a green filter as the color filter 102, and receives light in the green wavelength band of incident light. Similarly, a blue pixel is a pixel having a blue filter as the color filter 102 and receives light in the blue wavelength band, and a red pixel is a pixel having a red filter as the color filter 102 and receiving light in the red wavelength band. Receive light.

  In the present embodiment, an evaluation value is calculated for each of the plurality of unit blocks 131, and it is determined whether or not the evaluation value satisfies a predetermined determination condition. Accumulation, reading, and the like of the unit block 131 that satisfies the determination condition are completed, and for the unit block 131 that does not satisfy the determination condition, the image capturing condition is changed, accumulation, reading, evaluation value determination, and the like are repeated. An example of the evaluation value is the contrast in the unit block 131, and an example of the determination condition is whether or not the maximum value has been reached. An example of the imaging condition to be changed in this case is a focus position. Another example of the evaluation value is the luminance in the unit block 131, and an example of the determination condition is whether or not the maximum value has been reached. Examples of imaging conditions to be changed in this case are an aperture, a shutter speed, ISO sensitivity, and the like.

  FIG. 3 is a circuit diagram corresponding to the unit block 131 of the imaging chip 113. In the figure, a rectangle surrounded by a dotted line typically represents a circuit corresponding to one pixel. Note that at least some of the transistors described below correspond to the transistor 105 in FIG.

  As described above, the unit block 131 is formed of 16 pixels. The 16 PDs 104 corresponding to the respective pixels are respectively connected to the transfer transistors 302, and the gates of the transfer transistors 302 are connected to the TX wiring 307 to which transfer pulses are supplied. In the present embodiment, the TX wiring 307 is commonly connected to the 16 transfer transistors 302.

  The drain of each transfer transistor 302 is connected to the source of the corresponding reset transistor 303, and a so-called floating diffusion FD between the drain of the transfer transistor 302 and the source of the reset transistor 303 is connected to the gate of the amplification transistor 304. The drain of the reset transistor 303 is connected to a Vdd wiring 310 to which a power supply voltage is supplied, and the gate thereof is connected to a reset wiring 306 to which a reset pulse is supplied. In the present embodiment, the reset wiring 306 is commonly connected to the 16 reset transistors 303.

  The drain of each amplification transistor 304 is connected to a Vdd wiring 310 to which a power supply voltage is supplied. The source of each amplification transistor 304 is connected to the drain of each corresponding selection transistor 305. Each gate of the selection transistor is connected to a decoder wiring 308 to which a selection pulse is supplied. In the present embodiment, the decoder wiring 308 is provided independently for each of the 16 selection transistors 305. The source of each selection transistor 305 is connected to a common output wiring 309. The load current source 311 supplies current to the output wiring 309. That is, the output wiring 309 for the selection transistor 305 is formed by a source follower. Note that the load current source 311 may be provided on the imaging chip 113 side or on the signal processing chip 111 side. Further, instead of providing the decoder wirings 308 corresponding to the number of pixels, individual pixels may be selected by providing a matrix selection line and transistors corresponding thereto.

  Here, the flow from the start of charge accumulation to the output of the pixel signal after the end of accumulation will be described. When a reset pulse is applied to the reset transistor 303 through the reset wiring 306 and simultaneously a transfer pulse is applied to the transfer transistor 302 through the TX wiring 307, the potentials of the PD 104 and the floating diffusion FD are reset.

  When the application of the transfer pulse is canceled, the PD 104 converts the incident light to be received into charges and accumulates them. Thereafter, when the transfer pulse is applied again without the reset pulse being applied, the accumulated charge is transferred to the floating diffusion FD, and the potential of the floating diffusion FD changes from the reset potential to the signal potential after the charge accumulation. . When a selection pulse is applied to the selection transistor 305 through the decoder wiring 308, a change in the signal potential of the floating diffusion FD is transmitted to the output wiring 309 through the amplification transistor 304 and the selection transistor 305. Thereby, a pixel signal corresponding to the reset potential and the signal potential is output from the unit pixel to the output wiring 309.

  As shown in the drawing, in the present embodiment, the reset wiring 306 and the TX wiring 307 are common to the 16 pixels forming the unit block 131. That is, the reset pulse and the transfer pulse are simultaneously applied to all 16 pixels. Therefore, all the pixels forming the unit block 131 start charge accumulation at the same timing and end charge accumulation at the same timing. However, the pixel signal corresponding to the accumulated electric charge is sequentially applied to each selection transistor 305 by a selection pulse, and is selectively output to the output wiring 309. In addition, the reset wiring 306, the TX wiring 307, and the output wiring 309 are provided separately for each unit block 131.

  By configuring the circuit with the unit block 131 as a reference in this way, the charge accumulation time can be controlled for each unit block 131. In other words, pixel signals with different charge accumulation times can be output between adjacent unit blocks 131, respectively. More specifically, while one unit block 131 performs charge accumulation once, the other unit block 131 repeats charge accumulation several times and outputs a pixel signal each time. Each frame for moving images can be output at a different frame rate between the unit blocks 131. In addition, it is possible to control whether to read out pixel signals for each unit block 131.

  FIG. 4 is a block diagram illustrating a configuration of the imaging apparatus according to the present embodiment. The imaging apparatus 500 includes a photographic lens 520 as a photographic optical system, and the photographic lens 520 guides a subject luminous flux incident along the optical axis OA to the imaging element 100. The photographing lens 520 may be an interchangeable lens that can be attached to and detached from the imaging apparatus 500. The imaging apparatus 500 mainly includes an imaging device 100, a system control unit 501, a drive unit 502, a photometry unit 503, a work memory 504, a recording unit 505, a display unit 506, and a drive unit 514.

  The photographing lens 520 is composed of a plurality of optical lens groups, and forms an image of a subject light flux from the scene in the vicinity of its focal plane. In FIG. 4, the photographic lens 520 is representatively represented by a single virtual lens arranged in the vicinity of the pupil.

  The driving unit 514 drives the taking lens 520. More specifically, the drive unit 514 moves the optical lens group of the photographic lens 520 to change the focus position, and also drives the iris diaphragm in the photographic lens 520 to input the light amount of the subject luminous flux incident on the image sensor 100. To control.

  The drive unit 502 is a control circuit that executes charge accumulation control such as timing control and area control of the image sensor 100 in accordance with instructions from the system control unit 501. Further, the operation unit 508 receives an instruction from the photographer through a release button or the like.

  The image sensor 100 delivers the pixel signal to the image processing unit 511 of the system control unit 501. The image processing unit 511 performs various image processing using the work memory 504 as a work space, and generates image data. For example, when generating image data in JPEG file format, a compression process is executed after generating a color video signal from a signal obtained by the Bayer array. The generated image data is recorded in the recording unit 505, converted into a display signal, and displayed on the display unit 506 for a preset time.

  The photometric unit 503 detects the luminance distribution of the scene prior to a series of shooting sequences for generating image data. The photometry unit 503 includes, for example, an AE sensor having about 1 million pixels. The calculation unit 512 of the system control unit 501 receives the output of the photometry unit 503 and calculates the luminance for each area of the scene. The calculation unit 512 determines the shutter speed, aperture value, and ISO sensitivity according to the calculated luminance distribution. The light metering unit 503 may be shared by the image sensor 100. Note that the arithmetic unit 512 also executes various arithmetic operations for operating the imaging device 500.

  A part or all of the drive unit 502 may be mounted on the imaging chip 113, or part or all of the drive unit 502 may be mounted on the signal processing chip 111. A part of the system control unit 501 may be mounted on the imaging chip 113 or the signal processing chip 111.

  FIG. 5 is a block diagram showing a specific configuration as an example of the signal processing chip 111. The signal processing chip 111 has a function of the driving unit 502.

  The signal processing chip 111 performs overall control of the sensor control unit 441, the block control unit 442, the synchronization control unit 443, the signal control unit 444, the individual circuit unit 450A, and the like as shared control functions. Drive controller 420. The signal processing chip 111 further includes an I / F circuit 418 between the drive control unit 420 and the system control unit 501 of the imaging apparatus 500 main body. These sensor control unit 441, block control unit 442, synchronization control unit 443, signal control unit 444, and drive control unit 420 are provided one by one for the signal processing chip 111.

  On the other hand, the individual circuit units 450A, 450B, 450C, 450D, and 450E are provided for each unit block 131A, 131B, 131C, 131D, and 131E. Since the individual circuit units 450A, 450B, 450C, 450D, and 450E have the same configuration, the individual circuit unit 450A will be described below. The individual circuit unit 450A includes a CDS circuit 410, a multiplexer 411, an A / D conversion circuit 412, a demultiplexer 413, a pixel memory 414, and an arithmetic circuit 415. The arithmetic circuit 415 transmits and receives signals to and from the system control unit 501 via the I / F circuit 418.

  The individual circuit unit 450A is preferably arranged in a region that overlaps the region in which the pixels of the corresponding unit block 131A are arranged. Accordingly, the individual circuit unit 450A can be provided in each of the plurality of unit blocks 131A without increasing the size of each chip in the surface direction.

  The drive control unit 420 refers to the timing memory 430, converts an instruction from the system control unit 501 into a control signal that can be executed by each control unit, and delivers the control signal to each. In particular, the drive control unit 420 identifies the unit block 131A and the like that should be stored and read in the timing memory 430, and passes control parameters to each control unit together with information identifying the unit block 131A and the like. Examples of the control parameters include a frame rate, a thinning rate, the number of addition rows or addition columns to which pixel signals are added, a charge accumulation time or accumulation count, a digitization bit number, and the like.

  The sensor control unit 441 performs transmission control of control pulses that are transmitted to the imaging chip 113 and are related to charge accumulation and charge readout of each pixel. Specifically, the sensor control unit 441 controls the start and end of charge accumulation by sending a reset pulse and a transfer pulse to the target pixel, and sends a selection pulse to the readout pixel. A pixel signal is output to the output wiring 309.

  The block control unit 442 executes transmission of a specific pulse that specifies the unit block 131 to be controlled and is transmitted to the imaging chip 113. The transfer pulse and the reset pulse received by each pixel via the TX wiring 307 and the reset wiring 306 are the logical product of each pulse sent by the sensor control unit 441 and a specific pulse sent by the block control unit 442. In this way, each area can be controlled as an independent block.

  The synchronization control unit 443 sends a synchronization signal to the imaging chip 113. Each pulse becomes active in the imaging chip 113 in synchronization with the synchronization signal. For example, by adjusting the synchronization signal, random control, thinning control, and the like that control only specific pixels of pixels belonging to the same unit block 131A or the like are realized.

  The signal control unit 444 mainly performs timing control for the A / D conversion circuit 412. The pixel signal output via the output wiring 309 is input to the A / D conversion circuit 412 via the CDS circuit 410 and the multiplexer 411. The CDS circuit 410 removes noise from the pixel signal.

  The A / D conversion circuit 412 is controlled by the signal control unit 444 to convert the input pixel signal into a digital signal. The pixel signal converted into the digital signal is transferred to the demultiplexer 413 and stored as a pixel value of digital data in the pixel memory 414 corresponding to each pixel.

  The arithmetic circuit 415 calculates an evaluation value based on the pixel value stored in the pixel memory 414 for the corresponding unit block 131A, and determines whether or not the evaluation value satisfies a predetermined determination condition. The arithmetic circuit 415 outputs the result of this determination to the drive control unit 420 in association with information specifying the unit block 131A.

  The pixel memory 414 has a memory space that can store pixel values for the number of pixels that is twice or more the number of pixels included in the unit block 131A, and stores each pixel value that is accumulated and read out under different shooting conditions. . The pixel memory 414 is provided with a data transfer interface that transmits pixel signals in accordance with a delivery request. The data transfer interface is connected to a data transfer line connected to the image processing unit 511. The data transfer line is constituted by a data bus of the bus lines, for example. In this case, the delivery request from the system control unit 501 to the drive control unit 420 is executed by address designation using the address bus.

  Transmission of pixel signals by the data transfer interface is not limited to the addressing method, and various methods can be adopted. For example, when performing data transfer, a double data rate method in which processing is performed using both rising and falling edges of a clock signal used for synchronization of each circuit may be employed. Further, it is possible to adopt a burst transfer method in which data is transferred all at once by omitting a part of the procedure such as addressing and the like, and the speed is increased. Further, a bus system using a line in which a control unit, a memory unit, and an input / output unit are connected in parallel, or a serial system that transfers data one bit at a time can be combined.

  With this configuration, the image processing unit 511 can receive only the necessary pixel values, so that image processing can be completed at high speed, particularly when a low-resolution image is formed.

  The signal processing chip 111 has a timing memory 430 formed by a flash RAM or the like. The timing memory 430 stores control parameters such as the number of times of charge accumulation for which unit block 131A and the like is repeated in association with information specifying the unit block 131A and the like. The timing memory 430 further includes information indicating “whether or not the evaluation value satisfies the determination condition”, for example, a flag, which is output from the arithmetic circuit 415 such as the individual circuit unit 450A. Are stored in association with each other.

  FIG. 6 shows functional blocks of the arithmetic circuit 415. The arithmetic circuit 415 includes a high frequency component calculation unit 460, a contrast calculation unit 462, and a maximum detection unit 464. The high-frequency component calculation unit 460 reads the G pixel signal of each pixel of the unit block 131A stored in the pixel memory 414, and extracts a spatial high-frequency component Gh by performing high-pass filter processing based on the two-dimensional array. To do. Similarly, the high frequency component calculation unit 460 calculates the high frequency component Rh of the R pixel and the high frequency component Bh of the B pixel.

  The contrast calculation unit 462 calculates the sum of absolute values of the high frequency components Gh, Rh, and Bh for a plurality of pixels included in the unit block 131A. The contrast value is an example of an evaluation value. The method for calculating the contrast value is not limited to this.

  The maximum detection unit 464 detects whether or not the contrast value of the unit block 131A is maximum. In this case, the maximum detection unit 464 stores the contrast value history, and compares the newly calculated contrast value with the previously calculated contrast value to detect whether or not the maximum value has been reached. To do. Whether or not the local maximum has been reached is an example of a determination condition for determining whether or not the evaluation value of the unit block 131A satisfies the determination condition.

  The maximum detection unit 464 outputs information indicating whether or not the contrast value has reached a maximum to the drive control unit 420. Instead of or in addition to this, the local maximum detection unit 464 may directly write the information in the timing memory 430.

  FIG. 7 shows an example of the subject 170 used to explain the operation of the imaging apparatus 500. FIG. 8 shows the positional relationship between individual subjects included in the subject 170.

  7 and 8, the subject 170 includes subjects A, B, C, D, and E. As shown in FIG. 8, subjects A, B, C, D, and E are arranged in order of proximity to the imaging apparatus 500.

  A thin line in FIG. 7 indicates the boundary of the unit block 131 when the subject 170 is imaged in the entire imaging region of the imaging device 100. A thick line indicates a boundary of a unit block group 132A including a plurality of or a single unit block 131 including each of the subjects A and the like.

  FIG. 9 is an example of a flowchart illustrating the operation of the imaging apparatus 500. 10 to 14 show an example of an image captured by the operation. 10 to 14 are referred to as images for the sake of explanation, these images may not be generated in a format that can be displayed by the imaging apparatus 500.

  The flowchart in FIG. 9 starts when the release button of the imaging apparatus 500 is pressed halfway, for example. The system control unit 501 sets imaging conditions such as an aperture value and a shutter speed (S100). These imaging conditions may prioritize input from the photographer or may be preset by the system control unit 501 side. In the case where the system control unit 501 side sets, it is particularly preferable to set the aperture value to the open side and the shutter speed to the fast side.

  The system control unit 501 drives the photographing lens 520 by the driving unit 514 to set the in-focus position (S102). In this case, the system control unit 501 divides a range from the position of the imaging device 500 to infinity into a plurality of areas, and drives the photographing lens 520 so that the in-focus positions are respectively in the plurality of areas. As shown in FIG. 8, the area may be set so as to become longer as the distance from the taking lens 520 increases. The system control unit 501 stores the driving amount of the photographing lens 520 based on the above area in a table format or the like in advance. In the example of FIG. 9, the in-focus position is sequentially set from the side closer to the imaging device 500 to the side farther.

  The drive control unit 420 selects a unit block 131 to be stored and read based on an instruction from the system control unit 501 (S103). In this case, the drive control unit 420 refers to the timing memory 430 and identifies the unit block 131 that stores data. Note that the initial setting is preferably performed so that all unit blocks 131 are selected.

  The drive control unit 420 instructs the sensor control unit 441 and the like to store and read out the unit block 131 selected in step S103 (S104). Based on the read pixel signal, the arithmetic circuit 415 calculates the contrast value in each of the unit blocks 131 and determines the contrast value (S106).

  When the arithmetic circuit 415 determines that the contrast value is the maximum value (S107: Yes), the arithmetic circuit 415 outputs the pixel value of the unit block 131 to the image processing unit 511 (S108). At the same time, the arithmetic circuit 415 writes the fact that the contrast value satisfies the determination condition together with information for specifying the unit block 131 in the timing memory 430 via the drive control unit 420.

  On the other hand, when the arithmetic circuit 415 determines that the contrast value is not the maximum value (S107: No), the arithmetic circuit 415 writes the pixel value of the unit block 131 into the pixel memory 414, and proceeds to step S110. Therefore, the fact that the contrast value satisfies the determination condition is not written in the timing memory 430 for the unit block 131.

  The system control unit 501 repeats the above steps S102 to S110 until the current in-focus position is an area including infinity (S110: No). When the current in-focus position is an area including infinity (S110: Yes), the system control unit 501 refers to the timing memory 430 and identifies the unit block 131 that does not satisfy the determination condition at that time. Then, the pixel value stored in each pixel memory 414 for the unit block is output to the image processing unit 511 (S112).

  The image processing unit 511 generates an image corresponding to the entire imaging region based on the pixel values output in steps S108 and S112 (S114). Thus, the flowchart of FIG. 9 ends.

  For example, in the first imaging, a focus position is set in the left region of FIG. 8 in step S102, and an image 172 shown in FIG. 10 is obtained in step S104. Since this is the first imaging, all the unit blocks 131 in the imaging area are selected in step S103. Since the subject A is in the left area of FIG. 8, the subject A is in focus in the image 172, and has a high contrast value in the unit block group 132A. On the other hand, since the other subject B is out of the region, the other unit block group 132B has a low contrast value.

  In the first imaging, since there is no previous contrast value to be compared, in step S107, the maximum detection unit 464 does not determine that any unit block 131 has reached the maximum value. Therefore, the arithmetic circuit 415 writes the pixel value of the unit block 131 in the pixel memory 414 in step S109.

  In the next imaging, a focus position is set in the second region from the left in FIG. 8 in step S102, and an image 174 shown in FIG. 11 is obtained in step S104. Since the drive control unit 420 refers to the timing memory 430 and no unit block 131 stores information indicating that the determination condition is satisfied, charge accumulation and pixel signals are not stored in any unit block 131. Read is executed.

  The unit block group 132A having a high contrast value in FIG. 10 has a decreased contrast value in FIG. Therefore, the local maximum detection unit 464 determines in step S107 that the unit block 131 included in the unit block group 132A has taken the local maximum value in the previous imaging. Therefore, in step S108, the arithmetic circuit 415 reads out the pixel value of the unit block group 132A when the contrast value is maximum, that is, the pixel value of the first imaging from the pixel memory 414, and outputs it to the image processing unit 511. At the same time, the arithmetic circuit 415 writes the fact that the contrast value satisfies the determination condition together with information for specifying the unit block 131 in the timing memory 430 via the drive control unit 420.

  Note that in the image 174 of FIG. 11, subjects B and C are in focus and have a high contrast value in the unit block group 132A, but it is not yet determined that the maximum value has been reached. Since the subjects D and E are outside the region, the other unit block group 132B has a low contrast value. Therefore, the fact that the determination condition is satisfied is not written in the timing memory 430 for the unit block groups 132B, 132C, 132D, and 132E corresponding to these subjects B, C, D, and E. For the unit block groups 132B, 132C, 132D, and 132E, the pixel values for the current imaging are written into the pixel memory 414 while the pixel values for the previous imaging are retained.

  Further, in the next imaging, the in-focus position is set in the third area from the left in FIG. 8, and an image 176 shown in FIG. 12 is obtained. In this case, the drive control unit 420 refers to the timing memory 430 and does not accumulate charges and read pixel signals of the unit block group 132A, but accumulates charges and pixel signals of the unit blocks 131 other than the unit block group 132A. Is read out.

  Furthermore, since the contrast value of the unit block groups 132B and 132C is lower than the previous time, the local maximum detection unit 464 determines that the local image has the local maximum value in the previous imaging. Therefore, the arithmetic circuit 415 reads out the pixel values of the unit block groups 132B and 132C when the contrast value is maximum, that is, the pixel value of the previous imaging from the pixel memory 414, and outputs it to the image processing unit 511. At the same time, the arithmetic circuit 415 writes the fact that the contrast value satisfies the determination condition, together with information specifying the unit blocks 131B and 132C, to the timing memory 430 via the drive control unit 420.

  Further, in the next imaging, the in-focus position is set in the fourth area from the left in FIG. 8, and an image 178 shown in FIG. 13 is obtained. In this case, the drive control unit 420 refers to the timing memory 430, does not accumulate charges in the unit block groups 132A, 132B, and 132C, and does not read pixel signals, and unit blocks other than the unit block groups 132A, 132B, and 132C. 131 accumulates charges and reads out pixel signals.

  Furthermore, since the contrast value of the unit block group 132D is lower than the previous time, the local maximum detection unit 464 determines that the local image has the maximum value in the previous imaging. Further, the arithmetic circuit 415 reads out the pixel value of the unit block group 132D when the contrast value is maximum from the pixel memory 414, and outputs it to the image processing unit 511. At the same time, the arithmetic circuit 415 writes the fact that the contrast value satisfies the determination condition together with information for specifying the unit block 131D in the timing memory 430 via the drive control unit 420.

  Further, in the final imaging, the in-focus position is set in a region including infinity in FIG. 8, and an image 180 shown in FIG. In this case, the drive control unit 420 refers to the timing memory 430 and does not perform charge accumulation and pixel signal readout of the unit block group 132A or the like that has already been written to the effect that the determination condition has been satisfied. Accumulation of block charges and readout of pixel signals are performed. For convenience of explanation, it is determined in FIG. 13 that unit blocks other than the unit block group 132D have reached maximum values, and unit blocks that are not stored, read, or determined are indicated by hatching in FIG. Has been.

  Also in the image 176 of FIG. 14, it is assumed that the local maximum detection unit 464 cannot determine the local maximum value for the unit block group 132E. In this case, in step S112, the arithmetic circuit 415 reads out the pixel value of the current imaging in the unit block group 132E from the pixel memory 414, and outputs it to the image processing unit 511.

  Thereby, a pixel value is output from any unit block 131A or the like in the imaging region. The image processing unit 511 generates an image corresponding to the subject 170 based on the pixel value.

  As described above, according to the embodiment, it is possible to generate an in-focus image in each of the imaging regions by pressing the release button once. In this case, it is possible to generate an image without reducing the amount of exposure compared to pan-focusing with a larger aperture value. In addition, since the number of unit blocks 131 that store, read, and determine decreases as the image is captured, the processing speed can be increased and the power consumption can be reduced.

  Note that the number of regions from the imaging device 500 to infinity may be set in advance or may be set by the photographer. The areas may be equally divided. Alternatively, the system control unit 501 calculates the depth of field from the focal length and aperture value of the photographing lens 520, and determines any of the positions from the position of the imaging device 500. The focus position may be set so that the distance is included within the depth of field from any focus position. 9, the focus position is set from the side closer to the imaging device 500 to the side farther from the imaging device 500. However, even if the focus position is sequentially set from the infinity side toward the imaging device 500, the focus position is set. Good.

  When the image is generated in step S114, it is preferable to match the luminance by averaging the luminance values of at least several pixels at the boundary between the unit blocks where the pixel values are output by different imaging. It is preferable to make the boundary inconspicuous by adding blur, noise or the like to the boundary.

  Note that a unit block group 132A or the like corresponding to the subject A or the like may be selected in advance by face detection or the like, and the maximum contrast value may be evaluated for the entire unit block group 132A or the like. In this case, for the unit block 131 that did not become the maximum value in any imaging, the maximum value in the imaging when the contrast value of the selected unit block group 132 or the like A became the maximum was not reached. The pixel value of the unit block 131 may be output to the image processing unit 511.

  As another example, a unit block group 132A corresponding to the subject A or the like is selected in advance by face detection or the like, and the maximum contrast value is evaluated for each unit block 131 included in the unit block group 132A or the like. May be. In this case, for the unit blocks 131 other than the unit block group 132A and the like, the pixel value is output to the image processing unit 511 at the first shooting at the in-focus position including, for example, infinity, and steps S102 to S110 are repeated. You may decide not to.

  Instead of or in addition to the determination in step S110, the system control unit 501 repeats the subsequent steps when the number of unit blocks 131 determined to satisfy the determination condition exceeds a predetermined number. You may stop.

  Further, the evaluation value may be calculated by thinning out some unit blocks 131 among the plurality of unit blocks 131 included in the unit block group 132A. Further, in any of the above forms, the evaluation value may be calculated by thinning out pixels in each unit block 131.

  Further, when the image sensor 100 includes a phase difference pixel, it may be determined whether or not the phase difference pixel is in focus. In this case, the output of the phase difference pixel is an evaluation value, which is a condition for determining whether or not it is in focus.

  Further, instead of providing the individual circuit units 450A and the like for each unit block 131, the individual circuit units 450A and the like are provided for each of several unit blocks 131, and each evaluation value for each of the corresponding unit blocks 131 is obtained. May be calculated and determined. Alternatively, one individual circuit unit 450A may be provided for the plurality of unit blocks 131 included in the imaging region, and the evaluation values of the plurality of unit blocks 131 included in the imaging region may be calculated and determined. .

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. It will be apparent to those skilled in the art that various modifications or improvements can be added to the above-described embodiment. It is apparent from the scope of the claims that the embodiments added with such changes or improvements can be included in the technical scope of the present invention.

  The order of execution of each process such as operations, procedures, steps, and stages in the apparatus, system, program, and method shown in the claims, the description, and the drawings is particularly “before” or “prior to”. It should be noted that the output can be realized in any order unless the output of the previous process is used in the subsequent process. Regarding the operation flow in the claims, the description, and the drawings, even if it is described using “first”, “next”, etc. for convenience, it means that it is essential to carry out in this order. It is not a thing.

  100 imaging device, 101 microlens, 102 color filter, 103 passivation film, 104 PD, 105 transistor, 106 PD layer, 107 wiring, 108 wiring layer, 109 bump, 110 TSV, 111 signal processing chip, 112 memory chip, 113 imaging Chip, 131 unit block, 131A unit block, 131B unit block, 131C unit block, 131D unit block, 131E unit block, 132A unit block group, 132B unit block group, 132C unit block group, 132D unit block group, 132E unit block group , 170 subject, 172 images, 174 images, 176 images, 178 images, 180 images, 302 transfer transistors, 303 reset transistors, 04 amplification transistor, 305 selection transistor, 306 reset wiring, 307 TX wiring, 308 decoder wiring, 309 output wiring, 310 Vdd wiring, 311 load current source, 410 CDS circuit, 411 multiplexer, 412 A / D conversion circuit, 413 demultiplexer 414 pixel memory, 415 arithmetic circuit, 418 I / F circuit, 420 drive control unit, 430 timing memory, 441 sensor control unit, 442 block control unit, 443 synchronization control unit, 444 signal control unit, 450A individual circuit unit, 450B Individual circuit unit, 450C individual circuit unit, 450D individual circuit unit, 450E individual circuit unit, 460 high frequency component calculation unit, 462 contrast calculation unit, 464 maximum detection unit, 500 imaging device, 501 system control unit, 502 Moving parts, 503 photometric unit, 504 a working memory, 505 a recording unit, 506 display unit, 508 operation unit, 511 image processing unit, 512 operation unit, 514 drive, 520 a photographing lens

Claims (8)

For each of a plurality of regions obtained by dividing a plurality of imaging regions in which a plurality of pixels are arranged two-dimensionally, a reading unit that controls accumulation of pixels included in each of the plurality of regions and reads out pixel signals;
A calculation unit that calculates an evaluation value based on the pixel signal and determines whether or not the evaluation value satisfies a predetermined determination condition for each of the plurality of regions;
Among the plurality of regions, the pixel value based on the pixel signal is output to a region determined to satisfy the determination condition, and the evaluation value is determined for a region determined not to satisfy the determination condition A control unit that changes the imaging condition corresponding to and repeats the accumulation control and readout by the readout unit and the determination by the arithmetic unit,
An image pickup apparatus comprising: an image generation unit configured to generate an image corresponding to the image pickup region using the pixel value that has been determined and output for each of the plurality of regions to satisfy the determination condition.   The control unit performs the determination in an imaging condition when it is determined that another region satisfies the determination condition with respect to an region determined not to satisfy the determination condition even when repeated a predetermined number of times. The imaging device according to claim 1, wherein pixel values of pixels included in an area determined not to satisfy the condition are output.   When the number of regions determined to satisfy the determination condition among the plurality of regions exceeds a predetermined number, the control unit stops subsequent repetition and does not satisfy the determination condition. The imaging apparatus according to claim 1, wherein pixel values of pixels included in a region determined as being output.   4. The imaging apparatus according to claim 1, wherein one set of the reading unit, the calculation unit, and the control unit is provided for each of the plurality of regions. A storage unit that is provided for each of the plurality of regions and has a storage capacity corresponding to the number of pixels that is twice or more the number of pixels included in the corresponding region;
The imaging apparatus according to claim 4, wherein the control unit stores the pixel value in the storage unit for each reading.   The imaging apparatus according to claim 1, wherein the imaging condition is at least one of a focusing distance, an aperture value, a shutter speed, and sensitivity.   The imaging chip in which the imaging region is arranged and the signal processing chip in which at least a part of the readout unit, the arithmetic unit, the control unit, and the image generation unit are arranged are stacked. The imaging device according to any one of the above.   The imaging device according to claim 7, wherein the imaging chip is a backside illumination type CMOS chip.

Images