JP2015041854A - Image sensor and imaging apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reconcile image quality maintenance in the case of a global shutter (bulk storage) and distortion suppression in the case of a rolling shutter (sequential storage).SOLUTION: An image sensor includes photoelectric conversion parts each including a plurality of pixels arranged in the row and column directions, and having a plurality of unit blocks arranged in the row and column directions, a storage control part performing any one of bulk storage control for storing the pixel signals of all rows of a plurality of pixels collectively, for each of a plurality of unit blocks, or sequential storage control for storing the pixel signals of each row sequentially from a plurality of pixels, and a control selection part for selecting any one of the bulk storage control or sequential storage control for each of the plurality of unit blocks, and making the storage control part to run.

Description

  The present invention relates to an imaging element and an imaging apparatus.

When imaging is performed with a rolling shutter using an imaging element of a CMOS sensor, rolling shutter distortion may occur in the captured image (see, for example, Patent Document 1).
Patent Document 1 JP 2012-199802 A

  When shooting with a global shutter, rolling shutter distortion does not occur, but the effects of dark current, readout noise, and the like in the image sensor may appear in the captured image.

  According to the first aspect of the present invention, a photoelectric conversion unit including a plurality of unit blocks arranged in the row direction and the column direction, each including a plurality of pixels arranged in the row direction and the column direction, and the plurality of unit blocks For each of the above, accumulation is performed by performing either batch accumulation control that collectively accumulates pixel signals of all rows of a plurality of pixels or sequential accumulation control that sequentially accumulates pixel signals for each row from a plurality of pixels. An imaging device is provided that includes a control unit and a control selection unit that selects and executes either a batch accumulation control or a sequential accumulation control for each of a plurality of unit blocks.

  According to a second aspect of the present invention, there is provided an imaging device comprising the above imaging device.

  It should be noted that the above summary of the invention does not enumerate all the necessary features of the present invention. In addition, a sub-combination of these feature groups can also be an invention.

2 is a block diagram of an imaging apparatus 500. FIG. 1 is a cross-sectional view of an image sensor 100. FIG. 3 is a schematic diagram illustrating a layout of a light receiving substrate 113. FIG. It is a block diagram of a unit block 131 and related circuits. It is a circuit diagram of the pixel 150 alone. 3 is a timing chart showing the operation of the pixel 150. 3 is a block diagram of a driving unit 502. FIG. It is a schematic diagram which shows operation | movement of rolling shutter mode. 3 is a schematic diagram of a subject 300. FIG. It is a schematic diagram which shows the shape of the captured image 311. FIG. It is a schematic diagram which shows operation | movement of global shutter mode. 5 is a flowchart showing the operation of a control selection unit 474. 3 is a schematic diagram of a subject 300. FIG. It is a schematic diagram which shows the shape of the captured images 312,313,314. It is a schematic diagram which shows the shape of the picked-up image 322,323,324. 3 is a schematic diagram of a subject 300. FIG. It is a schematic diagram which shows the shape of the captured images 315, 316, and 317 before correction | amendment. 5 is a flowchart showing the operation of an image correction unit 476. It is a schematic diagram which shows the shape of the correction | amendment images 325 and 327. FIG.

  Hereinafter, the present invention will be described through embodiments of the invention, but the following embodiments do not limit the invention according to the claims. In addition, not all the combinations of features described in the embodiments are essential for the solving means of the invention.

  FIG. 1 is a block diagram of the imaging apparatus 500. The imaging apparatus 500 includes an optical system including an imaging lens 520, and an electronic circuit including an imaging element 100, a system control unit 501, a driving unit 502, a photometric unit 503, a work memory 504, a recording unit 505, and a display unit 506.

  The optical system including the imaging lens 520 guides the incident light beam propagating along the optical axis OA to the imaging device 100. In the drawing, a representative imaging lens 520 arranged in the vicinity of the pupil is representatively shown, but the optical system includes a plurality of optical lenses and forms an incident light beam on the imaging surface of the imaging device 100. . The imaging lens 520 may be an interchangeable type that can be attached to and detached from the imaging device 500.

  The drive unit 502 performs pixel signal accumulation control and readout control on the image sensor 100. Accordingly, the drive unit 502 performs pixel signal accumulation control, readout control, and the like on the image sensor 100 under the control of the system control unit 501.

  The photometric unit 503 has a light receiving element that receives a part of a light beam incident on the imaging lens 520 and detects luminance. As a result, the photometry unit 503 detects the luminance distribution and the like of the imaging scene. The photometry unit 503 may use a part of the image sensor 100 as a light receiving element.

  The work memory 504 is used as a work area when the system control unit 501 executes processing. The work memory 504 also functions as a buffer for information recorded in the recording unit 505, information displayed on the display unit 506, and the like.

  The system control unit 501 includes an image processing unit 511, a calculation unit 512, and the like. The image processing unit 511 performs image processing using the work memory 504 as a work area to generate image data. For example, when the image data to be generated is JPEG image data, white balance processing, gamma processing, compression processing, and the like are executed.

  Image data generated by the image processing unit 511 is recorded in the recording unit 505. Further, the generated image data may be displayed on the display unit 506. Furthermore, it may be transferred to an external storage unit such as a cloud server through a communication line. For example, the calculation unit 512 determines the shutter speed, aperture value, ISO sensitivity, and the like in the imaging operation of the imaging apparatus 500 based on the luminance information acquired by the system control unit 501 from the photometry unit 503.

  The image sensor 100 generates a pixel signal corresponding to the light beam incident on the imaging lens 520. The pixel signal generated by the image sensor 100 becomes image data in the image processing unit 511 of the system control unit 501.

  FIG. 2 is a cross-sectional view of the image sensor 100. The image sensor 100 includes a light receiving substrate 113, a signal processing substrate 111, and a memory substrate 112 that are stacked on each other. The light receiving substrate 113, the signal processing substrate 111, and the memory substrate 112 are electrically connected to each other by a conductive bump 109 such as Cu.

  Note that the subject light flux is incident on the image sensor 100 in the plus direction of the Z axis, as indicated by the white arrow in the figure. In the present embodiment, the surface on which the subject luminous flux is incident on the light receiving substrate 113 is referred to as the back surface in the following description. In addition, as indicated by the coordinate axes in the figure, the right direction in the figure orthogonal to the Z axis is described as the X axis plus direction, and the direction perpendicular to the Z axis and X axis and intersecting the paper surface toward the front is described as the Y axis plus direction. To do.

  The light receiving substrate 113 includes a photoelectric conversion element layer 106 and a wiring layer 108 as electrical elements. The photoelectric conversion element layer 106 is disposed on the back side of the wiring layer 108. The photoelectric conversion element layer 106 includes a plurality of photodiodes 104 and a plurality of transistors 105 arranged two-dimensionally. Each of the plurality of transistors 105 is provided corresponding to one of the plurality of photodiodes 104.

  The photoelectric conversion element layer 106 includes a photoelectric conversion element that outputs a pixel signal corresponding to incident light. An example of the light receiving substrate 113 is a back-illuminated CMOS image sensor, and the photoelectric conversion element layer 106 is provided with a photodiode 104 as a photoelectric conversion element.

  In the light receiving substrate 113, the wiring layer 108 includes a wiring 107 that transmits a pixel signal from the photoelectric conversion element layer 106 to the signal processing substrate 111. The wiring 107 may be arranged in multiple layers, and may further include a passive element and an active element.

  A plurality of bumps 109 are disposed on the surface of the wiring layer 108. The plurality of bumps 109 are aligned with the plurality of bumps 109 provided on the opposing surface of the signal processing substrate 111, and the light receiving substrate 113 and the signal processing substrate 111 are aligned by being pressed or the like. The bumps 109 are joined and electrically connected.

  The light receiving substrate 113 includes a color filter 102 and a microlens 101 as optical elements. In the light receiving substrate 113, the color filter 102 is disposed on the incident light incident side of the photoelectric conversion element layer 106 via the passivation film 103. Therefore, incident light enters each of the plurality of photodiodes 104 through the color filter 102.

  The color filter 102 includes a plurality of types that transmit different wavelength regions. The different types of color filters 102 have a specific arrangement corresponding to each of the photodiodes 104. A set of the color filter 102, the photodiode 104, and the transistor 105 forms one pixel.

  The microlens 101 is arranged corresponding to each pixel on the incident light incident side with respect to the color filter 102. Each of the microlenses 101 collects incident light toward the corresponding photodiode 104.

  The signal processing substrate 111 processes the pixel signal acquired from the light receiving substrate 113. The memory substrate 112 stores pixel signals. A plurality of bumps 109 are disposed on the surfaces of the signal processing substrate 111 and the memory substrate 112 facing each other. The bumps 109 are aligned with each other, and the signal processing substrate 111 and the memory substrate 112 are pressurized, so that the aligned bumps 109 are joined and electrically connected.

  The bonding between the bumps 109 is not limited to Cu bump bonding by solid phase diffusion, and micro bump bonding by solder melting may be employed. Further, for example, about one bump 109 may be provided for one output wiring described later. Therefore, the size of the bump 109 may be larger than the pitch of the photodiode 104. Further, a bump larger than the bump 109 corresponding to the pixel region may be provided in a peripheral region other than the pixel region where the pixels are arranged.

  The signal processing substrate 111 has TSVs (silicon through electrodes) 110 that connect circuits provided on the front and back surfaces to each other. The TSV 110 is preferably provided in the peripheral area. The TSV 110 may also be provided in the peripheral region of the light receiving substrate 113 and the memory substrate 112.

  FIG. 3 is a schematic diagram illustrating a layout of the pixels 150 and the unit blocks 131 in the photoelectric conversion element layer 106 of the image sensor 100. The layout in the figure shows a state in which the light receiving substrate 113 is viewed from the back side.

  A plurality of unit blocks 131 arranged in the row direction and the column direction are arranged in the photoelectric conversion element layer 106. Further, each unit block 131 includes a plurality of pixels 150 arranged in the same row direction and column direction as the unit block 131. Therefore, 20 million or more pixels 150 are arranged in a matrix on the image sensor 100.

  In the illustrated example, each of the unit blocks 131 is formed by 16 pixels 150 of 4 pixels × 4 pixels adjacent to each other. As shown in the enlarged partial view in the figure, the 16 pixels 150 forming the unit block 131 are, for example, four pixels of green pixels Gb, Gr, blue pixels B, and red pixels R according to the Bayer array. Either one is assigned.

  The green pixels Gb and Gr have a green filter as the color filter 102 and receive light in the green wavelength band of incident light. Similarly, the blue pixel B has a blue filter as the color filter 102 and receives light in the blue wavelength band, and the red pixel R has a red filter as the color filter 102 and receives light in the red wavelength band. .

  Of course, the number of pixels 150 forming the unit block 131 is not limited to the above. Further, the shape of the unit block 131 formed by the plurality of pixels 150 is not limited to a square. Therefore, when the total number of pixels of the light receiving substrate 113 is about 20 million pixels, the layout of the unit blocks 131 may be, for example, 64 rows and 32 columns, 48 rows and 114 columns, and the like.

  FIG. 4 shows an outline of the connection relationship for one of the unit blocks 131 formed in the image sensor 100. The illustrated unit block 131 includes pixels 150 of L rows and P columns, to which the row control unit 200 and the peripheral circuit 133 are connected.

  In the image sensor 100, the row control unit 200 is coupled to the unit block 131 through the wiring Rst_l (where l is an integer from 1 to L), the wiring Tx_l, and the wiring Sel_l. Each of the wiring Rst_l, the wiring Tx_l, and the wiring Sel_l is commonly connected to the P pixels 150 in the first row in the unit block 131. Similarly, the wiring Tx_l and the wiring Sel_l are also commonly connected to the P pixels 150 in the l-th row in the unit block 131.

  The row control unit 200 may be called a row selection unit, a vertical scanning circuit, or the like. The row control unit 200 is provided for each unit block 131. In the image sensor 100, the row control unit 200 may be provided on the signal processing board 111 side.

  In the image sensor 100, the peripheral circuit 133 is coupled to the unit block 131 through the column transmission path 170_p (where p is an integer from 1 to P). The column transmission path 170_p is provided for each pixel 150 in the same column.

  The column transmission path 170_p is connected in common to the L pixels 150 in the p-th column in the unit block 131. Accordingly, the column transmission path 170 is shared by the pixels 150 in the same column in the unit block 131, and transmits the pixel signal from the pixels 150 included in the column.

  In the image sensor 100, the column transmission path 170_p is connected to the peripheral circuit 133 mounted on the signal processing board 111 through the bump 109 from the light receiving board 113 side. The peripheral circuit 133 in the signal processing board 111 is arranged for each unit block 131 at a position corresponding to the unit block 131 in the light receiving board 113.

  Each of the peripheral circuits 133 includes an analog / digital conversion circuit 202 and a CDS circuit 204 connected in series for each column transmission path 170_p. P sets of analog / digital conversion circuits 202 and CDS circuits 204 are provided per unit block 131, which is the same as the number of columns.

  The peripheral circuit 133 further includes a shift register 206 disposed on the output side of the CDS circuit 204. In the illustrated example, the shift register 206 is provided for each unit block 131. The output of the shift register 206 is connected to the pixel memory 414 through the column bus line 172. The shift register 206 may be called a horizontal scanning circuit, a multiplexer, or the like.

  FIG. 5 is a circuit diagram showing the structure of the pixel 150 forming the unit block 131. However, the illustrated circuit includes two photodiodes 104-1 and 104-2 and corresponds to two pixels 150. These two pixels are included in the same column in the unit block 131.

  The photodiodes 104-1 and 104-2 have their cathodes individually coupled to one ends of the selection transistors 61-1 and 61-2 arranged corresponding to the photodiodes 104-1 and 104-2, respectively. Each of the photodiodes 104-1 and 104-2 accumulates electric charge according to the amount of received light.

  The selection transistors 61-1 and 61-2 switch whether to output a voltage corresponding to the amount of charge accumulated in the photodiodes 104-1 and 104-2. For example, the conduction state of the selection transistor 61 is controlled by a pixel selection signal provided through the wiring Tx_l provided from the row control unit 200.

  Further, the pixel 150 includes a reset transistor 59, an amplification transistor 60, a ground side transistor 65, a first transistor 78, a second transistor 63, an amplification transistor 66, and a transfer transistor 64 provided in common to the photodiodes 104-1 and 104-2. And a capacitor 10. These commonly provided transistors function as a switch unit that controls the state of the capacitor 10 (a state indicating what voltage is applied to both electrodes).

  In the illustrated example, the pixel 150 sequentially holds the feedthrough voltage and the signal voltage at the outputs of the photodiodes 104-1 and 104-2 by the single capacitor 10. Thereby, the area of the CDS circuit of each pixel 150 can be reduced while providing the CDS circuit in each of the pixels 150. Further, since the feedthrough voltage and the signal voltage are held by the common capacitor 10, the pixel value can be detected with high accuracy.

  The reset transistor 59 is connected to the photodiode 104 via each selection transistor 61. The reset transistor 59 switches whether to connect the cathodes of the photodiodes 104-1 and 104-2 to the reference voltage VCC. The reset transistor 59 is controlled by a reset signal applied through the wiring Rst_l of the row control unit 200.

  The gate of the amplification transistor 60 is connected to the photodiodes 104-1 and 104-2 via the selection transistors 61-1 and 61-2, respectively. The amplification transistor 60 amplifies the voltage output from one of the photodiodes 104-1 and 104-2 selected when the selection transistor 61 is turned on. The amplification transistor 60 is disposed between the reference voltage VCC and the ground potential.

  A ground-side transistor 65 is disposed between the amplification transistor 60 and the ground potential. A P control voltage is applied to the gate of the ground side transistor 65.

  The capacitor 10 has three electrodes, the first electrode and the second electrode are arranged to face each other, and the second electrode and the third electrode are arranged to face each other. As a result, the capacitor 10 forms a three-terminal capacitor, and is charged and discharged with a voltage corresponding to the amount of charge accumulated in the photodiodes 104-1 and 104-2.

  The first electrode of the capacitor 10 is connected to the output terminal of the amplification transistor 60 through the first transistor 78. The first transistor 78 functions as a first switch that switches whether the outputs of the photodiodes 104-1 and 104-2 amplified by the amplification transistor 60 are connected to the first electrode. The first transistor 78 is controlled by a first control signal SH1 applied to the gate.

  The second electrode of the capacitor 10 is connected to the output terminal of the amplification transistor 60 via the second transistor 63. The second transistor 63 functions as a second switch for switching whether or not the output of the photodiode 104 amplified by the amplification transistor 60 is connected to the second electrode. The second transistor 63 is controlled by a second control signal SH2 applied to the gate.

  The third electrode of the capacitor 10 is connected to the reference potential. Due to the capacitance between the second electrode and the third electrode, the potential of the second electrode with respect to the reference potential is maintained. Capacitor 10 includes a first electrode, a second electrode, and a third electrode that are formed in the same region. Therefore, the mounting area can be reduced as compared with the case of using a plurality of two-terminal capacitors. However, capacitor 10 may be formed using two elements, a two-terminal capacitor provided between node B and node C, and a two-terminal capacitor provided between node B and the reference potential.

  The amplification transistor 66 amplifies and outputs the voltage at the first electrode of the capacitor 10. The transfer transistor 64 functions as a transfer unit that switches whether to transfer the voltage output from the amplification transistor 66 to the read line 170-p. The transfer transistor 64 is controlled by a row selection signal supplied from the row control unit 200 through the wiring Sel_l.

  The capacitor 10 sequentially transitions between a first state in which a feedthrough voltage is applied to the second electrode and a second state in which a signal voltage is applied to the second electrode, in accordance with each control signal given from the row control unit 200. Let The peripheral circuit 133 turns on the transfer transistor 64 at least in the second state, and transfers the voltage level of the first electrode of the capacitor 10. Thereby, the peripheral circuit 133 can calculate the pixel value of the pixel signal based on the transferred voltage level.

  FIG. 6 is a timing chart showing the operation of the pixel 150, and shows signal waveforms at the nodes A, B, and C shown in FIG. Node A corresponds to the output terminal of the amplification transistor 60, node B corresponds to the second electrode of the capacitor 10, and node C corresponds to the first electrode of the capacitor 10.

  A case where the output of the photodiode 104-1 is read will be described. First, the reset signal RST becomes H level, and the reset transistor 59 is turned on. Further, the selection transistor 61-1 is controlled to be on. As a result, the cathode voltage of the photodiode 104-1 is reset, and the voltage at the node A becomes the reference voltage VCC. The amplification factors of the amplification transistors 60 and 66 are 1.

  When time elapses after resetting, the voltage at the node A becomes the feedthrough voltage Vft. The row control unit 200 sets the first control signal SH1 and the second control signal SH2 to the H level after a predetermined time has elapsed after receiving the reset signal RST. As a result, both the first transistor 78 and the second transistor 63 are turned on. At this time, the reference voltage VCC is applied to the first electrode (node C) of the capacitor 10, and the feedthrough voltage Vft is applied to the second electrode (node B) (first state).

  Next, the first control signal SH1 and the second control signal SH2 are set to L level. Thereby, the 1st electrode of the capacitor | condenser 10 will be in a floating state, and the voltage between electrodes of the capacitor | condenser 10 is maintained.

  Subsequently, the voltage at the node A becomes a voltage according to the voltage sig1 according to the amount of accumulated charge in the photodiode 104-1, at the timing when the light reception of the frame by the photodiode 104-1 is completed. Specifically, the voltage is lower than the feedthrough voltage Vft by the voltage sig1.

  Further, the second control signal SH2 is set to H level while maintaining the first control signal SH1 at L level. As a result, the second transistor 63 is turned on (second state).

  Since the first control signal SH1 is maintained at the L level, when the capacitor 10 transitions to the second state, the first electrode enters a floating state. Thereby, the signal voltage output from the photodiode 104-1 is applied to the second electrode while maintaining the voltage between the electrodes of the capacitor 10 in the first state. Since the voltage of the second electrode is decreased by the voltage sig1 compared to the first state, the voltage of the first electrode is also decreased by the voltage sig1 compared to the first state.

  The peripheral circuit 133 reads the voltage of the first electrode (node C) at least in the second state. Note that since the voltage of the first electrode in the first state is the reference voltage VCC, the peripheral circuit 133 may not read the voltage of the first electrode in the first state when the reference voltage is known. In this case, the peripheral circuit 133 detects the voltage sig1 by subtracting the voltage of the first electrode in the second state from the known reference voltage VCC.

  The peripheral circuit 133 may correct the value of the known reference voltage VCC according to the temperature of the image sensor 100 or the like. The peripheral circuit 133 may read the voltage of the first electrode (node C) and calculate the difference in each of the first state and the second state. As a result, the voltage sig1, that is, the pixel value corresponding to the accumulated charge amount in the photodiode 104-1, is calculated.

  Next, the output of the photodiode 104-2 is read. At this time, the reset signal RST becomes H level, and the reset transistor 59 is turned on. The selection transistor 61-2 is controlled to be in an on state. As a result, the cathode voltage of the photodiode 104-2 is reset, and the voltage at the node A becomes the reference voltage VCC. The subsequent operation is the same as that of the photodiode 104-1. The peripheral circuit 133 calculates a voltage sig2 corresponding to the amount of charge accumulated in the photodiode 104-2 based on the voltage read in the second state.

  With such a configuration and operation, the output of the photodiodes 104-1 and 104-2 can be correlated and sampled using one capacitor 10. As a result, the circuit area can be reduced, and errors due to capacitor variations can be eliminated. In the present embodiment, the two photodiodes 104 share one capacitor 10 and the peripheral switch portion, so that the circuit area can be further reduced.

  The capacitor 10 and the switch unit are provided corresponding to each photodiode 104. However, as illustrated, a set of the capacitor 10 and the switch unit is provided for the plurality of photodiodes 104-1, 104-2. You may provide in common. One set of capacitor 10 and switch unit may be provided for one photodiode 104.

  The switch unit is provided for each capacitor 10. When the capacitor 10 is shared by two or more photodiodes 104, the capacitor 10 holds the output values of the respective photodiodes 104-1 and 104-2 in order. The plurality of capacitors 10 may simultaneously hold the output values of the corresponding photodiodes 104-1 and 104-2.

  FIG. 7 is a block diagram of the drive unit 502 that drives the image sensor 100. The drive unit 502 includes a drive control unit 420, a sensor control unit 441, a block control unit 442, a synchronization control unit 443, a signal control unit 444, and a pixel memory 414.

  In the driving unit 502, the sensor control unit 441 transmits a control signal related to charge accumulation and charge readout of each pixel in the pixel 150 of the image sensor 100 to the light receiving substrate 113. More specifically, the sensor control unit 441 transmits a reset signal and a pixel selection signal to the row control unit 200 corresponding to each of the unit blocks 131, and starts and ends charge accumulation of the pixel 150 to be controlled. And control. Further, by transmitting a column selection signal or the like to the readout pixel, the pixel signal is output to the column transmission path 170.

  In the drive unit 502, the block control unit 442 transmits a control signal for specifying the unit block 131 to the light receiving substrate 113. More specifically, the block control unit 442 transmits a specific signal that specifies the unit block 131 to be controlled.

  The unit block specifying signal is a logical product of the selection signal sent from the sensor control unit 441 and the specific signal sent from the block control unit 442 for the transfer pulse received by each of the pixels 150 via the wiring Tx_j and the like. Thereby, the pixels 150 arranged substantially evenly on the light receiving substrate 113 can be controlled as independent blocks for each region.

  The block control unit 442 can simultaneously transmit measurement signals to the plurality of unit blocks 131. Thereby, the image sensor 100 can operate the plurality of unit blocks 131 in synchronization, or can execute the operation across the plurality of unit blocks 131.

  In the drive unit 502, the synchronization control unit 443 supplies a synchronization signal to the light receiving substrate 113. Thereby, a plurality of elements mounted on the light receiving substrate 113 can be activated synchronously. Therefore, for example, by adjusting the synchronization signal, it is possible to execute random control, thinning control, or the like that controls only specific pixels of pixels belonging to the same unit block 131.

  In the driving unit 502, the signal control unit 444 performs timing control for the analog / digital conversion circuit 202, the CDS circuit 204, and the shift register 206. Thereby, the image processing unit 511 can generate one image data at high speed and high quality from the pixel signals read from the plurality of unit blocks 131.

  In the driving unit 502, the pixel memory 414 has a memory space for storing pixel signals read from the pixels 150 and digitized. The pixel memory 414 may be provided corresponding to each unit block 131. The I / F circuit 418 is responsible for communication between the drive control unit 420 and the system control unit 501.

  The pixel memory 414 may further include a data transfer interface that transfers the pixel signal to the image processing unit 511 in accordance with the delivery request. The data transfer interface is connected to a data transfer line connected to the image processing unit 511. Such a data transfer line is formed by a part of the data bus, for example, and a delivery request from the system control unit 501 to the drive control unit 420 is executed by addressing using the address bus.

  In addition to the addressing method, the pixel signal is transmitted by the data transfer interface. For example, when data is transferred, double data is obtained by using both rising and falling edges of a clock signal used for synchronizing each circuit. A rate system can also be used. Further, it is possible to adopt a burst transfer method that speeds up data transfer at a time by omitting a part of procedures such as address designation.

  In addition, the pixel signal transmission by the data transfer interface includes a bus system using a line connecting the control unit, the memory unit, and the input / output unit in parallel, a serial system for transferring data one bit at a time, etc. They can also be used in combination.

  By using the data transfer interface as described above, the image processing unit 511 can receive only the requested pixel signal. Thereby, for example, when a low-resolution image is formed, the image processing can be completed at high speed. Note that the driving unit 502, the row control unit 200 in FIG. 5, and the peripheral circuit 133 function as a reading unit that sequentially reads pixel signals from the pixels 150 across the plurality of unit blocks 131.

  In the drive unit 502, the drive control unit 420 refers to the timing memory 430, converts the instruction received by the drive unit 502 from the system control unit 501 into a control signal that can be executed by each control unit, and delivers the converted control signal. The timing memory 430 is formed by a flash RAM or the like. In this way, the drive control unit 420 performs overall control of the sensor control unit 441, the block control unit 442, the synchronization control unit 443, and the signal control unit 444.

  Further, the illustrated driving unit 502 includes a correlation calculation unit 472, a control selection unit 474, and an image correction unit 476. The correlation calculation unit 472 detects rolling shutter distortion included in the pixel signal based on the pixel signal read from the pixel memory 414. The control selection unit 474 selects each operation mode of the unit block 131 based on the correlation value calculated by the correlation calculation unit 472. The image correction unit corrects rolling shutter distortion included in the pixel signal acquired from the unit block 131. The operations of the correlation calculation unit 472, the control selection unit 474, and the image correction unit 476 will be described later with reference to other drawings.

  FIG. 8 is a timing chart illustrating a rolling shutter mode, which is one operation mode of the unit block 131 in the image sensor 100. In the rolling shutter mode, in each of the unit blocks 131, the operation of selecting the pixel 150 and accumulating the pixel signal and the operation of reading the accumulated pixel signal are sequentially executed for each row.

  In the illustrated example, the drive unit 502 first supplies a drive signal to the wirings Rst_1 and Tx_1 for the pixels 150 in the first row via the row control unit 200. Thereby, charge accumulation of the pixels 150 in the first row of each unit block 131 is started.

  Next, the driving unit 502 supplies a driving signal to the wiring Sel_1 for the pixels 150 in the first row via the row control unit 200 at the end of the accumulation period. Thereby, reading of the pixel signal of the pixel 150 in the first row is started. The readout period is the time from when the pixel signal is read from each pixel 150 in the first row, processed by the analog / digital conversion circuit 202 and the CDS circuit 204, and sequentially written from the shift register 206 to the pixel memory 414. Including.

  Similarly, drive signals are applied to the wirings Rst_2, Tx_2, and Sel_2 for the pixels 150 in the second row, and accumulation and readout are performed. The length of the accumulation period for the pixels 150 in the second row is the same as that of the pixels 150 in the first row, but is delayed by about the readout period in time. From the third row to the Lth row, drive signals are given to the wirings Rst_l, Tx_l and Sel_l, and accumulation and reading are sequentially performed.

  The order of the selected rows is the same for any of the plurality of unit blocks 131. In the example shown in the figure, one row is selected from the -Y side row to the + Y side row. Is done. Further, the storage and reading timings of the same row are synchronized among the plurality of unit blocks 131. However, the timing of at least one of accumulation and reading of the same row does not have to be synchronized among the plurality of unit blocks 131.

FIG. 9 is a schematic diagram illustrating a subject 300 that is imaged in the rolling shutter mode using the image sensor 100. The subject 300 includes a vertical linear image 301 in the figure when the image light immediately before entering the image sensor 100 is viewed from the + Z side. Further, as indicated by an arrow in the figure, the straight line image 301 moves in the −X direction horizontally in the figure at a speed V ₀ .

  FIG. 10 is a schematic diagram showing an image obtained when the subject 300 including the moving linear image 301 is imaged in the rolling shutter mode. For the purpose of explanation, the subject 300 is drawn in the orientation when it is output, displayed, or the like as a captured image. In addition, it is assumed that the subject 300 is captured by four unit blocks 131 in the vertical and horizontal directions.

  As described with reference to FIG. 8, in the rolling shutter mode, the timing at which pixel signals are accumulated in each unit block 131 is sequentially shifted for each row. For this reason, the linear image 301 moving in the subject 300 is picked up at different positions according to the timing of accumulating pixel signals.

  In each of the unit blocks 131, the timing for accumulating pixel signals differs between the upper row in the figure and the lower row in the figure. For this reason, when the moving linear image 301 is captured, the obtained captured image 311 is inclined even though the linear image 301 is vertical. That is, in the captured image 311, the slower the pixel signal accumulation period is, the more the image is shifted in the + X direction, which is the moving direction, and rolling shutter distortion occurs for each unit block 131.

Note that the above-described shift means a displacement in each of the four unit blocks 131 with the row where the pixel signal is first accumulated as an initial position. In other words, it is considered that there is no shift in the row in which the pixel signal is first accumulated in each unit block 131. Therefore, in the entire image sensor 100, as indicated by a dotted line B in the drawing, a discontinuous shift occurs in the captured image 311 in the row of pixels 150 adjacent to each other at the boundary between adjacent unit blocks 131. Such rolling shutter distortion becomes more prominent as the horizontal component of the moving speed V ₀ of the linear image 301 increases.

  FIG. 11 is a timing chart illustrating a global shutter mode that is one operation mode of the unit block 131 in the image sensor 100. In the global shutter mode, all the pixels 150 included in the unit block 131 are simultaneously reset and then selected to accumulate pixel signals. In the global shutter mode, the accumulated pixel signals may be sequentially read out in units of rows as in the rolling shutter mode.

  In the global shutter mode, the timing of accumulating pixel signals in the unit block 131 is the same, so that the rolling shutter distortion does not occur in the captured image 311. However, since all the pixels 150 are reset at the same time, a large current flows, and charge leakage between adjacent rows is likely to occur. Therefore, compared to the rolling shutter mode, noise is more likely to be captured in the captured image. It has been known.

  FIG. 12 is a flowchart for explaining the operation of the control selection unit 474 in the drive unit 502. The control selection unit 474 operates the image sensor 100 in the rolling shutter mode initially (step S101). Thereby, in the drive part 502, the pixel memory 414 acquires a pixel signal (step S102).

  When the object 300 includes a moving object, rolling shutter distortion occurs in the captured image 311 obtained from the pixel signal of the image sensor 100. For this reason, in the boundary of the unit block 131, the captured image 311 does not continue in the row of the adjacent pixels 150. Therefore, in the drive unit 502, the correlation calculation unit 472 calculates the correlation of the captured image 311 obtained from the row of the adjacent pixels 150 at the boundary of the unit block 131 for each unit block 131 (step S103).

  Here, the control selection unit 474 compares the correlation value calculated by the correlation calculation unit 472 with a predetermined threshold value and checks whether the correlation value is larger than the threshold value (step S104). When it is found that the correlation value is larger than the threshold value in any unit block 131 (step S104 :: YES), that is, obtained from the row of adjacent pixels at the boundary between the unit block 131 and the adjacent unit block 131. If the captured images are continuous or substantially continuous, the control selection unit 474 returns the control to step S101 and continues imaging in the rolling shutter mode. In this way, in the unit block 131 that executes imaging in the rolling shutter mode, a high-quality pixel signal that is less affected by noise is output from the image sensor 100.

  On the other hand, when the calculated correlation value is not larger than the threshold value in step S104 (step S104: NO), the control selection unit 474 switches the operation of the image sensor 100 to the global shutter mode (step S105). Thus, in the unit block 131 that performs shooting in the global shutter mode, a pixel signal that forms a captured image 311 having no rolling shutter distortion is output from the image sensor.

  Note that the unit block 131 for which the control selection unit 474 has selected to operate in the rolling shutter mode through the above-described series of procedures is switched to the global shutter mode when rolling shutter distortion occurs in the captured image 311. . However, in the unit block 131 that has once started operating in the global shutter mode, it cannot be detected whether or not the subject 300 includes a moving object.

  Therefore, the control selection unit 474 monitors the duration of the global shutter mode for the unit block 131 that has selected to operate in the global shutter mode (step S106). Thereby, the unit block 131 continues the global shutter mode until the duration of the global shutter mode reaches a predetermined threshold (step S106: NO).

  On the other hand, when the duration of the global shutter mode reaches the threshold value (step S106: YES), the control selection unit 474 returns the control to step S101 and returns the operation of the unit block 131 to the rolling shutter mode again. Thereby, the control selection part 474 can judge again the operation | movement of the said unit block 131 according to the state of 300 which wants to know X at the time.

  It should be noted that the correlation value that is a material for determination in step S104 may be a difference between rows of adjacent pixels 150 on the boundary of the unit block 131. Further, it may be a correlation value calculated by a statistical process such as a residual sum of squares.

  Further, the duration of the global shutter mode may be the duration of the mode, the number of imaging frames, or the like. Further, for example, the threshold value of the global shutter mode duration may be changed according to the detected magnitude of the rolling shutter distortion.

FIG. 13 is a schematic diagram showing a subject 300 in which moving linear images 302 and 303 and a stationary linear image 304 are mixed. The subject 300 includes a linear image 302 that moves at a high speed at a speed V ₁ , a linear image 303 that moves at a low speed at a speed V ₂ , and a stationary linear image 304. Note that the straight images 302, 303, and 304 form vertical straight lines when the image light immediately before entering the image sensor 100 is viewed from the + Z side.

  FIG. 14 shows a captured image before correction when the subject 300 including the linear images 302, 303, and 304 as described above is captured in the rolling shutter mode in all the unit blocks 131. The control procedure shown in FIG. 12 corresponds to step S101. Note that the captured images 312, 313, and 314 are drawn in the direction in which they are output or displayed. In addition, it is assumed that the image sensor 100 is captured by twelve unit blocks 131 each including four vertical and horizontal pixels 150.

  A noticeable rolling shutter distortion occurs in the captured image 312 including the straight line image 302 that moves at high speed. For this reason, the correlation value of the row | line | column of the adjacent pixel 150 becomes low in the boundary of the unit block 131 arranged in the column direction. Therefore, the control selection unit 474 selects the imaging in the global shutter mode for the unit block that captures the captured image 312 including the straight line image 302.

  On the other hand, a slight rolling shutter distortion occurs in the captured image 313 including the straight line image 303 that moves at a low speed. For this reason, the correlation value of the row | line | column of the adjacent pixel 150 does not fall in the boundary of the unit block 131 arranged in the column direction. Therefore, the control selection unit 474 selects the imaging in the rolling shutter mode for the unit block that captures the captured image 313 including the straight line image 303. As a result, a slight rolling shutter distortion remains in the captured image 313, but the influence of noise is small.

  Furthermore, rolling shutter distortion does not occur in the captured image 314 including the stationary linear image 304. For this reason, the correlation value of the row | line | column of the adjacent pixel 150 does not fall at all in the boundary of the unit blocks 131 arranged in the column direction. Therefore, the control selection unit 474 selects the imaging in the rolling shutter mode for the unit block that captures the captured image 314 including the linear image 304. Thereby, the captured image 314 becomes a captured image with little influence of noise.

  In FIG. 15, captured images 322, 323, and 324 of the subject 300 obtained by the control selection unit 474 capturing some unit blocks 131 in the rolling shutter mode and other unit blocks capturing in the global shutter mode. It is a schematic diagram. As shown in the figure, the occurrence of rolling shutter distortion is prevented in the captured image 322 of the linear image 302 that moves at high speed. In the other captured images 323 and 324, a high-quality image with little influence of noise is obtained by the rolling shutter mode.

  As described above, the image pickup device 100 determines whether to operate in the rolling shutter mode or the global shutter mode for each unit block 131, thereby switching the rolling shutter distortion and maintaining the image quality. Can be compatible. When the unit block 131 includes special pixels such as a phase difference sensor for autofocus, it is preferable to remove the unit block 131 from the control target of the control switching unit 474 and always operate in the global shutter mode.

  In the example shown in FIG. 15, a slight rolling shutter distortion remains in the captured image 322. In the image sensor 100, rolling shutter distortion generated in the image of the unit block 131 can be removed at the stage of the image sensor 100.

  FIG. 16 shows another state of the subject 300 imaged by the image sensor 100. The illustrated subject 300 shows a state in which the image light immediately before entering the image sensor 100 is viewed from the + Z side.

  The illustrated subject 300 includes linear images 305 and 307 each extending in the Y direction and moving in the X direction, and a stationary linear image 306. Further, the linear image 305 moves faster than the linear image 307. However, in the illustrated example, it is assumed that the moving linear images 305 and 307 are moving at a speed within a range that is imaged in the rolling shutter mode.

  FIG. 17 shows a captured image when the subject 300 shown in FIG. 16 is imaged in the rolling shutter mode by the image sensor 100 and before the rolling shutter distortion is corrected. In the figure, the subject 300 imaged by the four unit blocks 131A, 131B, 131C, and 131D is shown in the direction in which the image is output, displayed, and the like as a captured image.

  In the area captured by the unit blocks 131A and 131B, the captured image 315 corresponding to the linear image 305 has rolling shutter distortion. For this reason, the captured image 315 is inclined obliquely. In other words, the longer the accumulation period, the greater the shift in the + X direction, which is the moving direction, in the captured image 315.

  Similarly, in the unit block 131D, the linear image 307 is distorted in the + X direction. However, the amount of distortion of the captured image 317 is small corresponding to the speed of the linear image 307 being lower than the speed of the linear image 305. In the unit blocks 131C and 131D, the rolling shutter distortion does not occur in the captured image 316 corresponding to the stationary linear image 306.

  Here, attention is paid to the row of adjacent pixels 150 at the boundary between adjacent unit blocks 131A and 131B in the same column. In the example shown in the drawing, attention is focused on the 10th row of the unit block 131A and the 1st row of the unit block 131B, which are surrounded by a dotted line B. Since these two rows are most distant from each other in the accumulation period, there is a shift at the boundary between the unit blocks 131A and 131C even though the straight line image 305 forming a continuous straight line is captured.

  FIG. 18 is a flowchart showing a control procedure for correcting rolling shutter distortion generated in the captured images 315 and 317 taken in the rolling shutter mode as shown in FIG. FIG. 19 is a schematic diagram showing a captured image corrected by the procedure shown in FIG. The flowchart in FIG. 18 is started when pixel signals corresponding to the pre-correction captured images 315, 316, and 317 shown in FIG. 17 are stored in the pixel memory 414.

  When correcting the rolling shutter distortion, the driving unit 502 first reads out the pixel signal of the boundary row in the unit blocks 131A and 131B adjacent in the same column from the pixel memory 414 in the correlation calculation unit 472 (step S201). For example, in the case of a set of unit blocks 131A and 131B, the pixel signal of the 10th row of the unit block 131A and the first row and the pixel signal of the unit block 131B are read out.

  Next, the correlation calculation unit 472 calculates the correlation value of the row of the adjacent pixels 150 at the boundary of the unit block 131 (step S202). It can be said that the magnitude of the correlation value is the degree of coincidence of the pixel signals in one column. The higher the correlation value, the more adjacent images match.

  As an example of the correlation value, the correlation calculation unit 472 squares the difference between the pixel signals of the pixels in the same column in the 10th row of the unit block 131A and the 1st row of the unit block 131B, and takes the sum of them. Calculate the sum of squared differences. Note that the smaller the residual sum of squares, the greater the correlation. Hereinafter, an example using the residual sum of squares as the correlation value will be described.

  The correlation calculation unit 472 determines whether the residual sum of squares is larger than the threshold (step S203). When the magnitude of the residual sum of squares is equal to or smaller than the threshold (step S203: NO), the flowchart in the unit blocks 131A and 131B adjacent in the same column is ended.

  When the residual sum of squares is smaller than the threshold, for example, in the example shown in FIG. 17, the tenth row on the unit block 131C side is later in time than the first row of the unit block 131D. It is estimated that the column position shift is small despite the accumulation and reading. Therefore, in the unit blocks 131C and 131D adjacent to each other in the same column, the unit block 131C having the tenth row that has been accumulated and read out later in time has no or no rolling shutter distortion. Is also estimated to be small.

  Therefore, the unit block 131C is temporarily excluded from the correction target in FIG. However, the captured image 316 of the unit block C may also be affected by the unit block 131A adjacent in the same column and become the correction block 132C.

  When the magnitude of the residual sum of squares is larger than the threshold value in the determination in step S203 (step S203: YES), the correlation calculation unit 472 performs one of the tenth row of the unit block 131A and the first row of the unit block 131B. Is shifted by one pixel in the row direction with respect to the other (step S204), that is, a residual sum of squares is calculated for pixels shifted by one column (step S205). The correlation calculation unit 472 repeats the above steps S204 and S205 for calculating the residual sum of squares by shifting by one pixel until the number of pixels p in the column is reached (step S206: NO). In this case, the pixels are shifted in the + X direction and the −X direction, respectively.

  The correlation calculation unit 472 specifies the number of pixels used for correction after shifting by p pixels (step S206: YES) (step S207). In this case, the correlation calculating unit 472 uses, for correction, the number of pixel shifts when the smallest residual sum of squares is calculated among the plurality of residual sums of squares calculated by repeating steps S204 to S206. The number of pixels. When there are a plurality of smallest residual sums of squares, the number of pixels is preferably smaller than the number of pixel shifts.

  In the example shown in FIG. 19, when the 10th row of the unit block 131A is shifted in the −X direction by 4 pixels with respect to the 1st row of the unit block 131B, the residual sum of squares becomes the smallest. Therefore, the correlation calculation unit 472 specifies “4” as the number of pixels used for correction.

  The image correction unit 476 corrects the pixel signal of the unit block 131A using the number of pixels specified in step S112 (step S208). Here, among the unit blocks 131A and 131B adjacent in the same column, the pixel signal of the unit block 131A having the tenth row that has been accumulated and read out in terms of time is corrected.

  In this case, in the unit block 131A, the image correction unit 476 equally allocates the shift amount to each row so that the 10th row is shifted by 4 pixels with respect to the first row, that is, by 4 columns. In the example of FIG. 11, a shift of one pixel is assigned to adjacent groups with two adjacent rows as one set.

  As illustrated in FIG. 19, the image correction unit 476 fixes the fifth and sixth rows in the center of the unit block 131A with respect to the other unit blocks 131B and the like, The correction block 132A is generated by replacing the pixel signal of the pixel whose position is shifted. Similarly, the correction blocks 132B and 132D are generated by shifting the column positions of the rows of the unit blocks 131A and 132D.

  The image correction unit 476 further corrects the pixel signal at the boundary between the adjacent unit blocks 131A and 131C in the same row. As shown in FIG. 13, when the correction blocks 132A and 132C are generated by correcting the adjacent unit blocks 131A and 131C in the same row, the overlapping pixel 142 in which the region of the blank pixel 140 and the pixels overlap each other are formed at the boundary between them. Resulting in a region. The image correction unit 476 assigns pixel signals from the periphery to the blank pixel 140 region. Further, the image correction unit 476 assigns the average value of the pixel signals of the respective pixels to the overlapping pixel 142.

  Thus, the correction block 132A and the like are generated. Here, although the pixel of the correction block 132C is not replaced by the correlation determination of step S104 with respect to the unit block 131C, the boundary is corrected under the influence of the correction from the adjacent correction block 132A. The image correction unit 476 outputs the pixel signal of the correction block 132A and the like to the pixel memory 414 (step S209), and ends this flowchart. As described above, rolling shutter distortion can be reduced with a simple configuration.

  In step S202, the correlation of the boundary rows in the adjacent unit blocks 131A and 131B in the same column is calculated. Instead, the correlation between the first row and the last row in the same unit block 131A may be calculated.

  In each of Steps S102 and S108, when the pixel signal has luminance and color signals, the correlation may be calculated using the luminance signal. Further, instead of this, the correlation may be calculated for pixels of the same color, for example, green pixels.

  Furthermore, instead of the residual sum of squares, a sum of absolute values of differences may be used as the correlation evaluation value. Further, instead of repeating p pixels in step S110, a so-called hill-climbing method may be used in which the repetition is stopped when a minimum value appears, as compared to calculating the residual sum of squares. .

  When the unit block 131 has a color filter such as a Bayer array, it is preferable to execute the above operation before performing the interpolation process. When the pixel signal subjected to analog / digital conversion is output as a captured image converted into a predetermined format such as JPEG, the operation in FIG. 12 is after the analog / digital conversion, It is preferably performed before converting to a format. Further, it is preferable that at least the correlation calculation unit 472 and the image correction unit 476 are provided in the signal processing board 111.

  As described above, in the image sensor 100, the generated rolling shutter distortion may be corrected for the unit block 131 photographed in the rolling shutter mode. However, in the image sensor 100, the unit block 131 that captures the linear image 302 that moves at high speed can be captured in the global shutter mode in which no rolling shutter distortion occurs. Therefore, the processing load caused by correcting the rolling shutter distortion is small.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. It will be apparent to those skilled in the art that various modifications or improvements can be added to the above-described embodiment. It is apparent from the scope of the claims that the embodiments added with such changes or improvements can be included in the technical scope of the present invention.

  The order of execution of each process such as operations, procedures, steps, and stages in the apparatus, system, program, and method shown in the claims, the description, and the drawings is particularly “before” or “prior to”. It should be noted that the output can be realized in any order unless the output of the previous process is used in the subsequent process. Regarding the operation flow in the claims, the description, and the drawings, even if it is described using “first”, “next”, etc. for convenience, it means that it is essential to carry out in this order. It is not a thing.

  10 capacitor, 60 amplification transistor, 61-1, 61-2 selection transistor, 59 reset transistor, 65 ground side transistor, 78 first transistor, 63 second transistor, 66 amplification transistor, 64 transfer transistor, 100 image sensor, 101 micro Lens, 102 Color filter, 103 Passivation film, 104, 104-1, 104-2 Photodiode, 105 Transistor, 106 Photoelectric conversion element layer, 107 Wiring, 108 Wiring layer, 109 Bump, 110 TSV, 111 Signal processing board, 112 Memory board, 113 light receiving board, 131 unit block, 132 correction block, 133 peripheral circuit, 140 blank pixel, 142 overlapping pixel, 150 pixel, 170 column transmission path, 172 column bus 200 row control unit 202 analog / digital conversion circuit 204 CDS circuit 206 shift register 300 subject 301 302 302 303 304 306 307 linear image 311, 312, 313, 314, 315 316, 317, 311, 322, 323, 324 Captured image, 325, 327 Corrected image, 414 Pixel memory, 418 I / F circuit, 420 Drive control unit, 430 Timing memory, 441 Sensor control unit, 442 Block control unit, 443 synchronization control unit, 444 signal control unit, 472 correlation calculation unit, 474 control selection unit, 476 image correction unit, 500 imaging device, 501 system control unit, 502 drive unit, 503 photometry unit, 504 work memory, 505 recording unit, 506 Display unit, 511 Image processing unit 512 arithmetic unit, 520 imaging lens

Claims (8)

A photoelectric conversion unit that includes a plurality of pixels arranged in the row direction and the column direction, each having a plurality of unit blocks arranged in the row direction and the column direction;
Collective accumulation control for collectively accumulating pixel signals of all rows of the plurality of pixels for each of the plurality of unit blocks, and sequential accumulation control for sequentially accumulating pixel signals for each row from the plurality of pixels. A storage control unit that executes any one of
An image pickup device comprising: a control selection unit that selects either the batch accumulation control or the sequential accumulation control for each of the plurality of unit blocks and causes the accumulation control unit to execute the selected one.   When the accumulation control unit continues the collective accumulation control beyond any of a predetermined number of times and a period for any of the plurality of unit blocks, the control selection unit The image pickup device according to claim 1, wherein the sequential accumulation control is executed on the unit block to select subsequent accumulation control in the unit block. When the accumulation control unit executes the sequential accumulation control in any of the plurality of unit blocks, the pixel signal in the row of the pixel in which the pixel signal is accumulated most recently in the unit block that has performed the sequential accumulation control, A correlation calculation unit that calculates a correlation value for a pixel signal in a row of pixels included in another unit block adjacent to the row of pixels;
The said control selection part selects the said batch accumulation control with respect to the said unit block, when the correlation value computed by the said correlation calculation part becomes lower than the predetermined threshold value. Image sensor.   Based on the correlation value calculated by the correlation calculation unit, the timing at which the plurality of pixel rows accumulate pixel signals with respect to the pixel signal acquired by the accumulation control unit from the unit block for which the correlation value is calculated. The imaging device according to claim 3, further comprising a pixel signal image correction unit that corrects distortion caused by the shift of the image signal. A plurality of pixels arranged in a row direction and a column direction; and a special pixel that is arranged in place of a part of the plurality of pixels and selectively receives a part of incident light. Special unit block arranged instead of a part,
A special accumulation control unit that performs the batch accumulation control on the special unit block;
The imaging device according to any one of claims 1 to 4, further comprising a special information acquisition unit that acquires special information other than luminance related to a subject based on a pixel signal read from the special pixel.   The image sensor according to claim 1, wherein the control selection unit is formed on another substrate stacked on the substrate on which the photoelectric conversion unit is formed.   The imaging device according to claim 1, wherein the photoelectric conversion unit includes a backside illumination type CMOS sensor that receives incident light through a substrate.   An imaging device comprising the imaging device according to any one of claims 1 to 7.

Images