JP6402806B2 - Imaging device - Google Patents

Description

  The present invention relates to an imaging element and an imaging apparatus.

There is a method of reducing the degradation of the quality of a moving image by acquiring and processing a plurality of images having different exposure times within one frame (for example, see Patent Document 1).
Patent document 1 JP2013-081060A

  The amount of pixel signals read from the image sensor increases significantly, and the processing load at the image processing stage also increases.

  According to the first aspect of the present invention, each of a plurality of unit blocks arranged in two dimensions, each including a plurality of pixels arranged two-dimensionally, and each of the plurality of unit blocks A correlation control unit that performs sequential accumulation control for sequentially accumulating pixel signals for each row from a plurality of pixels, and a correlation for calculating a correlation between captured images captured in adjacent pixel rows at a boundary of a plurality of unit blocks. An imaging device is provided that includes a calculation unit and an imaging control unit that individually determines a determination value of an imaging condition set in each of a plurality of unit blocks based on the correlation value calculated by the correlation calculation unit.

  According to a second aspect of the present invention, there is provided an imaging device comprising the above imaging device.

  It should be noted that the above summary of the invention does not enumerate all the necessary features of the present invention. In addition, a sub-combination of these feature groups can also be an invention.

2 is a block diagram of an imaging apparatus 500. FIG. 1 is a cross-sectional view of an image sensor 100. FIG. 3 is a schematic diagram illustrating a layout of a light receiving substrate 113. FIG. It is a block diagram of a unit block 131 and related circuits. It is a circuit diagram of the pixel 150 alone. 3 is a timing chart showing the operation of the pixel 150. 3 is a block diagram of a driving unit 502. FIG. It is a schematic diagram which shows operation | movement of rolling shutter mode. 3 is a schematic diagram of a subject 300. FIG. It is a schematic diagram which shows the shape of the captured images 311, 312, 313. 5 is a flowchart showing the operation of an imaging control unit 474. 5 is a flowchart showing the operation of a correlation calculation unit 472. 12 is a flowchart illustrating another operation of the imaging control unit 474. It is a figure which illustrates the contents of the table which imaging control part 474 refers. It is a schematic diagram which shows operation | movement of global shutter mode.

  Hereinafter, the present invention will be described through embodiments of the invention, but the following embodiments do not limit the invention according to the claims. In addition, not all the combinations of features described in the embodiments are essential for the solving means of the invention.

  FIG. 1 is a block diagram of the imaging apparatus 500. The imaging apparatus 500 includes an optical system including an imaging lens 520, and an electronic circuit including an imaging element 100, a system control unit 501, a driving unit 502, a photometric unit 503, a work memory 504, a recording unit 505, and a display unit 506.

  The optical system including the imaging lens 520 guides the incident light beam propagating along the optical axis OA to the imaging device 100. In the drawing, a representative imaging lens 520 arranged in the vicinity of the pupil is representatively shown, but the optical system includes a plurality of optical lenses and forms an incident light beam on the imaging surface of the imaging device 100. . The imaging lens 520 may be an interchangeable type that can be attached to and detached from the imaging device 500.

  The drive unit 502 performs pixel signal accumulation control and readout control on the image sensor 100. Accordingly, the drive unit 502 performs pixel signal accumulation control, readout control, and the like on the image sensor 100 under the control of the system control unit 501.

  The photometric unit 503 has a light receiving element that receives a part of a light beam incident on the imaging lens 520 and detects luminance. As a result, the photometry unit 503 detects the luminance distribution and the like of the imaging scene. The photometry unit 503 may use a part of the image sensor 100 as a light receiving element. In the illustrated imaging apparatus 500, the drive unit 502 also refers to the detection result of the photometry unit 503.

  The work memory 504 is used as a work area when the system control unit 501 executes processing. The work memory 504 also functions as a buffer for information recorded in the recording unit 505, information displayed on the display unit 506, and the like.

  The system control unit 501 includes an image processing unit 511, a calculation unit 512, and the like. The image processing unit 511 performs image processing using the work memory 504 as a work area to generate image data. For example, when the image data to be generated is JPEG image data, white balance processing, gamma processing, compression processing, and the like are executed.

  Image data generated by the image processing unit 511 is recorded in the recording unit 505. Further, the generated image data may be displayed on the display unit 506. Furthermore, it may be transferred to an external storage unit such as a cloud server through a communication line. For example, the calculation unit 512 determines the shutter speed, aperture value, ISO sensitivity, and the like in the imaging operation of the imaging apparatus 500 based on the luminance information acquired by the system control unit 501 from the photometry unit 503.

  The image sensor 100 generates a pixel signal corresponding to the light beam incident on the imaging lens 520. The pixel signal generated by the image sensor 100 becomes image data in the image processing unit 511 of the system control unit 501.

  FIG. 2 is a cross-sectional view of the image sensor 100. The image sensor 100 includes a light receiving substrate 113, a signal processing substrate 111, and a memory substrate 112 that are stacked on each other. The light receiving substrate 113, the signal processing substrate 111, and the memory substrate 112 are electrically connected to each other by a conductive bump 109 such as Cu.

  Note that the subject light flux is incident on the image sensor 100 in the plus direction of the Z axis, as indicated by the white arrow in the figure. In the present embodiment, the surface on which the subject luminous flux is incident on the light receiving substrate 113 is referred to as the back surface in the following description. In addition, as indicated by the coordinate axes in the figure, the right direction in the figure orthogonal to the Z axis is described as the X axis plus direction, and the direction perpendicular to the Z axis and X axis and intersecting the paper surface toward the front is described as the Y axis plus direction. To do.

  The light receiving substrate 113 includes a photoelectric conversion element layer 106 and a wiring layer 108 as electrical elements. The photoelectric conversion element layer 106 is disposed on the back side of the wiring layer 108. The photoelectric conversion element layer 106 includes a plurality of photodiodes 104 and a plurality of transistors 105 arranged two-dimensionally. Each of the plurality of transistors 105 is provided corresponding to one of the plurality of photodiodes 104.

  The photoelectric conversion element layer 106 includes a photoelectric conversion element that outputs a pixel signal corresponding to incident light. An example of the light receiving substrate 113 is a back-illuminated CMOS image sensor, and the photoelectric conversion element layer 106 is provided with a photodiode 104 as a photoelectric conversion element.

  In the light receiving substrate 113, the wiring layer 108 includes a wiring 107 that transmits a pixel signal from the photoelectric conversion element layer 106 to the signal processing substrate 111. The wiring 107 may be arranged in multiple layers, and may further include a passive element and an active element.

  A plurality of bumps 109 are disposed on the surface of the wiring layer 108. The plurality of bumps 109 are aligned with the plurality of bumps 109 provided on the opposing surface of the signal processing substrate 111, and the light receiving substrate 113 and the signal processing substrate 111 are aligned by being pressed or the like. The bumps 109 are joined and electrically connected.

  The light receiving substrate 113 includes a color filter 102 and a microlens 101 as optical elements. In the light receiving substrate 113, the color filter 102 is disposed on the incident light incident side of the photoelectric conversion element layer 106 via the passivation film 103. Therefore, incident light enters each of the plurality of photodiodes 104 through the color filter 102.

  The color filter 102 includes a plurality of types that transmit different wavelength regions. The different types of color filters 102 have a specific arrangement corresponding to each of the photodiodes 104. A set of the color filter 102, the photodiode 104, and the transistor 105 forms one pixel.

  The microlens 101 is arranged corresponding to each pixel on the incident light incident side with respect to the color filter 102. Each of the microlenses 101 collects incident light toward the corresponding photodiode 104.

  The signal processing substrate 111 processes the pixel signal acquired from the light receiving substrate 113. The memory substrate 112 stores pixel signals. A plurality of bumps 109 are disposed on the surfaces of the signal processing substrate 111 and the memory substrate 112 facing each other. The bumps 109 are aligned with each other, and the signal processing substrate 111 and the memory substrate 112 are pressurized, so that the aligned bumps 109 are joined and electrically connected.

  The bonding between the bumps 109 is not limited to Cu bump bonding by solid phase diffusion, and micro bump bonding by solder melting may be employed. Further, for example, about one bump 109 may be provided for one output wiring described later. Therefore, the size of the bump 109 may be larger than the pitch of the photodiode 104. Further, a bump larger than the bump 109 corresponding to the pixel region may be provided in a peripheral region other than the pixel region where the pixels are arranged.

  The signal processing substrate 111 has TSVs (silicon through electrodes) 110 that connect circuits provided on the front and back surfaces to each other. The TSV 110 is preferably provided in the peripheral area. The TSV 110 may also be provided in the peripheral region of the light receiving substrate 113 and the memory substrate 112.

  FIG. 3 is a schematic diagram illustrating a layout of the pixels 150 and the unit blocks 131 in the photoelectric conversion element layer 106 of the image sensor 100. The layout in the figure shows a state in which the light receiving substrate 113 is viewed from the back side.

  In the photoelectric conversion element layer 106, a plurality of unit blocks 131 arranged two-dimensionally in the row direction and the column direction are arranged. Further, each of the unit blocks 131 includes a plurality of pixels 150 that are two-dimensionally arranged in the same row direction and column direction as the unit block 131. Therefore, 20 million or more pixels 150 are two-dimensionally arranged on the image sensor 100.

  In the illustrated example, each of the unit blocks 131 is formed by 16 pixels 150 of 4 pixels × 4 pixels adjacent to each other. As shown in the enlarged partial view in the figure, the 16 pixels 150 forming the unit block 131 are, for example, four pixels of green pixels Gb, Gr, blue pixels B, and red pixels R according to the Bayer array. Either one is assigned.

  The green pixels Gb and Gr have a green filter as the color filter 102 and receive light in the green wavelength band of incident light. Similarly, the blue pixel B has a blue filter as the color filter 102 and receives light in the blue wavelength band, and the red pixel R has a red filter as the color filter 102 and receives light in the red wavelength band. .

  Of course, the number of pixels 150 forming the unit block 131 is not limited to the above. Further, the shape of the unit block 131 formed by the plurality of pixels 150 is not limited to a square. Therefore, when the total number of pixels of the light receiving substrate 113 is about 20 million pixels, the layout of the unit blocks 131 may be, for example, 64 rows and 32 columns, 48 rows and 114 columns, and the like.

  FIG. 4 shows an outline of the connection relationship for one of the unit blocks 131 formed in the image sensor 100. The illustrated unit block 131 includes pixels 150 of L rows and P columns, to which the row control unit 200 and the peripheral circuit 133 are connected.

  In the image sensor 100, the row control unit 200 is coupled to the unit block 131 through the wiring Rst_l (where l is an integer from 1 to L), the wiring Tx_l, and the wiring Sel_l. Each of the wiring Rst_l, the wiring Tx_l, and the wiring Sel_l is commonly connected to the P pixels 150 in the first row in the unit block 131. Similarly, the wiring Tx_l and the wiring Sel_l are also commonly connected to the P pixels 150 in the l-th row in the unit block 131.

  The row control unit 200 may be called a row selection unit, a vertical scanning circuit, or the like. The row control unit 200 is provided for each unit block 131. In the image sensor 100, the row control unit 200 may be provided on the signal processing board 111 side.

  In the image sensor 100, the peripheral circuit 133 is coupled to the unit block 131 through the column transmission path 170_p (where p is an integer from 1 to P). The column transmission path 170_p is provided for each pixel 150 in the same column.

  The column transmission path 170_p is connected in common to the L pixels 150 in the p-th column in the unit block 131. Accordingly, the column transmission path 170 is shared by the pixels 150 in the same column in the unit block 131, and transmits the pixel signal from the pixels 150 included in the column.

  In the image sensor 100, the column transmission path 170_p is connected to the peripheral circuit 133 mounted on the signal processing board 111 through the bump 109 from the light receiving board 113 side. The peripheral circuit 133 in the signal processing board 111 is arranged for each unit block 131 at a position corresponding to the unit block 131 in the light receiving board 113.

  Each of the peripheral circuits 133 includes an analog / digital conversion circuit 202 and a CDS circuit 204 connected in series for each column transmission path 170_p. P sets of analog / digital conversion circuits 202 and CDS circuits 204 are provided per unit block 131, which is the same as the number of columns.

  The peripheral circuit 133 further includes a shift register 206 disposed on the output side of the CDS circuit 204. In the illustrated example, the shift register 206 is provided for each unit block 131. The output of the shift register 206 is connected to the pixel memory 414 through the column bus line 172. The shift register 206 may be called a horizontal scanning circuit, a multiplexer, or the like.

  FIG. 5 is a circuit diagram showing the structure of the pixel 150 forming the unit block 131. However, the illustrated circuit includes two photodiodes 104-1 and 104-2 and corresponds to two pixels 150. These two pixels are included in the same column in the unit block 131.

  The photodiodes 104-1 and 104-2 have their cathodes individually coupled to one ends of the selection transistors 61-1 and 61-2 arranged corresponding to the photodiodes 104-1 and 104-2, respectively. Each of the photodiodes 104-1 and 104-2 accumulates electric charge according to the amount of received light.

  The selection transistors 61-1 and 61-2 switch whether to output a voltage corresponding to the amount of charge accumulated in the photodiodes 104-1 and 104-2. For example, the conduction state of the selection transistor 61 is controlled by a pixel selection signal provided through the wiring Tx_l provided from the row control unit 200.

  Further, the pixel 150 includes a reset transistor 59, an amplification transistor 60, a ground side transistor 65, a first transistor 78, a second transistor 63, an amplification transistor 66, and a transfer transistor 64 provided in common to the photodiodes 104-1 and 104-2. And a capacitor 10. These commonly provided transistors function as a switch unit that controls the state of the capacitor 10 (a state indicating what voltage is applied to both electrodes).

  In the illustrated example, the pixel 150 sequentially holds the feedthrough voltage and the signal voltage at the outputs of the photodiodes 104-1 and 104-2 by the single capacitor 10. Thereby, the area of the CDS circuit of each pixel 150 can be reduced while providing the CDS circuit in each of the pixels 150. Further, since the feedthrough voltage and the signal voltage are held by the common capacitor 10, the pixel value can be detected with high accuracy.

  The reset transistor 59 is connected to the photodiode 104 via each selection transistor 61. The reset transistor 59 switches whether to connect the cathodes of the photodiodes 104-1 and 104-2 to the reference voltage VCC. The reset transistor 59 is controlled by a reset signal applied through the wiring Rst_l of the row control unit 200.

  The gate of the amplification transistor 60 is connected to the photodiodes 104-1 and 104-2 via the selection transistors 61-1 and 61-2, respectively. The amplification transistor 60 amplifies the voltage output from one of the photodiodes 104-1 and 104-2 selected when the selection transistor 61 is turned on. The amplification transistor 60 is disposed between the reference voltage VCC and the ground potential.

  A ground-side transistor 65 is disposed between the amplification transistor 60 and the ground potential. A P control voltage is applied to the gate of the ground side transistor 65.

  The capacitor 10 has three electrodes, the first electrode and the second electrode are arranged to face each other, and the second electrode and the third electrode are arranged to face each other. As a result, the capacitor 10 forms a three-terminal capacitor, and is charged and discharged with a voltage corresponding to the amount of charge accumulated in the photodiodes 104-1 and 104-2.

  The first electrode of the capacitor 10 is connected to the output terminal of the amplification transistor 60 through the first transistor 78. The first transistor 78 functions as a first switch that switches whether the outputs of the photodiodes 104-1 and 104-2 amplified by the amplification transistor 60 are connected to the first electrode. The first transistor 78 is controlled by a first control signal SH1 applied to the gate.

  The second electrode of the capacitor 10 is connected to the output terminal of the amplification transistor 60 via the second transistor 63. The second transistor 63 functions as a second switch for switching whether or not the output of the photodiode 104 amplified by the amplification transistor 60 is connected to the second electrode. The second transistor 63 is controlled by a second control signal SH2 applied to the gate.

  The third electrode of the capacitor 10 is connected to the reference potential. Due to the capacitance between the second electrode and the third electrode, the potential of the second electrode with respect to the reference potential is maintained. Capacitor 10 includes a first electrode, a second electrode, and a third electrode that are formed in the same region. Therefore, the mounting area can be reduced as compared with the case of using a plurality of two-terminal capacitors. However, capacitor 10 may be formed using two elements, a two-terminal capacitor provided between node B and node C, and a two-terminal capacitor provided between node B and the reference potential.

  The amplification transistor 66 amplifies and outputs the voltage at the first electrode of the capacitor 10. The transfer transistor 64 functions as a transfer unit that switches whether to transfer the voltage output from the amplification transistor 66 to the read line 170-p. The transfer transistor 64 is controlled by a row selection signal supplied from the row control unit 200 through the wiring Sel_l.

  The capacitor 10 sequentially transitions between a first state in which a feedthrough voltage is applied to the second electrode and a second state in which a signal voltage is applied to the second electrode, in accordance with each control signal given from the row control unit 200. Let The peripheral circuit 133 turns on the transfer transistor 64 at least in the second state, and transfers the voltage level of the first electrode of the capacitor 10. Thereby, the peripheral circuit 133 can calculate the pixel value of the pixel signal based on the transferred voltage level.

  FIG. 6 is a timing chart showing the operation of the pixel 150, and shows signal waveforms at the nodes A, B, and C shown in FIG. Node A corresponds to the output terminal of the amplification transistor 60, node B corresponds to the second electrode of the capacitor 10, and node C corresponds to the first electrode of the capacitor 10.

  A case where the output of the photodiode 104-1 is read will be described. First, the reset signal RST becomes H level, and the reset transistor 59 is turned on. Further, the selection transistor 61-1 is controlled to be on. As a result, the cathode voltage of the photodiode 104-1 is reset, and the voltage at the node A becomes the reference voltage VCC. The amplification factors of the amplification transistors 60 and 66 are 1.

  When time elapses after resetting, the voltage at the node A becomes the feedthrough voltage Vft. The row control unit 200 sets the first control signal SH1 and the second control signal SH2 to the H level after a predetermined time has elapsed after receiving the reset signal RST. As a result, both the first transistor 78 and the second transistor 63 are turned on. At this time, the reference voltage VCC is applied to the first electrode (node C) of the capacitor 10, and the feedthrough voltage Vft is applied to the second electrode (node B) (first state).

  Next, the first control signal SH1 and the second control signal SH2 are set to L level. Thereby, the 1st electrode of the capacitor | condenser 10 will be in a floating state, and the voltage between electrodes of the capacitor | condenser 10 is maintained.

  Subsequently, the voltage at the node A becomes a voltage according to the voltage sig1 according to the amount of accumulated charge in the photodiode 104-1, at the timing when the light reception of the frame by the photodiode 104-1 is completed. Specifically, the voltage is lower than the feedthrough voltage Vft by the voltage sig1.

  Further, the second control signal SH2 is set to H level while maintaining the first control signal SH1 at L level. As a result, the second transistor 63 is turned on (second state).

  Since the first control signal SH1 is maintained at the L level, when the capacitor 10 transitions to the second state, the first electrode enters a floating state. Thereby, the signal voltage output from the photodiode 104-1 is applied to the second electrode while maintaining the voltage between the electrodes of the capacitor 10 in the first state. Since the voltage of the second electrode is decreased by the voltage sig1 compared to the first state, the voltage of the first electrode is also decreased by the voltage sig1 compared to the first state.

  The peripheral circuit 133 reads the voltage of the first electrode (node C) at least in the second state. Note that since the voltage of the first electrode in the first state is the reference voltage VCC, the peripheral circuit 133 may not read the voltage of the first electrode in the first state when the reference voltage is known. In this case, the peripheral circuit 133 detects the voltage sig1 by subtracting the voltage of the first electrode in the second state from the known reference voltage VCC.

  The peripheral circuit 133 may correct the value of the known reference voltage VCC according to the temperature of the image sensor 100 or the like. Further, the peripheral circuit 133 may read the voltage of the first electrode (node C) and calculate the difference in each of the first state and the second state. As a result, the voltage sig1, that is, the pixel value corresponding to the accumulated charge amount in the photodiode 104-1, is calculated.

  Next, the output of the photodiode 104-2 is read. At this time, the reset signal RST becomes H level, and the reset transistor 59 is turned on. The selection transistor 61-2 is controlled to be in an on state. As a result, the cathode voltage of the photodiode 104-2 is reset, and the voltage at the node A becomes the reference voltage VCC. The subsequent operation is the same as that of the photodiode 104-1. The peripheral circuit 133 calculates a voltage sig2 corresponding to the amount of charge accumulated in the photodiode 104-2 based on the voltage read in the second state.

  With such a configuration and operation, the output of the photodiodes 104-1 and 104-2 can be correlated and sampled using one capacitor 10. As a result, the circuit area can be reduced, and errors due to capacitor variations can be eliminated. In the present embodiment, the two photodiodes 104 share one capacitor 10 and the peripheral switch portion, so that the circuit area can be further reduced.

  The capacitor 10 and the switch unit are provided corresponding to each photodiode 104. However, as illustrated, a set of the capacitor 10 and the switch unit is provided for the plurality of photodiodes 104-1, 104-2. You may provide in common. One set of capacitor 10 and switch unit may be provided for one photodiode 104.

  The switch unit is provided for each capacitor 10. When the capacitor 10 is shared by two or more photodiodes 104, the capacitor 10 holds the output values of the respective photodiodes 104-1 and 104-2 in order. The plurality of capacitors 10 may simultaneously hold the output values of the corresponding photodiodes 104-1 and 104-2.

  FIG. 7 is a block diagram of the drive unit 502 that drives the image sensor 100. The drive unit 502 includes a drive control unit 420, a sensor control unit 441, a block control unit 442, a synchronization control unit 443, a signal control unit 444, and a pixel memory 414.

  In the driving unit 502, the sensor control unit 441 transmits a control signal related to charge accumulation and charge readout of each pixel in the pixel 150 of the image sensor 100 to the light receiving substrate 113. More specifically, the sensor control unit 441 transmits a reset signal and a pixel selection signal to the row control unit 200 corresponding to each of the unit blocks 131, and starts and ends charge accumulation of the pixel 150 to be controlled. And control. Further, by transmitting a column selection signal or the like to the readout pixel, the pixel signal is output to the column transmission path 170.

  In the drive unit 502, the block control unit 442 transmits a control signal for specifying the unit block 131 to the light receiving substrate 113. More specifically, the block control unit 442 transmits a specific signal that specifies the unit block 131 to be controlled.

  The unit block specifying signal is a logical product of the selection signal sent from the sensor control unit 441 and the specific signal sent from the block control unit 442 for the transfer pulse received by each of the pixels 150 via the wiring Tx_j and the like. Thereby, the pixels 150 arranged substantially evenly on the light receiving substrate 113 can be controlled as independent blocks for each region.

  The block control unit 442 can simultaneously transmit measurement signals to the plurality of unit blocks 131. Thereby, the image sensor 100 can operate the plurality of unit blocks 131 in synchronization, or can execute the operation across the plurality of unit blocks 131.

  In the drive unit 502, the synchronization control unit 443 supplies a synchronization signal to the light receiving substrate 113. Thereby, a plurality of elements mounted on the light receiving substrate 113 can be activated synchronously. Therefore, for example, by adjusting the synchronization signal, it is possible to execute random control, thinning control, or the like that controls only specific pixels of pixels belonging to the same unit block 131.

  In the driving unit 502, the signal control unit 444 performs timing control for the analog / digital conversion circuit 202, the CDS circuit 204, and the shift register 206. Thereby, the image processing unit 511 can generate one image data at high speed and high quality from the pixel signals read from the plurality of unit blocks 131.

  In the driving unit 502, the pixel memory 414 has a memory space for storing pixel signals read from the pixels 150 and digitized. The pixel memory 414 may be provided corresponding to each unit block 131. The I / F circuit 418 is responsible for communication between the drive control unit 420 and the system control unit 501.

  The pixel memory 414 may further include a data transfer interface that transfers the pixel signal to the image processing unit 511 in accordance with the delivery request. The data transfer interface is connected to a data transfer line connected to the image processing unit 511. Such a data transfer line is formed by a part of the data bus, for example, and a delivery request from the system control unit 501 to the drive control unit 420 is executed by addressing using the address bus.

  In addition to the addressing method, the pixel signal is transmitted by the data transfer interface. For example, when data is transferred, double data is obtained by using both rising and falling edges of a clock signal used for synchronizing each circuit. A rate system can also be used. Further, it is possible to adopt a burst transfer method that speeds up data transfer at a time by omitting a part of procedures such as address designation.

  In addition, the pixel signal transmission by the data transfer interface includes a bus system using a line connecting the control unit, the memory unit, and the input / output unit in parallel, a serial system for transferring data one bit at a time, etc. They can also be used in combination.

  By using the data transfer interface as described above, the image processing unit 511 can receive only the requested pixel signal. Thereby, for example, when a low-resolution image is formed, the image processing can be completed at high speed. Note that the driving unit 502, the row control unit 200 in FIG. 5, and the peripheral circuit 133 function as a reading unit that sequentially reads pixel signals from the pixels 150 across the plurality of unit blocks 131.

  In the drive unit 502, the drive control unit 420 refers to the timing memory 430, converts the instruction received by the drive unit 502 from the system control unit 501 into a control signal that can be executed by each control unit, and delivers the converted control signal. The timing memory 430 is formed by a flash RAM or the like. In this way, the drive control unit 420 performs overall control of the sensor control unit 441, the block control unit 442, the synchronization control unit 443, and the signal control unit 444.

  Further, the illustrated driving unit 502 includes a correlation calculating unit 472 and an imaging control unit 474. The correlation calculation unit 472 detects the magnitude of rolling shutter distortion included in the pixel signal based on the pixel signal read from the pixel memory 414. Based on the correlation value calculated by the correlation calculation unit 472, the imaging control unit 474 determines a value set as an imaging condition for each unit block 131, such as a frame rate and an exposure time. The operations of the correlation calculation unit 472 and the imaging control unit 474 will be described again with reference to other drawings.

  FIG. 8 is a timing chart for explaining the rolling shutter mode in the image sensor 100. In the rolling shutter mode, in each of the unit blocks 131, the operation of selecting the pixel 150 and accumulating the pixel signal and the operation of reading the accumulated pixel signal are sequentially executed for each row.

  In the illustrated example, the drive unit 502 first supplies a drive signal to the wirings Rst_1 and Tx_1 for the pixels 150 in the first row via the row control unit 200. Thereby, charge accumulation of the pixels 150 in the first row of each unit block 131 is started.

  Next, the driving unit 502 supplies a driving signal to the wiring Sel_1 for the pixels 150 in the first row via the row control unit 200 at the end of the accumulation period. Thereby, reading of the pixel signal of the pixel 150 in the first row is started. The readout period is the time from when the pixel signal is read from each pixel 150 in the first row, processed by the analog / digital conversion circuit 202 and the CDS circuit 204, and sequentially written from the shift register 206 to the pixel memory 414. Including.

  Similarly, drive signals are applied to the wirings Rst_2, Tx_2, and Sel_2 for the pixels 150 in the second row, and accumulation and readout are performed. The length of the accumulation period for the pixels 150 in the second row is the same as that of the pixels 150 in the first row, but is delayed by about the readout period in time. From the third row to the Lth row, drive signals are given to the wirings Rst_l, Tx_l and Sel_l, and accumulation and reading are sequentially performed.

  The order of the selected rows is the same for any of the plurality of unit blocks 131. In the example shown in the figure, one row is selected from the -Y side row to the + Y side row. Is done. Further, the storage and reading timings of the same row are synchronized among the plurality of unit blocks 131. However, the timing of at least one of accumulation and reading of the same row does not have to be synchronized among the plurality of unit blocks 131.

  FIG. 9 is a schematic diagram illustrating a subject 300 that is imaged in the rolling shutter mode using the image sensor 100. The subject 300 is shown in a state in which the image light immediately before entering the image sensor 100 is viewed from the + Z side.

The subject 300 includes a linear image 301 that moves at high speed at a speed V ₁ , a linear image 302 that moves at low speed at a speed V ₂ , and a stationary linear image 303. The straight images 301, 302, and 303 form vertical straight lines when the image light immediately before entering the image sensor 100 is viewed from the + Z side. Further, the moving linear images 301 and 303 are both moving in the −X direction horizontally in the figure.

  FIG. 10 is a schematic diagram showing captured images 311, 312, and 313 obtained when the subject 300 including the moving linear images 301, 302, and 303 is captured by the image sensor 100 operating in the rolling shutter mode. . The captured images 311, 312, and 313 are drawn in the direction in which they are output and displayed.

  Assume that the captured images 311, 312, and 313 are captured by twelve unit blocks 131 each including four pixels 150 in the vertical and horizontal directions. As described with reference to FIG. 8, in the rolling shutter mode, the timing at which pixel signals are accumulated in each unit block 131 is sequentially shifted for each row. For this reason, the linear images 301 and 303 that are moving in the subject 300 are picked up at different positions according to the timing at which the pixel signals are accumulated.

  In each of the column A and B unit blocks 131 that capture the moving linear images 301 and 302, the upper row in the figure and the lower row in the figure have different timings for storing pixel signals. For this reason, both the captured images 311 and 312 are tilted even though the linear image 301 in the subject 300 is a vertical straight line. That is, in the captured image 311, the slower the pixel signal accumulation period is, the more the image is shifted in the + X direction, which is the moving direction, and rolling shutter distortion occurs for each unit block 131.

  Note that the above-described shift means a displacement in each of the four unit blocks 131 with the row where the pixel signal is first accumulated as an initial position. In other words, it can be considered that there is no deviation in the row in which the pixel signal is first accumulated in each unit block 131 (the lowest row in the illustrated example). Therefore, in the entire image sensor 100, as indicated by a dotted line B in the drawing, a discontinuous shift occurs in the captured image 311 in the row of pixels 150 adjacent to each other at the boundary between adjacent unit blocks 131.

The shift due to the rolling shutter distortion as described above becomes more prominent as the horizontal component of the moving speed V ₀ of the linear images 301 and 302 increases. In other words, by detecting the shift amount of the captured images 311 and 312 at the boundary of the unit block 131, the moving speed of the linear images 301 and 302 that move in the subject 300 can be detected.

  FIG. 11 is a flowchart illustrating an operation procedure of the imaging control unit 474 in the driving unit 502. The illustrated procedure is started when the pixel signals corresponding to the captured images 311, 312, and 313 are stored in the pixel memory 414 from the unit block 131 to be controlled.

  First, the imaging control unit 474 selects one of the unit blocks 131 included in the imaging device 100 as a control target (step S101). Next, the imaging control unit 474 detects the moving speed of the moving component included in the subject 300 of the selected unit block 131 (step S102). The contents of step S102 will be described later with reference to FIG.

  When the moving speed of the moving component in the subject 300 is determined, the imaging control unit 474 determines the value of the shooting condition in the unit block 131 to be controlled according to the determined moving speed. The value of the shooting condition determined here is, for example, a frame rate when a moving image is captured using the image sensor 100.

  That is, when the moving component of the subject 300 is moving at a high speed, the frame rate in the unit block 131 is increased to capture a clearer image. On the other hand, when the moving speed of the moving component of the subject 300 is low or still, the information amount of the pixel signal output from the image sensor can be reduced by not increasing or rather decreasing the frame rate. In this case, even if the frame rate is reduced, the quality of the captured image does not deteriorate.

  Therefore, the imaging control unit 474 can calculate an appropriate frame rate in the unit block 131 to be controlled by acquiring the moving speed of the moving component of the subject 300 detected in step S102 (step S103). Further, the imaging control unit 474 sets the calculated frame rate in the unit block 131 to be controlled (step S104), and executes an imaging operation under appropriate imaging conditions.

  In this way, the imaging control unit 474 that has calculated and set the imaging condition value for one unit block 131 next checks whether or not there remains any unit block 131 for which the imaging condition has not yet been set (step). S105) When such a unit block 131 remains, the unit block 131 is specified as a control target (step S101), and the procedure from step S102 is executed again.

  If it is determined in step S105 that imaging conditions have been set for all unit blocks 131, the illustrated procedure is terminated. However, when the image sensor 100 continues image capturing, the series of procedures from step S101 may be repeated again.

  FIG. 12 is a flowchart showing an operation procedure when the moving speed of the moving component in the subject 300 is detected, and corresponds to the content of step S102. The illustrated procedure starts when a pixel signal from the image sensor 100 is stored in the pixel memory 414.

  Each of the unit blocks 131 generates a pixel signal in the rolling shutter mode. The correlation calculation unit 472 reads out, from the pixel memory 414, the pixel signal corresponding to the row of the pixels 150 adjacent at the boundary of the unit blocks 131 for the unit blocks 131 adjacent in the column direction (step S201).

  Next, the correlation calculation unit 472 calculates the correlation value of the row of the adjacent pixels 150 at the boundary of the unit block 131 (step S202). It can be said that the magnitude of the correlation value is the degree of coincidence of the pixel signals in one column. The higher the correlation value, the more adjacent images match.

  Examples of the correlation value include calculation of a residual square sum obtained by squaring the difference between pixel signals in adjacent pixel rows and taking the sum thereof. Here, the smaller the residual sum of squares, the greater the correlation.

  The correlation calculation unit 472 compares the residual sum of squares value with a predetermined threshold value, and determines whether rolling shutter distortion has occurred (step S203). When the residual sum of squares is larger than the threshold (step S203: NO), the correlation calculation unit 472 determines that the component of the subject 300 has not moved and ends the process.

  On the other hand, if the residual sum of squares is smaller than the threshold (step S203: YES), it is determined that the moving component of the subject 300 is moving, and a correlation value corresponding to the moving speed is calculated. That is, the correlation calculation unit 472 shifts one of the rows of adjacent pixels 150 at the boundary of the unit block 131 by one pixel in the row direction with respect to the other (step S204), and calculates the residual sum of squares again (step S204). Step S205). Thereafter, the correlation calculation unit 472 repeats the calculation of the residual sum of squares until the number of columns 150 of the pixels 150 in the unit block 131 is shifted (step S206: NO).

  After shifting by p pixels (step S206: YES), the correlation calculation unit 472 identifies the shift that has occurred at the boundary of the unit block 131 due to rolling shutter distortion as the number of pixels (step S207). In this way, the magnitude of the rolling shutter distortion caused by the moving component of the subject 300 in the unit block 131 is known, and the moving speed of the moving component can be detected.

  FIG. 13 is a flowchart illustrating another operation procedure of the imaging control unit 474 in the drive unit 502. The operation procedure shown has the same contents as the operation procedure shown in FIG. 11 except for the steps described below. Therefore, common steps are denoted by the same reference numerals, and redundant description is omitted.

  The operation procedure shown in FIG. 13 is the same as that shown in FIG. 11 in that step S106 in which the imaging control unit 474 also refers to the luminance information obtained from the photometry unit 503 is added before the imaging condition is determined in step S103. The procedure is different. That is, in the operation procedure illustrated in FIG. 13, the imaging control unit 474 refers to the subject luminance detected by the photometry unit 503 in addition to the moving speed of the moving component of the subject 300 detected from the calculation result of the correlation calculation unit 472. The imaging conditions are determined.

  In addition, the operation procedure shown in FIG. 13 is similar to that shown in FIG. 11 in step S107 in which, when the imaging control unit 474 determines the value of the imaging condition, the table is searched and the determined value is selected instead of step S103 in FIG. 11 is different from the operation procedure shown in FIG. That is, in the operation procedure shown in FIG. 13, the imaging control unit 474 searches for imaging conditions to be determined from candidate values stored in advance in the table using the acquired moving speed and subject brightness as keys.

  FIG. 14 is a diagram illustrating the contents of the table referred to by the imaging control unit 474 in step S107 as described above. As shown in the figure, the table shows two imaging conditions of the frame length and the exposure time length in the unit block 131 with respect to two reference conditions of the movement speed of the moving component in the subject 300 and the subject brightness. Store the values corresponding to each other. Thereby, even when the type of information to be referred to, the type of imaging condition to be determined, the number of unit blocks 131 to be controlled, and the like increase, the processing load can be suppressed.

  When the unit block 131 has a color filter such as a Bayer array, it is preferable to execute the above operation before performing the interpolation process. Further, when the pixel signal subjected to analog / digital conversion is output as a captured image converted into a predetermined format such as JPEG, the operation of FIG. 13 is after the analog / digital conversion, It is preferably performed before converting to a format. Further, it is preferable that at least the correlation calculation unit 472 and the imaging control unit 474 are provided in the signal processing board 111.

  As described above, in the image sensor 100, the setting value of the imaging condition can be changed for each unit block 131 according to the imaging target in the subject 300. Therefore, even when objects with different moving speeds, luminances, and the like are included in one object field, appropriate imaging conditions can be individually set for each unit block 131. Thereby, the quality of finally obtained image data can be improved.

  In addition, when the imaging target corresponding to one of the unit blocks 131 is stationary or bright enough, the imaging device 100 reduces the frame rate or shortens the exposure time, thereby reducing the imaging device. Pixel signals output from 100 can be reduced. This facilitates the transfer of pixel signals and reduces the image processing load.

  FIG. 15 is a timing chart illustrating a global shutter mode, which is another operation mode of the unit block 131 in the image sensor 100. In the global shutter mode, all the pixels 150 included in the unit block 131 are simultaneously reset and then selected to accumulate pixel signals. In the global shutter mode, the accumulated pixel signals may be sequentially read out in units of rows as in the rolling shutter mode.

  In the global shutter mode as described above, the timing of accumulating pixel signals in the unit block 131 is the same, so that no rolling shutter distortion occurs in the captured image 311. Accordingly, when particularly large rolling shutter distortion occurs in a certain unit block 131, the imaging control unit 474 may switch the accumulation control in the unit block 131 to the global shutter mode.

  However, the speed of the moving component of the subject cannot be detected for the unit block 131 operating in the global shutter mode in which no rolling shutter distortion occurs. Therefore, the imaging control unit 474 operates in the rolling shutter mode without executing the operation for setting the imaging conditions according to the moving speed of the subject for the unit block 131 that accumulates the pixel signals in the global shutter mode. The processing shown in FIGS. 11 and 13 is executed only for the unit block 131.

  Further, in the global shutter mode, all the pixels 150 in the unit block 121 are reset at the same time, so that a large current flows, and charge leakage is likely to occur between adjacent rows. , Noise is easy to ride on the captured image. Therefore, when the speed of the moving component of the subject decreases, the unit block 131 switched to the global shutter mode may be returned to the rolling shutter mode.

  Therefore, in an image sensor that operates in the global shutter mode, the operation mode of the unit block 131 may be temporarily or intermittently set to the rolling shutter mode for the purpose of detecting the speed of the moving component of the subject. Since the operation in the rolling shutter mode is executed for the purpose of detecting the moving component of the subject, it is not necessary to generate image data from the pixel signal obtained in this process.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. It will be apparent to those skilled in the art that various modifications or improvements can be added to the above-described embodiment. It is apparent from the scope of the claims that the embodiments added with such changes or improvements can be included in the technical scope of the present invention.

  The order of execution of each process such as operations, procedures, steps, and stages in the apparatus, system, program, and method shown in the claims, the description, and the drawings is particularly “before” or “prior to”. It should be noted that the output can be realized in any order unless the output of the previous process is used in the subsequent process. Regarding the operation flow in the claims, the description, and the drawings, even if it is described using “first”, “next”, etc. for convenience, it means that it is essential to carry out in this order. It is not a thing.

  10 capacitor, 60 amplification transistor, 61-1, 61-2 selection transistor, 59 reset transistor, 65 ground side transistor, 78 first transistor, 63 second transistor, 66 amplification transistor, 64 transfer transistor, 100 image sensor, 101 micro Lens, 102 Color filter, 103 Passivation film, 104, 104-1, 104-2 Photodiode, 105 Transistor, 106 Photoelectric conversion element layer, 107 Wiring, 108 Wiring layer, 109 Bump, 110 TSV, 111 Signal processing board, 112 Memory substrate, 113 light receiving substrate, 131 unit block, 133 peripheral circuit, 150 pixels, 170 column transmission path, 172 column bus line, 200 row control unit, 202 analog / digital conversion circuit 204 CDS circuit, 206 shift register, 300 subject, 301, 302, 303 linear image, 311, 312, 313 captured image, 414 pixel memory, 418 I / F circuit, 420 drive control unit, 430 timing memory, 441 sensor control unit 442 Block control unit 443 Synchronization control unit 444 Signal control unit 472 Correlation calculation unit 474 Imaging control unit 500 Imaging device 501 System control unit 502 Drive unit 503 Photometry unit 504 Work memory 505 Recording unit 506 Display unit 511 Image processing unit 512 Calculation unit 520 Imaging lens

Claims (9)

An imaging device in which a plurality of imaging regions for imaging a subject are arranged in a first direction and a second direction intersecting the first direction;
A correlation calculation unit for calculating a correlation between images of subjects imaged in a first imaging region and a second imaging region arranged adjacent to each other among the plurality of imaging regions arranged in the imaging element;
A setting unit for setting the imaging conditions of the first imaging region and the second imaging region based on the correlation calculated by the correlation calculation unit;
An imaging apparatus comprising: In the imaging region, a pixel including a photoelectric conversion unit that converts light into electric charge is disposed,
The imaging apparatus according to claim 1, wherein the setting unit sets a charge accumulation time of the photoelectric conversion unit as the imaging condition.   The imaging apparatus according to claim 2, wherein the setting unit sets a charge accumulation start time of the photoelectric conversion unit as the imaging condition.   The imaging apparatus according to claim 2, wherein the setting unit sets a charge accumulation end time of the photoelectric conversion unit as the imaging condition.   The imaging device according to any one of claims 2 to 4, wherein the imaging region includes a plurality of the pixels.   The signal line for outputting the signal which was connected to the pixel and was generated by the electric charge converted by the photoelectric conversion part is arranged in the image pick-up field. The imaging device described.   The imaging device includes: an imaging chip in which a plurality of imaging regions are arranged; and a signal processing chip that is connected to the imaging chip and includes a processing circuit that processes an image signal of a subject imaged in the imaging region. The imaging device according to any one of claims 1 to 6.   The imaging apparatus according to claim 7, wherein the signal processing chip includes a conversion unit that converts the image signal into a digital value.   The imaging apparatus according to claim 8, wherein the signal processing chip includes a storage unit that stores the image signal converted into a digital value by the conversion unit.

Images