JP2018007282A - Image pickup device and imaging apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve the problem that a wiring layout of a global shutter becomes complex.SOLUTION: An image pickup device comprises: an imaging part in which a plurality of unit blocks including a plurality of pixels arranged in matrix are arranged in matrix; a reading part that selects the pixel in the same row of the unit block in each unit block and sequentially repeats the operation for reading a pixel signal in each row; a correlation calculation part that calculates a correlation of the pixel signal between rows of a boundary of the unit blocks adjacent in the same column; and a correction part that corrects at least one pixel signal of the unit blocks adjacent in the same column on the basis of correlation calculated by the correlation calculation part.SELECTED DRAWING: Figure 11

Description

  The present invention relates to an imaging element and an imaging apparatus.

A so-called rolling shutter system is known for an image sensor having a plurality of pixels arranged in a matrix. In the rolling shutter method, the operation of selecting and storing pixel signals in the same row and sequentially storing pixel signals is sequentially repeated for each row. Furthermore, it is known that a global shutter is used instead of the rolling shutter system to prevent rolling distortion that occurs when a moving object is imaged (see, for example, Patent Document 1).
Patent Document 1 Republished 2010/023903

  However, the global shutter has a problem that the wiring layout is complicated.

  In the first aspect of the present invention, there is provided an imaging device in which a plurality of unit blocks each having a plurality of pixels arranged in a matrix form an imaging unit in which a plurality of pixels are arranged in a matrix, A readout unit that sequentially repeats, for each row, an operation of selecting a pixel and reading out a pixel signal, a correlation calculation unit that calculates a correlation of pixel signals between adjacent rows of unit blocks in the same column, and a correlation calculation unit And a correction unit that corrects at least one pixel signal of adjacent unit blocks in the same column based on the correlation calculated by the above.

  It should be noted that the above summary of the invention does not enumerate all the necessary features of the present invention. In addition, a sub-combination of these feature groups can also be an invention.

It is a sectional view of a back irradiation type image sensor concerning this embodiment. It is a figure explaining the pixel arrangement | sequence and unit block of an imaging chip. It is a circuit diagram corresponding to a pixel. An outline of a unit block and its peripheral circuits and their connection relation will be shown. It is a block diagram which shows the structure of the imaging device which concerns on this embodiment. It is a block diagram which shows the specific structure of a drive part. The functional block of an arithmetic circuit is shown. 4 shows a timing chart of operations such as charge accumulation and reading in each row of a unit block. An example of a subject image incident on an image sensor is shown. The captured image before correction | amendment is shown. It is a flowchart which shows operation | movement of an arithmetic circuit. The captured image after correction | amendment is shown.

  Hereinafter, the present invention will be described through embodiments of the invention, but the following embodiments do not limit the invention according to the claims. In addition, not all the combinations of features described in the embodiments are essential for the solving means of the invention.

  FIG. 1 is a cross-sectional view of a back-illuminated image sensor 100 according to this embodiment. The imaging element 100 includes an imaging chip 113 that outputs a pixel signal corresponding to incident light, a signal processing chip 111 that processes the pixel signal, and a memory chip 112 that stores the pixel signal. The imaging chip 113, the signal processing chip 111, and the memory chip 112 are stacked, and are electrically connected to each other by a conductive bump 109 such as Cu.

  As shown in the figure, incident light is incident mainly in the positive direction of the Z-axis indicated by a white arrow. In the present embodiment, in the imaging chip 113, the surface on the side where incident light is incident is referred to as a back surface. Further, as indicated by the coordinate axes, the right direction on the paper orthogonal to the Z axis is the X axis plus direction, and the front side of the paper orthogonal to the Z axis and the X axis is the Y axis plus direction. In the following several figures, the coordinate axes are displayed so that the orientation of each figure can be understood with reference to the coordinate axes of FIG.

  An example of the imaging chip 113 is a back-illuminated MOS image sensor. The PD layer is disposed on the back side of the wiring layer 108. The PD layer 106 includes a plurality of PDs (photodiodes) 104 arranged two-dimensionally and a transistor 105 provided corresponding to the PD 104.

  A color filter 102 is provided on the incident side of incident light in the PD layer 106 via a passivation film 103. The color filter 102 has a plurality of types that transmit different wavelength regions, and has a specific arrangement corresponding to each of the PDs 104. The arrangement of the color filter 102 will be described later. A set of the color filter 102, the PD 104, and the transistor 105 forms one pixel.

  On the incident light incident side of the color filter 102, a microlens 101 is provided corresponding to each pixel. The microlens 101 condenses incident light toward the corresponding PD 104.

  The wiring layer 108 includes a wiring 107 that transmits the pixel signal from the PD layer 106 to the signal processing chip 111. The wiring 107 may be multilayer, and a passive element and an active element may be provided.

  A plurality of bumps 109 are disposed on the surface of the wiring layer 108. The plurality of bumps 109 are aligned with the plurality of bumps 109 provided on the opposing surfaces of the signal processing chip 111, and the imaging chip 113 and the signal processing chip 111 are pressed and aligned. The bumps 109 are joined and electrically connected.

  Similarly, a plurality of bumps 109 are disposed on the mutually facing surfaces of the signal processing chip 111 and the memory chip 112. The bumps 109 are aligned with each other, and the signal processing chip 111 and the memory chip 112 are pressurized, so that the aligned bumps 109 are joined and electrically connected.

  The bonding between the bumps 109 is not limited to Cu bump bonding by solid phase diffusion, and micro bump bonding by solder melting may be employed. Further, for example, about one bump 109 may be provided for one output wiring described later. Therefore, the size of the bump 109 may be larger than the pitch of the PD 104. Further, a bump larger than the bump 109 corresponding to the pixel region may be provided in a peripheral region other than the pixel region where the pixels are arranged.

  The signal processing chip 111 has a TSV (silicon through electrode) 110 that connects circuits provided on the front and back surfaces to each other. The TSV 110 is preferably provided in the peripheral area. The TSV 110 may also be provided in the peripheral area of the imaging chip 113 and the memory chip 112.

  FIG. 2 is a diagram for explaining the pixel array and the unit block 131 of the imaging chip 113. In particular, a state where the imaging chip 113 is observed from the back side is shown. The imaging chip 113 has an imaging unit in which 20 million or more pixels are arranged in a matrix. In the example of FIG. 2, adjacent 16 pixels of 4 pixels × 4 pixels form one unit block 131. The grid lines in the figure indicate the concept that adjacent pixels are grouped to form a unit block 131.

  As shown in the partially enlarged view of the image pickup unit, the unit block 131 includes four so-called Bayer arrays including four pixels of green pixels Gb, Gr, blue pixels B, and red pixels R vertically and horizontally. The green pixels Gb and Gr have a green filter as the color filter 102 and receive light in the green wavelength band of incident light. Similarly, the blue pixel B has a blue filter as the color filter 102 and receives light in the blue wavelength band, and the red pixel R has a red filter as the color filter 102 and receives light in the red wavelength band. .

  In FIG. 2, the example in which the unit block 131 includes 16 pixels of 4 pixels × 4 pixels has been described for the purpose of simplifying the description. There are no particular restrictions on the number of rows and the number of columns, but when there are about 20 million pixels in the entire image pickup unit, for example, the number is 64 rows and 32 columns. Further, although there are no restrictions on the number of rows and the number of columns of the unit block 131 included in the imaging unit, for example, 48 rows and 114 columns are arranged.

  FIG. 3 is a circuit diagram corresponding to the pixel 150. In FIG. 3, a rectangle surrounded by a dotted line typically represents a circuit corresponding to one pixel 150. Note that at least some of the transistors described below correspond to the transistor 105 in FIG.

  The PD 104 is connected to the transfer transistor 154. The gate of the transfer transistor 154 is connected to the wiring Tx_j to which the transfer pulse is supplied. The subscript j is a serial number in the unit block 131 that identifies a row number in the unit block 131.

  The drain of the transfer transistor 154 is connected to the source of the reset transistor 152. As a result, a so-called FD (floating diffusion) 156 is formed between the drain of the transfer transistor 154 and the source of the reset transistor 152. The drain of the reset transistor 152 is connected to the wiring Vdd supplied with the power supply voltage, and the gate thereof is connected to the wiring Rst_j supplied with the reset pulse.

  One end of the FD 156 is further connected to the gate of the amplification transistor 162. The drain of the amplification transistor 162 is connected to a wiring Vdd to which a power supply voltage is supplied. The source of the amplification transistor 162 is connected to the drain of the corresponding selection transistor 164. A gate of the selection transistor 164 is connected to a wiring Sel_j to which a selection pulse is supplied.

  The source of the selection transistor 164 is connected to the column transmission line 170. The load current source 166 supplies current to the column transmission line 170. That is, the column transmission line 170 for the selection transistor 164 is formed by a source follower.

  FIG. 4 shows an outline of the unit block 131 and its peripheral circuit 133 and their connection relation. In the unit block 131 of FIG. 4, a total of (P × L) pixels 150 are arranged in L rows and P columns.

  The wiring Rst_l (where l is an integer from 1 to L) is connected to the row control unit 200 and is commonly connected to the P pixels 150 in the l-th row in the unit block 131. Similarly, the wiring Tx_l and the wiring Sel_l are also connected to the row control unit 200 and are commonly connected to the P pixels 150 in the l-th row in the unit block 131.

  The row control unit 200 may be called a row selection unit, a vertical scanning circuit, or the like. The row control unit 200 is provided for each unit block 131. The row control unit 200 may be provided on the signal processing chip 111 side.

  The column transmission path 170 is provided for each pixel 150 in the same column. These column transmission paths 170_p (where p is an integer from 1 to P) are commonly connected to the L pixels 150 in the p-th column in the unit block 131. Thus, the column transmission path 170 is shared by the pixels 150 in the same column in the unit block 131, and transmits signals from the pixels 150 included in the column.

  These column transmission paths 170_p are connected from the imaging chip 113 side to the peripheral circuit 133 provided on the signal processing chip 111 side via the bump 109. The peripheral circuit 133 is provided for each unit block 131 and is arranged so as to overlap the unit block 131 in the imaging chip 113 when viewed from the stacking direction.

  The peripheral circuit 133 includes a CDS circuit 202 and an A / D conversion circuit 204 connected in series for each column transmission path 170_p. In the example shown in FIG. 4, P sets of CDS circuits 202 and A / D conversion circuits 204 are provided for each unit block 131. The CDS circuit 202 removes noise from the pixel signal. The A / D conversion circuit 204 converts the pixel signal from which noise has been removed by the CDS circuit 202 into a digital signal.

  The peripheral circuit 133 further includes a shift register 206 disposed on the output side of the P A / D conversion circuits 204. In the example of FIG. 4, one shift register 206 is arranged for each unit block 131. The output of the shift register 206 is connected to the pixel memory 414 via the column bus line 172. The shift register 206 is sometimes called a horizontal scanning circuit, a multiplexer, or the like.

  FIG. 5 is a block diagram illustrating a configuration of the imaging apparatus according to the present embodiment. The imaging apparatus 500 includes a photographic lens 520 as a photographic optical system, and the photographic lens 520 guides a subject luminous flux incident along the optical axis OA to the imaging element 100. The photographing lens 520 may be an interchangeable lens that can be attached to and detached from the imaging apparatus 500. The imaging apparatus 500 mainly includes an imaging device 100, a system control unit 501, a drive unit 502, a photometry unit 503, a work memory 504, a recording unit 505, and a display unit 506.

  The photographing lens 520 is composed of a plurality of optical lens groups, and forms an image of a subject light flux from the scene in the vicinity of its focal plane. In FIG. 5, a representative single lens arranged in the vicinity of the pupil is shown as a representative. The drive unit 502 executes charge accumulation control of the image sensor 100, pixel signal readout control, and the like in accordance with instructions from the system control unit 501.

  The image sensor 100 delivers the pixel signal to the image processing unit 511 of the system control unit 501. The image processing unit 511 performs various image processing using the work memory 504 as a work space, and generates image data. For example, when generating image data in JPEG file format, compression processing is executed after white balance processing, gamma processing, and the like are performed. The generated image data is recorded in the recording unit 505, converted into a display signal, and displayed on the display unit 506 for a preset time.

  The photometric unit 503 detects the luminance distribution of the scene prior to a series of shooting sequences for generating image data. The photometry unit 503 includes, for example, an AE sensor having about 1 million pixels. The calculation unit 512 of the system control unit 501 receives the output of the photometry unit 503 and calculates the luminance for each area of the scene. The calculation unit 512 determines the shutter speed, aperture value, and ISO sensitivity according to the calculated luminance distribution. Note that the pixels used in the AE sensor may be provided in the image sensor 100. In this case, the photometric unit 503 separate from the image sensor 100 may not be provided.

  FIG. 6 is a block diagram showing a specific configuration of the drive unit 502. The drive unit 502 controls the sensor control unit 441, the block control unit 442, the synchronization control unit 443, the signal control unit 444, the pixel memory 414, the arithmetic circuit 415, and the respective control units as shared control functions. Drive control unit 420. The drive unit 502 further includes an I / F circuit 418 between the drive control unit 420 and the system control unit 501 of the imaging apparatus 500 main body.

  The drive control unit 420 refers to the timing memory 430, converts an instruction from the system control unit 501 into a control signal that can be executed by each control unit, and delivers the control signal to each. The timing memory 430 is formed by a flash RAM or the like.

  The sensor control unit 441 performs transmission control of control pulses that are transmitted to the imaging chip 113 and are related to charge accumulation and charge readout of each pixel. Specifically, the sensor control unit 441 controls the start and end of charge accumulation of the target pixel by sending a reset pulse and a transfer pulse to the row control unit 200 of each unit block 131, and for the readout pixel By sending a selection pulse, the pixel signal is output to the column transmission path 170.

  The block control unit 442 executes transmission of a specific pulse that specifies the unit block 131 to be controlled and is transmitted to the imaging chip 113. A transfer pulse received by each pixel via the wiring Tx_j or the like is a logical product of each pulse transmitted by the sensor control unit 441 and a specific pulse transmitted by the block control unit 442. In this way, each area can be controlled as an independent block. In addition, when using a pulse synchronized with a plurality of unit blocks 131 and when performing an operation across a plurality of unit blocks 131, the block control unit 442 specifies a specific pulse for specifying each of the plurality of unit blocks. Are sent simultaneously.

  The synchronization control unit 443 sends a synchronization signal to the imaging chip 113. Each pulse becomes active in the imaging chip 113 in synchronization with the synchronization signal. For example, by adjusting the synchronization signal, random control, thinning control, or the like that controls only specific pixels of pixels belonging to the same unit block 131 is realized. The signal control unit 444 is responsible for timing control for the CDS circuit 202, the A / D conversion circuit 204, and the shift register 206.

  The arithmetic circuit 415 calculates an AE evaluation value based on the pixel signal stored in the pixel memory 414. The arithmetic circuit 415 outputs the calculation result to the drive control unit 420.

  The pixel memory 414 has a memory space in which the pixel signal from the pixel 150 of the imaging unit can be stored, and stores each pixel signal read out from each pixel and digitized. The pixel memory 414 is preferably provided corresponding to each unit block 131.

  The pixel memory 414 is provided with a data transfer interface that transmits pixel signals in accordance with a delivery request. The data transfer interface is connected to a data transfer line connected to the image processing unit 511. The data transfer line is constituted by a data bus of the bus lines, for example. In this case, the delivery request from the system control unit 501 to the drive control unit 420 is executed by address designation using the address bus.

  Transmission of pixel signals by the data transfer interface is not limited to the addressing method, and various methods can be adopted. For example, when performing data transfer, a double data rate method in which processing is performed using both rising and falling edges of a clock signal used for synchronization of each circuit may be employed. Further, it is possible to adopt a burst transfer method in which data is transferred all at once by omitting a part of the procedure such as addressing and the like, and the speed is increased. Further, a bus system using a line in which a control unit, a memory unit, and an input / output unit are connected in parallel, or a serial system that transfers data one bit at a time can be combined.

  With this configuration, the image processing unit 511 can receive only necessary pixel signals, so that image processing can be completed at high speed, particularly when a low-resolution image is formed. Note that the drive unit 502, the row control unit 200 in FIG. 4, and the peripheral circuit 133 function as a reading unit that sequentially reads pixel signals of the pixels 150 included in the imaging unit across the plurality of unit blocks 131.

  FIG. 7 shows functional blocks of the arithmetic circuit 415. The arithmetic circuit 415 includes a correlation calculation unit 472 and a correction unit 474 in addition to the functions described above. The correlation calculation unit 472 reads out the pixel signal from the pixel memory 414, and calculates the correlation of the pixel signal between the rows at the boundary of the adjacent unit blocks 131 in the same column. The correcting unit 474 corrects the pixel signals in the adjacent unit blocks 131 in the same column based on the correlation calculated by the correlation calculating unit 472 and writes the corrected pixel signals in the pixel memory 414.

  FIG. 8 shows a timing chart of operations such as charge accumulation and reading of each row of the unit block 131A and the like. In each unit block, the operation in which the pixels 150 in the same row are selected and the pixel signals are accumulated and read out is operated in a so-called rolling shutter system, which is sequentially repeated for each row.

  The driving unit 502 starts charge accumulation of the pixels 150 in the first row by giving driving signals to the wirings Rst_1 and Tx_1 for the pixels 150 in the first row via the row control unit 200. Further, after the accumulation period, the driving unit 502 gives a driving signal to the wiring Sel_1 for the pixels 150 in the first row via the row control unit 200, and starts reading out the pixel signals of the pixels 150 in the first row. The readout period is the time from when the pixel signal is read from each pixel 150 in the first row, processed by the CDS circuit 202 and the A / D conversion circuit 204, and sequentially written from the shift register 206 to the pixel memory 414. Including.

  Similarly, driving signals are applied to the wirings Rst_2, Tx_2, and Sel_2 for the pixels 150 in the second row, and accumulation and reading are performed. The length of the accumulation period for the pixels 150 in the second row is the same as that of the pixels 150 in the first row, but is delayed by about the readout period in time. From the third row to the Lth row, drive signals are given to the wirings Rst_l, Tx_l and Sel_l, and accumulation and reading are sequentially performed.

  The order of the selected rows is the same for any of the plurality of unit blocks 131A and 131B. In the example of FIG. 8, for each case, one row is selected from the −Y side row to the + Y side row. Further, the storage and reading timings of the same row are synchronized between the plurality of unit blocks 131A and 131B. However, the timing of at least one of accumulation and reading of the same row may not be synchronized between the plurality of unit blocks 131A and 131B.

  FIG. 9 shows an example of a subject image 300 incident on the image sensor 100. For the sake of explanation, the image light immediately before entering the image sensor 100 is drawn as viewed from the + Z side.

  In the example shown in FIG. 9, the subject image 300 includes two moving body images 302 and 306 moving in the X direction and long in the Y direction, and one stationary body image 304 long in the Y direction that is stationary. ing. Further, the moving object image 302 moves faster than the moving object image 306.

  FIG. 10 shows a captured image before correction. For the sake of explanation, it is shown in the direction of output, display, etc. as a captured image. Further, description will be made assuming that the subject image 300 of FIG. 9 is incident on the four unit blocks 131A, 131B, 131C, and 131D.

  As shown in the unit blocks 131A and 131B in FIG. 10, the moving body image 302 is distorted in an oblique direction in the unit block. Since this distortion is different in the imaging time for each row by the A / D conversion time, the moving body image 302 is shifted in the + X direction that is the moving direction. This deviation is called rolling distortion. Similarly, in the unit block 131D, the moving body image 306 is distorted in the + X direction. However, the amount of distortion is small corresponding to the fact that the speed of the moving object image 306 is smaller than the speed of the moving object image 302. On the other hand, as shown in the unit blocks 131C and 131D, the stationary body image 304 is not distorted.

  Further, attention is paid to pixels in the boundary row in the adjacent unit blocks 131A and 131B in the same column. That is, attention is focused on the 10th row of the unit block 131A and the 1st row of the unit block 131B. These two rows are most distant from the timing of the accumulation period. Reflecting this, the position of the pixel corresponding to the moving object image 302 moving in the + X direction as a whole is shifted to a different column position between these two rows. On the other hand, the column positions of the pixels corresponding to the stationary body image 304 are not in the tenth row of the unit block 131C and the first row of the unit block 131D.

  FIG. 11 is a flowchart showing the operation of the arithmetic circuit 415, and FIG. 12 shows a captured image after correction. The flowchart in FIG. 11 is started when the pixel signal before correction in FIG. 10 is acquired and stored in the pixel memory 414.

  The correlation calculation unit 472 reads out the pixel signal of the boundary row in the adjacent unit blocks 131A and 131B in the same column from the pixel memory 414 (S100). For example, in the case of a set of unit blocks 131A and 131B, the pixel signal of the 10th row of the unit block 131A and the first row and the pixel signal of the unit block 131B are read out.

  The correlation calculation unit 472 calculates the correlation of the boundary row (S102). It can be said that the magnitude of the correlation is the degree of coincidence of a row of pixel signals. As an example of an evaluation value for evaluating the correlation, the correlation calculation unit 472 squares the difference between pixel signals of pixels in the same column in the 10th row of the unit block 131A and the 1st row of the unit block 131B, and sums them. The sum of squared residuals is calculated by taking Note that the smaller the residual sum of squares, the greater the correlation. Hereinafter, an example using the residual sum of squares as an evaluation value will be described.

  The correlation calculation unit 472 determines whether or not the residual sum of squares is larger than the threshold (S104). When the magnitude of the residual sum of squares is equal to or smaller than the threshold (S104: No), the flowchart in the unit blocks 131A, 131B and the like adjacent in the same column is ended.

  When the residual sum of squares is smaller than the threshold, for example, as shown in FIG. 10, the tenth row on the unit block 131C side is accumulated later in time than the first row of the unit block 131D. It is estimated that the column position shift is small even though the reading is performed. Therefore, in the unit blocks 131C and 131D adjacent to each other in the same column, the unit block 131C having the tenth row that has been stored and read out later in time has no or no rolling distortion. Presumed to be small. Therefore, the unit block 131C is temporarily excluded from the correction target in FIG. However, correction may be performed due to the influence from unit blocks 131A adjacent to each other in the same column, resulting in a correction block 132C.

  On the other hand, when the magnitude of the residual sum of squares is larger than the threshold value (S104: Yes), the correlation calculation unit 472 compares one of the unit block 131A and the first block of the unit block 131B with respect to the other. The pixel is shifted by one pixel in the row direction (S106), that is, a residual sum of squares is calculated for pixels shifted by one column (S108). The correlation calculation unit 472 repeats the above steps S106 and S108 for calculating the residual sum of squares by shifting by one pixel until the number of pixels p in the column is reached (S110: No). In this case, the pixels are shifted in the + X direction and the −X direction, respectively.

  The correlation calculation unit 472 specifies the number of pixels used for correction after shifting by p pixels (S110: Yes) (112). In this case, the correlation calculation unit 472 uses, for correction, the number of pixel shifts when the smallest residual sum of squares is calculated among the plurality of residual sums of squares calculated by repeating steps S106 to S110. The number of pixels. When there are a plurality of smallest residual sums of squares, the number of pixels is preferably smaller than the number of pixel shifts.

  In the example shown in FIG. 10, if the 10th row of the unit block 131A is shifted in the −X direction by 4 pixels with respect to the 1st row of the unit block 131B, the residual sum of squares becomes the smallest. Therefore, the correlation calculation unit 472 specifies “4” as the number of pixels used for correction.

  The correcting unit 474 corrects the pixel signal of the unit block 131A using the number of pixels specified in step S112 (S114). Here, among the unit blocks 131A and 131B adjacent in the same column, the pixel signal of the unit block 131A having the tenth row that has been accumulated and read out in terms of time is corrected. In this case, the correction unit 474 equally allocates the shift amount to each row so that the 10th row is shifted by 4 pixels with respect to the first row in the unit block 131A, that is, by 4 columns. In the example of FIG. 10, a shift of one pixel is assigned to adjacent groups, with two adjacent rows as one set.

  As shown in FIG. 12, the correction unit 474 fixes the fifth and sixth rows at the center of the unit block 131A with respect to the other unit blocks 131B and the like, and converts the pixel signals of the pixels in the other rows to the column positions. The correction block 132A is generated by replacing the pixel signal of the shifted pixel. Similarly, the correction blocks 132B and 132D are generated by shifting the column positions of the rows of the unit blocks 131A and 132D.

  The correcting unit 474 further corrects the pixel signal at the boundary between the adjacent unit blocks 131A and 131C in the same row. As shown in FIG. 12, when unit blocks 131A and 131C adjacent to each other in the same row are corrected to generate correction blocks 132A and 132C, a blank pixel 140 region and overlapping pixels 142 in which the pixels overlap with each other at their boundaries. Resulting in a region. The correction unit 474 assigns pixel signals from the periphery of the blank pixel 140 area. Further, the correction unit 474 assigns the average value of the pixel signals of the respective pixels to the overlapping pixel 142.

  Thus, the correction block 132A and the like are generated. Here, although the pixel of the correction block 132C is not replaced by the correlation determination of step S104 with respect to the unit block 131C, the boundary is corrected under the influence of the correction from the adjacent correction block 132A. The correction unit 474 outputs pixel signals of the correction block 132A and the like to the pixel memory 414 (S116), and ends this flowchart. As described above, rolling distortion can be reduced with a simple configuration.

  In step S102, the correlation between the first row and the last row in the same unit block 131A is calculated instead of calculating the correlation between the boundary rows in the adjacent unit blocks 131A and 131B in the same column. May be. In each of steps S102 and S108, when the pixel signal has luminance and color signals, the correlation may be calculated using the luminance signal. Alternatively, the correlation may be calculated for pixels of the same color, for example, green pixels. Instead of the residual sum of squares, the sum of absolute values of differences may be used as the correlation evaluation value. Further, instead of repeating p pixels in step S110, a so-called hill-climbing method may be used in which the repetition is stopped when a minimum value appears, as compared to calculating the residual sum of squares. .

  When the unit block 131 has a color filter such as a Bayer array, the operation of FIG. 11 is preferably performed before the interpolation process. When the A / D converted pixel signal is output as a captured image converted to a predetermined format such as JPEG, the operation of FIG. 11 is after the A / D conversion and is converted to the format. It is preferably performed before conversion. Further, it is preferable that at least the correlation calculation unit 472 and the correction unit 474 are provided in the signal processing chip 111.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. It will be apparent to those skilled in the art that various modifications or improvements can be added to the above-described embodiment. It is apparent from the scope of the claims that the embodiments added with such changes or improvements can be included in the technical scope of the present invention.

  The order of execution of each process such as operations, procedures, steps, and stages in the apparatus, system, program, and method shown in the claims, the description, and the drawings is particularly “before” or “prior to”. It should be noted that the output can be realized in any order unless the output of the previous process is used in the subsequent process. Regarding the operation flow in the claims, the description, and the drawings, even if it is described using “first”, “next”, etc. for convenience, it means that it is essential to carry out in this order. It is not a thing.

  100 imaging device, 101 microlens, 102 color filter, 103 passivation film, 104 PD, 105 transistor, 106 PD layer, 107 wiring, 108 wiring layer, 109 bump, 110 TSV, 111 signal processing chip, 112 memory chip, 113 imaging Chip, 131 unit block, 132 correction block, 133 peripheral circuit, 140 blank pixel, 142 overlapping pixel, 150 pixel, 152 reset transistor, 154 transfer transistor, 156 FD, 162 amplification transistor, 164 selection transistor, 166 load current source, 170 Column transmission line, 172 column bus line, 200 row control unit, 202 CDS circuit, 204 A / D conversion circuit, 206 shift register, 300 subject image, 302 moving object image, 04 still image, 306 moving image, 414 pixel memory, 415 arithmetic circuit, 418 I / F circuit, 420 drive control unit, 430 timing memory, 441 sensor control unit, 442 block control unit, 443 synchronization control unit, 444 signal control Unit, 472 correlation calculation unit, 474 correction unit, 500 imaging device, 520 photographing lens, 501 system control unit, 502 drive unit, 503 photometry unit, 504 work memory, 505 recording unit, 506 display unit, 511 image processing unit, 512 Calculation unit

Claims (8)

A plurality of pixels arranged in a matrix, a control line that is connected to the pixel and reads a pixel signal from the pixel, and a signal line that is connected to the pixel and outputs a pixel signal read from the pixel And an imaging unit in which a plurality of unit blocks having a matrix shape are arranged, and
For each unit block, a reading unit that sequentially repeats, for each row, an operation of selecting the pixels in the same row of the unit block by the control line and reading a pixel signal to the signal line
A correlation calculation unit that calculates the correlation of pixel signals between the rows of the boundary of adjacent unit blocks in the same column;
An imaging device comprising: a correction unit that corrects at least one pixel signal of unit blocks adjacent in the same column based on the correlation calculated by the correlation calculation unit. The correlation calculation unit specifies the number of pixels in the row direction when the correlation increases while shifting the column position of the pixels between the rows of the adjacent unit blocks in the same column,
The correction unit converts pixel signals of pixels in at least one row of adjacent unit blocks in the same column into pixel signals at pixel positions when the column position is shifted in accordance with the specified number of pixels. The imaging device according to claim 1, wherein the image sensor is corrected by replacement.   The imaging device according to claim 1, wherein the correction unit corrects a pixel signal at a boundary between adjacent unit blocks in the same row based on the correlation calculated by the correlation calculation unit.   In the same column, when the correlation calculation unit calculates that the correlation when the column position of the pixel is not shifted between adjacent rows of unit blocks in the same column is greater than a threshold value, The imaging device according to claim 2 or 3, wherein at least one of the adjacent unit blocks is excluded from correction targets. An AD converter for digitizing the pixel signal;
The imaging device according to claim 1, wherein the correlation calculation unit calculates a correlation of the digitized pixel signal. The imaging unit is provided on the first substrate,
The imaging device according to claim 1, wherein the correlation calculation unit and the correction unit are provided on a second substrate stacked with the first substrate.   The imaging device according to claim 1, wherein the imaging unit is a backside illumination type CMOS sensor.   An imaging device comprising the imaging device according to claim 1.

Images