JP5791975B2 - Solid-state imaging device and imaging device - Google Patents

Description

  The present invention relates to a solid-state imaging device, an imaging device, and the like.

  An imaging device that converts an optical image into an electrical signal is mounted on an imaging device such as a digital camera or a digital video camera. In recent years, in this imaging device, replacement from a CCD (Charge Coupled Device) type to a CMOS (Complementary Metal Oxide Semiconductor) type is accelerating.

  This CMOS image sensor can read out various types of pixel signals in addition to full scan by taking advantage of its features. For example, it is possible to read out the number of pixels corresponding to various resolutions by movie scanning for extracting a signal obtained by adding a plurality of pixels in the horizontal and vertical directions.

JP-A-2005-278135 JP 2008-294913 A

  Now, an image obtained by pixel addition has a lower resolution than when pixels are not added. Therefore, in order to obtain a high-definition still image during moving image shooting, for example, a method of performing still image shooting only when the shutter is pressed during moving image shooting can be considered.

  However, with this method, it is difficult to acquire a still image by pressing the shutter at a decisive moment. Therefore, it is conceivable to shoot a moving image by pixel addition and generate a high-definition still image at an arbitrary timing later from the moving image.

  As such a method, an addition unit that is a range of pixels to be added is superimposed and set, and a pixel value included in the addition unit is weighted and added to obtain a low-resolution image. A technique for restoring high-definition still images can be considered.

  However, conventional pixel addition methods (for example, Patent Documents 1 and 2) have a problem that pixel addition and weighted addition in which addition units are superimposed cannot be realized.

  According to some aspects of the present invention, it is possible to provide a solid-state imaging device, an imaging device, and the like that can perform pixel addition by superimposing addition units.

  One embodiment of the present invention includes a pixel array unit in which a plurality of pixels that photoelectrically convert incident light are arranged, a row scanning unit that selects a row of the pixel array unit, performs vertical scanning, and An A / D converter that converts an analog signal from a pixel into a pixel value of a digital signal, a first to nth memory area that stores the pixel value, and a first to nth memory area, A write control unit that performs control to write the pixel value; a read control unit that performs control to read out a pixel value included in an addition unit that is a range of pixels to be added from the first to nth memory regions; A weighted addition unit that performs weighted addition of pixel values included in the read addition unit and outputs the pixel value after addition as an added pixel value, and the write control unit is included in the addition unit Each pixel value is Each of the first to n-th memory areas is stored in a different memory area, and the readout control unit is common to the first addition unit and the control to read out the pixel value included in the first addition unit. The present invention relates to a solid-state imaging device that performs control to read out pixel values included in a second addition unit including the pixels.

  According to one aspect of the present invention, each pixel value included in the addition unit is stored in a different memory area among the first to nth memory areas, and is shared from the first to nth memory areas. Control is performed to read out pixel values included in the first and second addition units having the pixels. Then, weighted addition of the read pixel values is performed, and an added pixel value for each addition unit is output. This makes it possible to perform pixel addition by superimposing addition units. Further, it becomes possible to perform weighted addition of pixel values in increments.

  In the aspect of the invention, each of the first to nth memory areas may be a memory area having an independent address space, and the read control unit may determine the pixel value included in the addition unit as the first value. The data may be read from the nth memory area in parallel.

  In this way, it is possible to read out an addition unit having a common pixel, perform a weighting operation, and an addition operation without increasing the processing speed of each component compared with a conventional solid-state imaging device that does not perform superposition shift addition. Is possible.

  In the aspect of the invention, the weighting addition unit may weight the common pixel in the first addition unit and the common pixel in the second addition unit with different weighting factors. Also good.

  In this way, different weights can be applied to the common pixels in the addition operation of the first and second addition units. Thereby, the weighted addition of each addition unit can be realized in the addition unit to be superimposed.

  In the aspect of the invention, when the first addition unit includes pixels of 2 rows and 2 columns, the first and second memory areas of the first to nth memory areas are 1 line buffer, storing pixel values of odd columns and even columns of the first row of the pixel array unit, respectively, the third and fourth memory regions of the first to nth memory regions are The second line buffer is configured to store the pixel values of the odd and even columns of the second row of the pixel array unit, respectively, and the read control unit is configured to store two rows and two columns configuring the first addition unit. These pixel values may be read from the first and second line buffers.

  In one aspect of the present invention, when the second addition unit is an addition unit of 2 rows and 2 columns obtained by shifting the first addition unit by 1 row and 1 column, the first to nth memories. The fifth and sixth memory regions of the region constitute a third line buffer and store the pixel values of the odd and even columns in the third row of the pixel array unit, respectively, and the read control unit The pixel values of 2 rows and 2 columns constituting the second addition unit may be read from the second and third line buffers.

  In this way, three line buffers can be constituted by two banks, and the first to sixth memory areas (n = 6) can be constituted by the six banks. In addition, in the first and second addition units, overlapping shift addition can be performed by reading the pixel values redundantly from the second line buffer.

  In one embodiment of the present invention, the third addition unit is a 2-row, 2-column addition unit obtained by shifting the first addition unit by one column, and the fourth addition unit is the first addition unit. When the addition unit of 2 rows and 2 columns shifted by 1 row is used, the readout control unit reads out the pixel values of the first and second addition units in the first frame, In the next second frame, the pixel values of the third and fourth addition units may be read out.

  In this way, it is possible to obtain the addition pixel values of the first to fourth addition units that are shifted in superposition. Thereby, it becomes possible to apply the estimation calculation process mentioned later, and it becomes possible to obtain a high-resolution image by a simple estimation calculation process.

  In the aspect of the invention, the A / D conversion unit may be a column parallel A / D conversion unit that converts the analog signal of each column of the row selected by the row scanning unit into the digital signal in parallel. There may be.

  In this way, analog signals from one row of pixels can be converted into digital signals in parallel. This eliminates the need for high-speed A / D conversion, so that the operation speed of the solid-state imaging device can be reduced.

  Another aspect of the present invention includes the solid-state imaging device according to any one of the above and an estimation calculation unit that estimates a pixel value included in the addition unit based on the addition pixel value. When the first position addition unit set to the position of the second position addition unit set to the second position where the first position is shifted overlaps, the weighting addition unit The first and second addition pixel values which are the addition pixel values in the first and second position addition units are output, and the estimation calculation unit is configured to output the first addition pixel value and the second addition pixel value. The first intermediate pixel value, which is the added pixel value of the first region obtained by removing the overlap region from the first position addition unit, and the first value obtained by removing the overlap region from the second addition unit. The second medium that is the added pixel value of area 2 A relational expression with a pixel value is expressed using the difference value, the first and second intermediate pixel values are estimated using the relational expression, and the addition is performed using the estimated first intermediate pixel value. The present invention relates to an imaging device that obtains a pixel value of each pixel included in a unit.

  According to another aspect of the present invention, it is possible to estimate an intermediate pixel value from an addition pixel value of an addition unit that has been pixel shifted while being superimposed, and obtain a final estimated pixel value from the estimated intermediate pixel value. Thereby, a captured image can be estimated by simple processing.

  In another aspect of the present invention, the estimation calculation unit is included in the intermediate pixel value pattern when successive intermediate pixel values including the first and second intermediate pixel values are used as an intermediate pixel value pattern. A relational expression between intermediate pixel values is represented using the addition pixel value, the intermediate pixel value pattern represented by the relational expression between the intermediate pixel values is compared with the addition pixel value, and similarity is evaluated, Based on the similarity evaluation result, an intermediate pixel value included in the intermediate pixel value pattern may be determined so that the similarity becomes the highest.

  In this way, an intermediate pixel value can be obtained by estimation based on a plurality of added pixel values acquired by an addition unit that is pixel-shifted while being superimposed.

Explanatory drawing about superposition shift addition. Explanatory drawing about superposition shift addition. Explanatory drawing about superposition shift addition. 1 is a configuration example of a solid-state imaging device of the present embodiment. 2 is a detailed configuration example of a solid-state imaging device according to the present embodiment. Explanatory drawing about the superimposition shift addition operation | movement of a solid-state imaging device. Explanatory drawing about the write-in operation | movement with respect to a line buffer. Explanatory drawing about the read-out operation from a line buffer. An example of a timing chart of the present embodiment. 10A and 10B are explanatory diagrams of an estimated pixel value and an intermediate pixel value. Explanatory drawing about a restoration estimation process. Explanatory drawing about a restoration estimation process. Explanatory drawing about a restoration estimation process. 2 is a configuration example of an imaging device.

  Hereinafter, preferred embodiments of the present invention will be described in detail. The present embodiment described below does not unduly limit the contents of the present invention described in the claims, and all the configurations described in the present embodiment are indispensable as means for solving the present invention. Not necessarily.

1. Outline of the present embodiment In this embodiment, as will be described later with reference to FIG. 2 and the like, a low-resolution moving image is acquired by setting an addition unit that is shifted by superimposition and adding pixel values included in the addition unit. Then, as will be described later with reference to FIGS. 10A to 13, a high-resolution still image and a high-resolution moving image corresponding to the resolution of the image sensor are restored from the low-resolution moving image.

  Since a high-resolution image can be obtained after the fact, the user can specify a desired timing, and an image at a decisive moment can be easily obtained. Further, since the addition unit is shifted in a superimposed manner, it is possible to perform restoration by a simple process.

2. Superposition shift addition method First, superposition shift addition performed by the present embodiment will be described. FIG. 1 is an explanatory diagram of weighted addition in superposition shift addition.

As shown in FIG. 1, the first pixel A (reference pixel) is weighted with the weight “1”, the second pixel B is weighted with the weight “1/2”, and the third pixel C is weighted with “1”. / 2 "and the fourth pixel D is weighted by the weight" 1/4 ". That is, the added pixel value GP is obtained by the following equation (1).
GP = 1 * A + (1/2) * B + (1/2) * C + (1/4) * D (1)

Although the number of added pixels is 2 × 2 in the above, the present embodiment is not limited to this, and for example, the number of added pixels may be 3 × 3. In the above description, the weights are set to 1, 1/2, and 1/4. However, the present embodiment is not limited to this, and for example, the weights are set to 1, 1 / r, 1 / r as shown in the following formula (2). ² (r ≧ 1).
GP = 1 × A + (1 / r) × B + (1 / r) × C + (1 / r ² ) × D (2)

  Next, the superposition shift of the above pixel addition will be described with reference to FIGS. Hereinafter, a solid-state imaging device in which pixels are arranged in a matrix of 8 rows and 8 columns will be described as an example.

As shown in FIG. 2, in the odd frame (odd field), the above weighted addition is performed on the groups (addition units) GP11, GP13,... And the groups GP22, GP24,. For example, the group GP22 is obtained by shifting the group GP11 by 1 row and 1 column. The added pixel value is obtained by the following expression (3). Here, Dij (i and j are natural numbers) is the pixel value of the pixel Pij.
GP11 = 1 × D11 + (1/2) × D12 +
(1/2) × D21 + (1/4) × D22,
GP13 = 1 × D13 + (1/2) × D14 +
(1/2) × D23 + (1/4) × D24,
...
GP22 = 1 × D22 + (1/2) × D23 +
(1/2) × D32 + (1/4) × D33,
GP24 = 1 × D24 + (1/2) × D25 +
(1/2) × D34 + (1/4) × D35,
(3)

As shown in FIG. 3, in the even frame (even field), the above weighted addition is performed for the groups GP12, GP14,... And the groups GP21, GP23,. For example, the group GP12 is obtained by shifting the group GP11 of FIG. 2 by one column, and the group GP21 is obtained by shifting the group GP11 by one column. The added pixel value is obtained by the following expression (4).
GP12 = 1 × D12 + (1/2) × D13 +
(1/2) × D22 + (1/4) × D23,
GP14 = 1 × D14 + (1/2) × D15 +
(1/2) × D24 + (1/4) × D25,
...
GP21 = 1 × D21 + (1/2) × D22 +
(1/2) × D31 + (1/4) × D32,
GP23 = 1 × D23 + (1/2) × D24 +
(1/2) × D33 + (1/4) × D34,
(4)

  In this way, the combination of pixels to be added in the odd frame and the even frame is changed, and the pixel value overlap shift addition is performed. This superposition shift addition will be described in detail using the pixel P22 as an example.

  As shown in FIG. 2, when adding four pixels of GP11 in an odd frame, the pixel P11 is used as a reference pixel, and a weight 1/4 is given to the pixel P22 to perform addition. When adding the four pixels GP22, the pixel P22 is used as a reference pixel, and weight 1 is added to the pixel P22 to perform addition.

  Also, as shown in FIG. 3, when adding four pixels of GP12 in an even frame, the pixel P12 is used as a reference pixel, and weight 1/2 is added to the pixel P22 for addition. When adding the four pixels of GP21, the pixel P21 is used as a reference pixel, and the addition is performed with a weight 1/2 applied to the pixel P22.

  As described above, in the superposition shift addition, addition operations having different combinations are performed twice within one frame for a specific pixel common to two addition units. In addition, in the addition operation performed twice, the weight value for the specific pixel is changed. Also, in odd frames and even frames, the combination of pixels to be added is changed, and the weight value for a specific pixel is changed.

  Here, an odd frame is one of frames that appear alternately in successive moving image frames, and an even frame is the other. The frame is, for example, a timing at which an image is picked up by an image sensor or a timing at which one picked-up image is processed in image processing. Alternatively, one image in the image data is also referred to as a frame as appropriate.

3. Solid-State Imaging Device FIG. 4 shows a configuration example of the solid-state imaging device of the present embodiment that performs the above-described superposition shift addition. The solid-state imaging device includes a pixel array unit 100, a row scanning unit 110 (row scanning circuit), an A / D conversion unit 120 (A / D converter), a control unit 140, a writing control unit 141 (memory recording control unit), and readout. A control unit 142 (memory read control unit), a memory 150 (storage unit), a weight processing unit 160 (wait processing circuit), an addition unit 170 (addition circuit), and a data output control unit 180 are included.

  This configuration example is an example of a configuration as a CMOS image sensor equipped with a column parallel AD converter.

  Specifically, in the pixel array unit 100, unit pixels P11 to P88 including photoelectric conversion elements are two-dimensionally arranged in a matrix. The unit pixels P11 to P88 are composed of, for example, a photodiode or a phototransistor. The pixel array unit 100 is provided with a row selection line 101 corresponding to each row of unit pixels and a column signal line 102 corresponding to each column of unit pixels.

  The row scanning unit 110 performs vertical scanning. Specifically, the row scanning unit 110 performs scanning by selecting unit pixels arranged in the horizontal direction with the row selection line 101 and sequentially selecting the row selection line 101 along the vertical direction.

  A signal from each unit pixel in the selected row is input to the A / D converter 120 via the column signal line 102. The A / D conversion unit 120 performs column parallel A / D conversion. That is, the A / D conversion unit 120 includes a plurality of A / D conversion units AD1 to AD8 (A / D converter arrays) corresponding to the respective columns, and signals from unit pixels for one row are collectively A. / D conversion processing.

  The memory 150 (memory group) is a memory group including memory cells that store pixel data converted into digital data by the A / D conversion unit 120. Specifically, the memory 150 has a structure divided into a plurality of memory regions MR1 to MRn (n is a natural number).

  The write control unit 141 performs control to write pixel data into the memory 150. Specifically, the writing control unit 141 performs control to store each pixel data of the above-described addition unit in different memory areas. The write control unit 141 performs, for example, address control, data bus control, and write control as write control for the memory 150.

  The read control unit 142 performs control to read pixel data from the memory 150. Specifically, the read control unit 142 is configured to read in parallel to each memory area, and reads pixel data of addition units stored in different memory areas in parallel. The read control unit 142 performs, for example, address control, data bus control, and read control as read control from the memory 150.

  The control unit 140 performs overall control of the row scanning unit 110, the write control unit 141, and the read control unit 142.

  The weight processing unit 160 performs weight processing on the pixel data read from the memory 150. The adding unit 170 performs a process of adding the weighted pixel data, and outputs the added pixel data as an added pixel value. The data output control unit 180 performs control to output the added pixel value to the subsequent stage. Alternatively, the data output control unit 180 performs control to output pixel data from the read control unit 142 when pixel addition is not performed.

4). Detailed Configuration Example of Solid-State Imaging Device FIG. 5 shows a detailed configuration example of the solid-state imaging device of the present embodiment. Hereinafter, a case where the pixel array is 8 × 8 pixels and the superposition shift addition of 4 pixels described above with reference to FIGS. 1 to 3 is performed will be described as an example.

  The solid-state imaging device includes a pixel array unit 100, a row scanning unit 110, an A / D conversion unit 120, a control unit 140, a weight processing unit 160, an addition unit 170, a data output control unit 180, a line buffer selector 241, a data selector 242, First to third line buffers LB1 to LB3 are included. In the following, the same components as those described above with reference to FIG. 4 are denoted by the same reference numerals, and description thereof will be omitted as appropriate.

  The line buffers LB1 to LB3 constitute the memory 150 in FIG. Each line buffer is composed of two banks. A bank corresponds to the memory area of FIG. 4, and each bank is composed of four memory cells. Specifically, the line buffer LB1 includes memory cells m11 to m18. Of the memory cells m11 to m18, odd-numbered memory cells constitute a first bank BK1, and even-numbered memory cells constitute a second bank BK2. Similarly, the line buffers LB2 and LB3 are configured by memory cells m21 to m28 and m31 to m38, respectively. The third to sixth banks BK3 to BK6 are composed of odd and even columns of memory cells m21 to m28, and odd and even columns of memory cells m31 to m38, respectively.

  The line buffer selector 241 sequentially stores the pixel data of each row of the pixel array unit 100 in the line buffers LB1 to LB3. Specifically, the line buffer selector 241 stores the pixel data of the first row, the fourth row, and the seventh row in the line buffer LB1, and the pixel data of the second row, the fifth row, and the eighth row. The pixel data of the third row and the sixth row are stored in the line buffer LB3.

  The data selector 242 simultaneously accesses two line buffers among the line buffers LB1 to LB3. That is, the data selector 242 can access the four banks in parallel, and simultaneously reads out four pixel data from one memory cell in each bank. For example, the banks BK1 to BK6 are memory areas having independent address spaces, and a plurality of banks are accessed in parallel by designating addresses of the banks.

  The weight processing unit 160 includes first to fourth multiplication units MP1 to MP4. Multipliers MP1 to MP4 multiply the four pixel data read in parallel by weights 1, 1/2, 1/2, and 1/4, respectively.

  The data output control unit 180 communicates with the subsequent stage by, for example, LVDS (Low Voltage Differential Signaling), and transmits the data of the added pixel value to the subsequent stage.

5. Operation Next, the superimposed shift addition operation of the solid-state imaging device will be described with reference to FIGS. In the following, pixel data (pixel value) of the pixel Pij is assumed to be Dij.

  First, the write operation will be described. As shown in FIG. 6, the pixel data D11 to D18 of the pixels P11 to P18 in the first row are stored in the line buffer LB1. At this time, the data of the pixels arranged at adjacent positions in the pixel array are stored in different banks in one line buffer. Thereafter, the pixel data D21 to D28 of the pixels P21 to P28 in the second row are stored in the line buffer LB2, and the operations in which the pixel data D31 to D38 of the pixels P31 to P38 in the third row are stored in the line buffer LB3 are sequentially performed. Is called.

  For example, a writing operation of pixel data in the first row is described with reference to FIG. As shown in FIG. 7, for example, the data of the pixels P11 and P12 are stored in the same address “0” of the banks BK1 and BK2. That is, the memory cells m11 and m12 of the banks BK1 and BK2 are memory cells accessed by the same address. Similarly, the data of the pixels P13 and P14 are stored at the address “1”, the data of the pixels P15 and P16 are stored at the address “2”, and the data of the pixels P17 and P18 are stored at the address “3”.

  Next, the read operation and the weighted addition process will be described. In the present embodiment, there are three combinations of line buffers to be subjected to parallel reading corresponding to the addition unit of 2 × 2 pixels described in FIG. Specifically, pixel data in an addition unit composed of the first row and the second row, the fourth row and the fifth row, and the seventh row and the eighth row are read from the line buffers LB1 and LB2. Pixel data in the addition unit composed of the second row and the third row, and the fifth row and the sixth row are read from the line buffers LB2 and LB3. Pixel data in the addition unit composed of the third row and the fourth row, and the sixth row and the seventh row are read from the line buffers LB3 and LB1.

  Recording / reading (Write / Read) of pixel data to each bank in this combination of line buffers will be described with reference to FIG. In FIG. 8, “W” represents the pixel data recording state in each bank, and “R” represents the pixel data read state from each bank. As shown in FIG. 8, when reading from two line buffers is performed, pixel data is recorded in the remaining line buffers. For example, when pixel data is read from the banks BK1 to BK4 of the line buffers LB1 and LB2, the pixel data is written to the banks BK5 and BK6 of the line buffer LB3.

  FIG. 6 schematically shows an operation state when the pixel data of the first row and the second row are read from the line buffer. Hereinafter, the weighted addition process will be described using this case as an example.

  As shown in FIG. 6, the pixels P31 to P38 in the third row are selected by the row scanning unit 110, and the pixel data D31 to D38 are output by the A / D conversion unit 120. The line buffer selector 241 occupies the banks BK5 and BK6 of the line buffer LB3, and performs an operation of writing the pixel data D31 to D38 to the banks BK5 and BK6.

  The bank data BK1 and D18 of the first row are stored in the banks BK1 and BK2 of the line buffer LB1, and the pixel data D21 to D28 of the second row are stored in the banks BK3 and BK4 of the line buffer LB2. It is in a state. The data selector 242 sequentially reads pixel data of the groups GP11, GP13, GP15, and GP17 from the banks BK1 to BK4. The data selector 242 outputs the read four pixel data to the multiplication units MP1 to MP4 of the weight processing unit 160 in accordance with the combination of weight calculations.

  For example, taking the group GP11 as an example, the data selector 242 simultaneously reads out the pixel data stored in the address “0” of the banks BK1 to BK4. The data selector 242 outputs the pixel data D11, D12, D21, and D22 to the corresponding weight multiplication units MP1, MP2, MP3, and MP4, respectively.

  The weight processing unit 160 performs a predetermined weight calculation process on each input pixel data. Specifically, the multiplication unit MP1 having the weight “1” uses the input pixel data D11 as an output value as it is. The weights “1/2” multiplication units MP2 and MP3 shift the input pixel data values D12 and D21 to the right by 1 bit to obtain output values. The multiplication unit MP4 having the weight “1/4” shifts the input pixel data value D22 to the right by 2 bits to obtain an output value. The weight processing unit 160 outputs each pixel data value subjected to the weight calculation to the subsequent addition unit 170.

  The addition unit 170 performs an addition operation of the four pixel data subjected to the weight operation. The result obtained by the addition calculation is a calculation value obtained by performing the weight calculation four-pixel addition on the unit pixels P11, P12, P21, and P22, and the calculation value is output to the data output control unit 180 at the subsequent stage. The above weight calculation and addition calculation processes are sequentially performed, and the calculations of the groups GP11, GP13, GP15, and GP17 are performed.

  The data output control unit 180 outputs the addition pixel data sequentially input from the addition unit 170 to the subsequent stage through an interface corresponding to an interface (not shown) of the subsequent process. For example, the data output control unit 180 sets the sequentially added pixel data as a set of 8 pixel data, converts the data into a LVDS data structure, and outputs the data.

  Next, the operation timing of the above operation will be described with reference to FIG. FIG. 9 shows a timing chart in the pixel data readout period from the first row to the fourth row of the pixel array unit 100.

  As shown by A1 in FIG. 9, the pixel data of the first row is output from the A / D conversion unit 120 in the scanning period of the first row. As shown in A2, the access state of the line buffer LB1 is set to the write state, and the pixel data of the first row is sequentially stored in the addresses “00” to “03” of the banks BK1 and BK2 under the control of the line buffer selector 241. Is done. At this time, since two columns are written for one address, the write operation may be performed once in two columns, and the number of write operations may be ½ compared to the case of all pixel readout.

  Next, as shown in A3, the pixel data of the second row is output in the scanning period of the second row, and as shown in A4, the access state of the line buffer LB2 is set to the write state, and the pixel data of the second row is set. Are stored in the banks BK3 and BK4.

  Next, as shown in A5, the line buffers LB1 and LB2 are set in the reading state in the scanning period of the third row, and the pixel data of the first row and the second row are transferred from the banks BK1 to BK4 under the control of the data selector 242. Read out. At this time, four pixel data of the addition unit is simultaneously read for one address, and the four addition units are sequentially read by designating the addresses “00” to “03”. Similar to the writing operation, the number of reading operations may be ½ compared to the case of all-pixel reading. Further, as shown in A6, the pixel data of the third row is output in the scanning period of the third row, and as shown in A7, the access state of the line buffer LB3 is set to the write state, and the pixel data of the third row is changed. Stored in the banks BK5 and BK6.

  Next, as shown at A8, the four-pixel data read from the banks BK1 to BK4 is input from the data selector 242 to the weight processing unit 160. The weight processing unit 160 performs weight calculations of x1, x1 / 2, x1 / 2, and x1 / 4 on the data of the four pixels. As indicated by A9, the result of the weight calculation is input to the adding unit 170, the addition calculation is performed, and the result is output as the added pixel value. These processes are performed four times, which is the number of addition units for one row, in the scanning period of the third row.

  In this embodiment, it is possible to perform conventional all-pixel readout instead of the above-described superposition shift addition. When the solid-state imaging device of this embodiment performs pixel readout, the data selector 242 performs an operation corresponding to a conventional column scanning circuit. That is, the data selector 242 sequentially reads out pixel data for one row from two banks of the line buffer one by one, and outputs the pixel data to the data output control unit 180.

  Here, in the above embodiment, the case of the addition processing using the addition unit and weight shown in FIGS. 1 to 3 has been described as an example, but the present embodiment is not limited to this. For example, it is possible to cope with other addition combinations by changing the structure of the line buffer and its internal bank and changing the control method of the write control unit and the read control unit.

  In the above embodiment, the case where the weight shown in FIG. 1 is realized by a simple bit shift operation of digital data has been described as an example. However, the present embodiment is not limited to this. For example, the weight processing for assigning other weights may be performed by setting the weight processing unit to a multiplier / divider configuration or a lookup table configuration.

  According to the above embodiment, as shown in FIG. 5, the solid-state imaging device includes the pixel array unit 100, the row scanning unit 110, the A / D conversion unit 120, the first to nth memory regions MR1 to MRn, and the write control. Unit 141, read control unit 142, and weighting addition unit (weight processing unit 160, addition unit 170).

  In the pixel array unit 100, a plurality of pixels P11 to P88 that photoelectrically convert incident light are arranged. The row scanning unit 110 selects a row of the pixel array unit 100 and performs vertical scanning. The A / D converter 120 converts the analog signal from the pixel in the selected row into a pixel value (pixel data) of a digital signal. The first to nth memory areas MR1 to MRn store the pixel values. The write control unit 141 performs control to write pixel values to the first to nth memory regions MR1 to MRn. The read control unit 142 performs control to read pixel values included in an addition unit (a group of pixels) that is a range of pixels to be added from the first to nth memory regions MR1 to MRn. The weighted addition unit performs weighted addition of the pixel values included in the read addition unit, and outputs the added pixel value as the added pixel value.

  In this case, the writing control unit 141 stores each pixel value included in the addition unit in a different memory area among the first to nth memory areas MR1 to MRn. The read control unit 142 reads out the pixel value included in the first addition unit (eg, GP11 in FIG. 2), and the second addition unit (GP22) including the pixel (P22) common to the first addition unit. The pixel value included in is controlled to be read.

  In this way, pixel addition can be performed with the addition units superimposed. That is, the pixel values of the first and second addition units having a common pixel can be read from the memory areas MR1 to MRn, and the pixel values can be added. Further, the weighted addition unit can perform weighted addition by weighting the read pixel values.

  In the present embodiment, as described with reference to FIG. 7 and the like, the first to nth memory areas MR1 to MRn (banks BK1 to BK6) are memory areas having independent address spaces. The read control unit 142 reads pixel values included in the addition unit in parallel from the first to nth memory regions MR1 to MRn.

  In this way, compared with a conventional solid-state imaging device that does not perform superposition shift addition, superimposition readout is performed without increasing the processing speed (for example, clock frequency) of each component (for example, A / D conversion unit, memory, etc.). In addition, it is possible to perform weight calculation and addition calculation. For example, assume that the pixel values of the addition units GP11 and GP22 shown in FIG. In this case, since it is necessary to read the superimposed pixel P22 twice, it is necessary to increase the reading processing speed compared to the case where the superimposed pixel P22 is not superimposed. In this respect, according to the present embodiment, pixel values in increments are read out in parallel from the memory regions MR1 to MRn, so that the number of readouts can be reduced and superposition shift addition can be performed without increasing the processing speed.

  In addition, the parallel readout can shorten the processing time in each row as compared with the case of all-pixel readout, so that a high frame rate can be achieved. For example, when the addition unit is 2 × 2 pixels, it is only necessary to perform the readout operation once in two columns in each row, so that the readout processing time can be halved compared to the readout of all pixels. If the weighted addition unit (the weight processing unit 160 or the addition unit 170) is processed in parallel, the processing speed can be further reduced.

  Further, since there is no need to increase the processing speed, line buffer access control can be performed by a control system synchronized with the horizontal synchronizing signal (HS) and the vertical synchronizing signal (VS) as described with reference to FIG. Thereby, it is possible to cope with a complicated addition calculation process such as superposition shift addition without constructing a complicated control system.

  In this embodiment, as described in the above formulas (3) and (4), the weighted addition unit includes the common pixel (P22) in the first addition unit (GP11) and the second addition unit (GP22). ) Is weighted with different weighting factors (1/4, 1) for the common pixel (P22).

  In this way, in the addition operation of the first and second addition units, different weights can be applied to the pixels to be superimposed, and weighted addition in the superimposed shift addition can be realized.

  In the present embodiment, the first addition unit (GP11) is composed of pixels in 2 rows and 2 columns. In this case, as shown in FIG. 6, the first and second memory areas (banks BK1, BK2) constitute a first line buffer LB1, and each of the odd-numbered columns in the first row of the pixel array unit 100, Store the pixel values of even columns. The third and fourth memory areas (banks BK3 and BK4) constitute the second line buffer LB2, and store the pixel values of the odd and even columns in the second row of the pixel array unit 100, respectively. The read control unit (data selector 242) reads out the pixel values of 2 rows and 2 columns constituting the first addition unit (GP11) from the first and second line buffers LB1 and LB2.

  In the present embodiment, the second addition unit (GP22) is an addition unit of 2 rows and 2 columns obtained by shifting the first addition unit (GP11) by 1 row and 1 column. In this case, the fifth and sixth memory areas (banks BK5 and BK6) constitute the third line buffer LB3, and store the pixel values of the odd and even columns in the third row of the pixel array unit 100, respectively. To do. The read control unit (data selector 242) reads the pixel values of 2 rows and 2 columns constituting the second addition unit (GP22) from the second and third line buffers (LB2, LB3).

  In this way, a memory is constituted by three line buffers each having two banks, and the first to sixth memory regions MR1 to MR6 (n = 6) can be constituted by the six banks. In addition, in the first and second addition units, the overlap shift addition can be performed by reading the pixel values redundantly from the second line buffer LB2.

  Further, in the present embodiment, the third addition unit (eg, GP12 in FIG. 3) is set as a 2 × 2 addition unit obtained by shifting the first addition unit (GP11) by one column, and the fourth addition unit (GP21 ) Is an addition unit of 2 rows and 2 columns obtained by shifting the first addition unit (GP11) by 1 row. In this case, the readout control unit (data selector 242) reads out the pixel values of the first and second addition units (GP11, GP22) in the first frame (odd number frame), and next to the first frame. In the second frame (even frame), the pixel values of the third and fourth addition units (GP12, GP21) are read out.

  By doing this, it is possible to obtain the addition pixel values of the four addition units (GP11, GP12, GP21, GP22) shifted in superposition. As a result, it is possible to apply a restoration estimation process to be described later, and a high-resolution image can be obtained by a simple restoration estimation process.

  In the present embodiment, the A / D conversion unit 120 is a column parallel A / D conversion unit that converts the analog signals of the respective columns in the row selected by the row scanning unit 110 into digital signals in parallel.

  In this way, analog signals from one row of pixels can be converted into digital signals in parallel. This eliminates the need for high-speed A / D conversion, so that the operation speed of the solid-state imaging device can be reduced.

6). Restoration Estimation Processing Next, processing for restoring a high-resolution image by estimation from the addition pixel value acquired by superposition shift addition will be described in detail. In the following, the addition pixel values {a ₀₀ , a ₁₀ , a ₁₁ , a ₀₁ } will be described as an example (i and j are integers of 0 or more), but the same applies to other addition pixel values. Further, the case where the addition unit is set for every 2 × 2 pixels will be described as an example, but the present invention is not limited to this, and may be, for example, every 3 × 3 pixels.

10A and 10B are explanatory diagrams of the estimated pixel value and the intermediate pixel value. The added pixel values {a ₀₀ , a ₁₁ } shown in FIG. 10A correspond to the added pixel values of the groups GP11 and GP22 described with reference to FIG. The addition pixel values {a ₁₀ , a ₀₁ } correspond to the addition pixel values of the groups GP12 and GP21 described with reference to FIG. In the estimation process, final estimated pixel values v _{00 to} v ₂₂ are estimated using the added pixel values. The estimated pixel value v _ij corresponds to the pixel value Dij of the pixel Pij described in FIG.

As shown in FIG. 10B, first, intermediate pixel values b _{00 to} b ₂₁ (intermediate estimated pixel values) are estimated from the added pixel values a _{00 to} a ₁₁ . Intermediate pixel value corresponds to 2-pixel sum values, for example, _{b 00} corresponds to the sum of the pixel values _{v 00} and _{v 01.} The final pixel values v _{00 to} v ₂₂ are estimated from these intermediate pixel values b _{00 to} b ₂₁ .

First, a process for estimating the intermediate pixel value will be described. Hereinafter, a case where the intermediate pixel values b _{00 to} b ₂₀ of the first row in the horizontal direction are estimated will be described as an example. The intermediate pixel values b _{01 to} b _{21 in the} next row are estimated by the same method.

As shown in FIG. 11, the intermediate pixel values b _{00 to} b ₂₀ are estimated based on the added pixel values a ₀₀ and a ₁₀ in the first row in the horizontal direction. For the sake of simplicity, for example, assuming that the weighting factor r = 2, the addition pixel values a ₀₀ and a ₁₀ are expressed by the following expression (5).
a ₀₀ = v ₀₀ + (1/2) v ₀₁ + (1/2) v ₁₀ + (1/4) v ₁₁ ,
_{a 10 = v 10 + (1/2} ) v 11 + (1/2) v 20 + (1/4) v 21 (5)

B ₀₀ , b ₁₀ , and b ₂₀ are defined as shown in the following formula (6).
b ₀₀ = v ₀₀ + (1 / r) v ₀₁ = v ₀₀ + (1/2) v ₀₁ ,
b ₁₀ = v ₁₀ + (1 / r) v ₁₁ = v ₁₀ + (1/2) v ₁₁ ,
b ₂₀ = v ₂₀ + (1 / r) v ₂₁ = v ₂₀ + (1/2) v ₂₁ (6)

Next, when the above equation (5) is transformed using the above equation (6), the following equation (7) is established.
a ₀₀ = b ₀₀ + (1/2) b ₁₀ ,
a ₁₀ = b ₁₀ + (1/2) b ₂₀ (7)

In the above equation (7), a difference δi ₀ is obtained by multiplying a ₀₀ and a ₁₀ by a predetermined weighting factor, and the following equation (8) is established.
δi ₀ = a ₁₀ -2a ₀₀
= (1/2) b ₂₀ -2b ₀₀ (8)

If b ₀₀ is an unknown (initial variable), intermediate pixel values b ₁₀ and b ₂₀ can be obtained as a function of b ₀₀ as shown in the following equation (9). In this way, a high-definition combination pattern of intermediate pixel values {b ₀₀ , b ₁₀ , b ₂₀ } is obtained with b ₀₀ as an unknown.
b ₀₀ = (unknown number),
b ₁₀ = 2 (a ₀₀ −b ₀₀ ),
b ₂₀ = 4b ₀₀ + 2δi ₀ = 4b ₀₀ +2 (a ₁₀ −2a ₀₀ ) (9)

Next, a description will be given of a method of obtaining the unknown _{b 00.} As shown in FIG. 12, the addition pixel value pattern {a ₀₀ , a ₁₀ } and the intermediate pixel value pattern {b ₀₀ , b ₁₀ , b ₂₀ } are compared. Then, an unknown number b ₀₀ that minimizes the error E is derived and set as the intermediate pixel value b ₀₀ .

Specifically, the relationship of the following formula (10) is established between the added pixel value {a _ij } and the intermediate pixel value {b _ij , b _{(i + 1) j} }. Considering the weighting by the following equation (10), the evaluation function Ej shown in the following equation (11) is obtained. Then, with this evaluation function Ej, similarity evaluation between the pattern {a ₀₀ , a ₁₀ } and the pattern {b ₀₀ , b ₁₀ , b ₂₀ } is performed.
a _ij = b _ij + (1/2) b _{(i + 1) j} (10)

As shown in FIG. 13, an unknown b ₀₀ (= α) that minimizes Ej is obtained, and the value of b ₀₀ can be determined. Then, the estimated value of b ₀₀ is substituted into the above equation (9) to obtain b ₁₀ and b ₂₀ .

Next, a method for _{obtaining the} final estimated pixel value v _ij using the obtained intermediate pixel value b _ij will be described. In the following, description will be given taking the leftmost vertical column (i = 0 column) as an example. The final estimated pixel value v _ij is obtained in the same manner as the method for _obtaining the intermediate pixel value b _ij . That is, if the above equation (7) is replaced with the following equation (12), the subsequent processing is the same.
b ₀₀ = v ₀₀ + (1/2) v ₀₁ ,
b ₀₁ = v ₀₁ + (1/2) v ₀₂ (12)

7). Imaging Device FIG. 14 shows a configuration example of an imaging device that performs the above-described superposition shift sampling and restoration processing. The imaging device includes an imaging optical system 300 (lens), an optical low-pass filter 310, a solid-state imaging device 320, a display processing unit 330, a monitor display unit 340, a data recording unit 350 (storage unit), and a pixel value estimation calculation unit 360 (estimation calculation). Part) and an image output part 370.

  Note that the imaging apparatus according to the present embodiment is not limited to this configuration, and various modifications such as omitting some of the components or adding other components are possible. For example, the pixel value estimation calculation unit 360 and the image output unit 370 may be configured by an image processing device (for example, a PC) outside the imaging device.

  The imaging optical system 300 forms an image of a subject. The optical low-pass filter 310 passes a band corresponding to the resolution of the solid-state imaging device 320, for example. The solid-state imaging device 320 (for example, 12 megapixels) is configured by, for example, a CCD or a CMOS sensor. The solid-state imaging device 320 controls setting of addition units and addition reading, and acquires a fusion frame. A fusion frame is an image obtained by superposition shift sampling. The data recording unit 350 is realized by a memory card or the like, for example, and records a moving image using a fusion frame. The monitor display unit 340 displays a live view of a moving image and a reproduced moving image.

  The pixel value estimation calculation unit 360 estimates a final estimated pixel value. The image output unit 370 outputs a still image or a moving image based on the final estimated pixel value. The image output unit 370 includes anti-aliasing filters 371 and 374, a low-pass filter 372, and an undersampling unit 373.

  The anti-aliasing filter 371 performs anti-aliasing processing on the final estimated pixel value and outputs a high-resolution still image (for example, 12 megapixels). The low-pass filter 372 limits the final estimated pixel value to the high vision band. The undersampling unit 373 undersamples the band-limited final estimated pixel value to the number of high-definition pixels. The anti-aliasing filter 374 performs anti-aliasing processing on the undersampled image and outputs a high-definition moving image (for example, 2 megapixels). Note that a high-resolution moving image (for example, 12 megapixels) may be output without undersampling.

According to the above embodiment, as illustrated in FIG. 14, the imaging device (imaging system, for example, a digital camera) includes the solid-state imaging device 320 and the estimation calculation unit (pixel value estimation calculation unit 360). As described with reference to FIG. 10A, the estimation calculation unit, based on the addition pixel values {a ₀₀ , a ₁₀ , a ₁₁ , a ₀₁ }, the pixel values {v ₀₀ , v of pixels included in the addition unit. ₁₀ , v ₁₁ , v ₀₁ }.

As shown in FIG. 10A, the first position addition unit (for example, a ₀₀ ) set to the first position and the second position addition set to the second position where the first position is shifted. Units (eg, a ₁₀ ) overlap.

In this case, the weighting addition unit (the weight processing unit 160 and the addition unit 170 in FIG. 4) performs the first and second addition pixel values (a ₀₀ , a which are addition pixel values in the first and second position addition units). ₁₀ ) is output. As shown in the above equation (8), the estimation calculation unit obtains a difference value δi ₀ between the first and second addition pixel values a ₀₀ and a ₁₀ . As shown in FIG. 10B, the first intermediate pixel value b ₀₀ is an addition of the first area (v ₀₀ , v ₀₁ ) obtained by removing the overlapping area (v ₁₀ , v ₁₁ ) from the addition unit a _00. It is a pixel value. The second intermediate pixel value b ₂₀ is an addition pixel value of the second region (v ₂₀ , v ₂₁ ) obtained by removing the overlap region (v ₁₀ , v ₁₁ ) from the addition unit a ₁₀ . As shown in the above equation (9), a relational expression between the first and second intermediate pixel values b ₀₀ and b ₂₀ is expressed using a difference value δi ₀ . As shown in FIG. 12 and the like, the first and second intermediate pixel values b ₀₀ and b ₂₀ are estimated using the relational expression. Using the estimated first intermediate pixel value b ₀₀ , pixel values {v ₀₀ , v ₁₀ , v ₁₁ , v ₀₁ } of each pixel included in the addition unit are obtained.

  In this way, it is possible to simplify the estimation process of the high-resolution image by once estimating the intermediate pixel value from the addition pixel value subjected to the superposition shift and obtaining the estimation pixel value from the intermediate pixel value subjected to the superposition shift. . For example, complicated processing such as a repetitive calculation of a two-dimensional filter (Japanese Patent Laid-Open No. 2009-124621) and a search for a part suitable for setting an initial value (Japanese Patent Laid-Open No. 2008-243037) are not required.

Here, superimposing means having an area where the addition unit and the addition unit overlap. For example, as shown in FIG. 10A, the addition unit a ₀₀ and the addition unit a ₁₀ are two estimated pixels v _10. is to share the _{v 11.}

  Further, the position of the addition unit is the position and coordinates of the addition unit in the captured image, or the position and coordinates of the addition unit on the estimated pixel value data (image data) in the estimation process. The shifted position is a position shifted from the original position by a pixel, and is a position where the position and coordinates do not coincide with the original position.

In this embodiment, continuous intermediate pixel values including the first and second intermediate pixel values (for example, b ₀₀ , b ₂₀ ) are set as intermediate pixel value patterns {b ₀₀ , b ₁₀ , b ₂₀ }. As shown in the above equation (9), the estimation calculation unit represents the relational expression between the intermediate pixel values included in the intermediate pixel value pattern using the added pixel values a ₀₀ and a ₁₀ . As shown in FIG. 12, the similarity is evaluated by comparing the intermediate pixel value pattern represented by the relational expression between the intermediate pixel values and the added pixel value. Based on the similarity evaluation result, the intermediate pixel values b ₀₀ , b ₁₀ , and b ₂₀ included in the intermediate pixel value pattern are determined so that the similarity is the highest.

  In this way, the intermediate pixel value can be estimated based on a plurality of added pixel values acquired by the addition unit that is pixel-shifted while being superimposed.

  Here, the intermediate pixel value pattern is a data string (a set of data) of intermediate pixel values in a range used for estimation processing. The addition pixel value pattern is a data string of addition pixel values in a range used for the estimation process.

Further, in the present embodiment, as shown in the above equation (11), the estimation calculation unit includes the intermediate pixel value pattern {b ₀₀ , b ₁₀ , b ₂₀ } represented by the relational expression between the intermediate pixel values and the addition pixel. An evaluation function Ej representing an error from the values {a ₀₀ , a ₁₀ } is obtained. The intermediate pixel values b ₀₀ , b ₁₀ and b ₂₀ included in the intermediate pixel value pattern are determined so that the value of the evaluation function Ej is minimized.

  In this way, the value of the intermediate pixel value can be estimated by expressing the error by the evaluation function and obtaining the intermediate pixel value corresponding to the minimum value of the evaluation function. For example, as described above, the initial value of the intermediate pixel estimation can be set with a simple process by obtaining the unknown using the least square method. For example, searching for an image portion suitable for initial value setting (Japanese Patent Laid-Open No. 2008-243037) is unnecessary.

In the present embodiment, as shown in the above equation (5), an addition pixel value (a ₀₀ ) obtained by weighting and adding each pixel value (for example, v ₀₀ , v ₁₀ , v ₀₁ , v ₁₁ ) of the addition unit is used. get. Based on the obtained addition pixel value (a ₀₀ , a ₁₀ ) of the addition unit, the pixel value (v ₀₀ , v ₁₀ , v ₀₁ , v ₁₁ ) of each pixel of the addition unit is estimated.

  If it does in this way, each pixel value of an addition unit will carry out weighted addition, an addition image will be acquired, and the pixel value of a high-resolution image can be estimated from the acquired addition image. Thereby, in the estimation process, the reproducibility of the high-frequency component of the subject can be improved. That is, when the pixel values of the addition unit are simply added, a rectangular window function is convoluted for imaging. On the other hand, when the pixel value of the addition unit is weighted and added, a window function containing more high frequency components than the rectangle is convoluted for imaging. Therefore, it is possible to acquire an added image that includes more high-frequency components of the subject, and to improve the reproducibility of the high-frequency components in the estimated image.

  Although the present embodiment has been described in detail as described above, it will be easily understood by those skilled in the art that many modifications can be made without departing from the novel matters and effects of the present invention. Accordingly, all such modifications are intended to be included in the scope of the present invention. For example, a term described with a different term having a broader meaning or the same meaning at least once in the specification or the drawings can be replaced with the different term in any part of the specification or the drawings. In addition, the configuration and operation of the solid-state imaging device, the imaging device, and the like are not limited to those described in the present embodiment, and various modifications can be made.

100 pixel array section, 101 row selection line, 102 column signal line,
110 line scanning unit, 120 A / D conversion unit, 140 control unit,
141 write control unit, 142 read control unit, 150 memory,
160 weight processing unit, 170 addition unit, 180 data output control unit,
241 Line buffer selector, 242 Data selector,
300 imaging optical system, 310 optical low-pass filter,
320 solid-state imaging device, 330 display processing unit, 340 monitor display unit,
350 data recording unit, 360 pixel value estimation calculation unit, 370 image output unit,
371 Anti-aliasing filter, 372 Low-pass filter,
373 Undersampling unit, 374 Anti-aliasing filter,
AD1 to AD8 A / D conversion units, BK1 to BK6, first to sixth banks,
D11 to D38 pixel data, Ej evaluation function, GP11 addition unit,
LB1 to LB3 first to third line buffers,
MP1 to MP4, first to fourth multiplication units,
MR1 to MRn 1st to nth memory areas, P11 to P88 pixels,
P22 common pixel, a _ij addition pixel value, b _ij intermediate pixel value,
m11 to m38 memory cell, v _ij estimated pixel value, δi ₀ difference value

Claims (9)

A pixel array unit in which a plurality of pixels that photoelectrically convert incident light are arranged;
A row scanning unit that selects a row of the pixel array unit and performs vertical scanning;
An A / D converter that converts an analog signal from the pixel in the selected row into a pixel value of a digital signal;
First to nth memory areas for storing the pixel values;
A write control unit that performs control to write the pixel values to the first to nth memory areas;
A read control unit that performs control to read out pixel values included in an addition unit that is a range of pixels to be added from the first to nth memory areas;
A weighted addition unit that performs weighted addition of pixel values included in the read addition unit and outputs the pixel value after addition as an added pixel value;
Including
The write control unit
Each pixel value included in the addition unit is stored in a different memory area among the first to nth memory areas,
The read control unit
A control to read the pixel values included in the first addition unit, have row control for reading the pixel values included in the second summing unit including a common pixel and the first summation unit,
When the first addition unit is composed of pixels of 2 rows and 2 columns,
The first and second memory areas of the first to nth memory areas are:
Forming a first line buffer, storing pixel values of odd columns and even columns of the first row of the pixel array unit, respectively;
The third and fourth memory areas of the first to nth memory areas are:
Configuring a second line buffer, storing pixel values of odd columns and even columns of the second row of the pixel array unit, respectively;
The read control unit
2. A solid-state imaging device , wherein pixel values of 2 rows and 2 columns constituting the first addition unit are read from the first and second line buffers . In claim 1,
The first to nth memory areas are
Each memory area has an independent address space,
The read control unit
A solid-state imaging device, wherein pixel values included in the addition unit are read in parallel from the first to nth memory areas. In claim 1 or 2,
The weighted addition unit includes:
A solid-state imaging device, wherein the common pixel in the first addition unit and the common pixel in the second addition unit are weighted with different weighting factors. In claim 1 ,
When the second addition unit is an addition unit of 2 rows and 2 columns obtained by shifting the first addition unit by 1 row and 1 column,
The fifth and sixth memory areas of the first to nth memory areas are:
Configuring a third line buffer, storing pixel values of odd columns and even columns of the third row of the pixel array unit, respectively;
The read control unit
2. A solid-state imaging device, wherein pixel values of 2 rows and 2 columns constituting the second addition unit are read from the second and third line buffers. In claim 4 ,
The third addition unit is an addition unit of 2 rows and 2 columns obtained by shifting the first addition unit by 1 column, and the fourth addition unit is 2 rows 2 obtained by shifting the first addition unit by 1 row. If the column addition unit is used,
The read control unit
Reading out the pixel values of the first and second addition units in the first frame, and reading out the pixel values of the third and fourth addition units in the second frame next to the first frame. A solid-state imaging device. In any one of Claims 1 thru | or 5 ,
The A / D converter is
A solid-state imaging device, comprising: a column parallel A / D conversion unit that converts the analog signal of each column of a row selected by the row scanning unit into the digital signal in parallel. A solid-state imaging device according to any one of claims 1 to 6 ;
An estimation calculation unit that estimates a pixel value included in the addition unit based on the addition pixel value;
Including
When the first position addition unit set in the first position and the second position addition unit set in the second position shifted from the first position overlap,
The weighted addition unit includes:
Outputting first and second addition pixel values which are the addition pixel values of the first and second position addition units;
The estimation calculation unit includes:
A difference value between the first addition pixel value and the second addition pixel value is obtained;
A first intermediate pixel value, which is an added pixel value of the first area obtained by removing the overlapping area from the first position addition unit, and an added pixel of the second area obtained by removing the overlapping area from the second addition unit. A relational expression with the second intermediate pixel value that is a value is expressed using the difference value,
The first and second intermediate pixel values are estimated using the relational expression, and the pixel value of each pixel included in the addition unit is obtained using the estimated first intermediate pixel value. Imaging device. In claim 7 ,
The estimation calculation unit includes:
When successive intermediate pixel values including the first and second intermediate pixel values are used as an intermediate pixel value pattern, a relational expression between intermediate pixel values included in the intermediate pixel value pattern is obtained using the added pixel value. Represent,
Comparing the intermediate pixel value pattern represented by the relational expression between the intermediate pixel values and the added pixel value to evaluate similarity;
An image pickup apparatus, comprising: determining an intermediate pixel value included in the intermediate pixel value pattern based on the similarity evaluation result so that the similarity becomes the highest. A pixel array unit in which a plurality of pixels that photoelectrically convert incident light are arranged;
A row scanning unit that selects a row of the pixel array unit and performs vertical scanning;
An A / D converter that converts an analog signal from the pixel in the selected row into a pixel value of a digital signal;
First to nth memory areas for storing the pixel values;
A write control unit that performs control to write the pixel values to the first to nth memory areas;
A read control unit that performs control to read out pixel values included in an addition unit that is a range of pixels to be added from the first to nth memory areas;
A weighted addition unit that performs weighted addition of pixel values included in the read addition unit and outputs the pixel value after addition as an added pixel value;
An estimation calculation unit that estimates a pixel value included in the addition unit based on the addition pixel value;
Including
The write control unit
Each pixel value included in the addition unit is stored in a different memory area among the first to nth memory areas,
The read control unit
A control to read the pixel values included in the first addition unit, have row control for reading the pixel values included in the second summing unit including a common pixel and the first summation unit,
When the first position addition unit set in the first position and the second position addition unit set in the second position shifted from the first position overlap,
The weighted addition unit includes:
Outputting first and second addition pixel values which are the addition pixel values of the first and second position addition units;
The estimation calculation unit includes:
A difference value between the first addition pixel value and the second addition pixel value is obtained;
A first intermediate pixel value, which is an added pixel value of the first area obtained by removing the overlapping area from the first position addition unit, and an added pixel of the second area obtained by removing the overlapping area from the second addition unit. A relational expression with the second intermediate pixel value that is a value is expressed using the difference value,
The first and second intermediate pixel values are estimated using the relational expression, and the pixel value of each pixel included in the addition unit is obtained using the estimated first intermediate pixel value. Imaging device.

Images